autopush.db

Database Interaction

WebPush Sort Keys

Messages for WebPush are stored using a partition key + sort key, originally the sort key was:

CHID : Encrypted(UAID: CHID)

The encrypted portion was returned as the Location to the Application Server. Decrypting it resulted in enough information to create the sort key so that the message could be deleted and located again.

For WebPush Topic messages, a new scheme was needed since the only way to locate the prior message is the UAID + CHID + Topic. Using Encryption in the sort key is therefore not useful since it would change every update.

The sort key scheme for WebPush messages is:

VERSION : CHID : TOPIC

To ensure updated messages are not deleted, each message will still have an update-id key/value in its item.

Non-versioned messages are assumed to be original messages from before this scheme was adopted.

VERSION is a 2-digit 0-padded number, starting at 01 for Topic messages.

DynamoDB Table Functions

autopush.db.create_router_table(tablename='router', read_throughput=5, write_throughput=5)[source]

Create a new router table

The last_connect index is a value used to determine the last month a user was seen in. To prevent hot-keys on this table during month switchovers the key is determined based on the following scheme:

(YEAR)(MONTH)(DAY)(HOUR)(0001-0010)

Note that the random key is only between 1-10 at the moment, if the key is still too hot during production the random range can be increased at the cost of additional queries during GC to locate expired users.

autopush.db.create_storage_table(tablename='storage', read_throughput=5, write_throughput=5)[source]

Create a new storage table for simplepush style notification storage

autopush.db.get_router_table(tablename='router', read_throughput=5, write_throughput=5)[source]

Get the main router table object

Creates the table if it doesn’t already exist, otherwise returns the existing table.

autopush.db.get_storage_table(tablename='storage', read_throughput=5, write_throughput=5)[source]

Get the main storage table object

Creates the table if it doesn’t already exist, otherwise returns the existing table.

Utility Functions

autopush.db.preflight_check(storage, router, uaid='deadbeef00000000deadbeef00000000')[source]

Performs a pre-flight check of the storage/router/message to ensure appropriate permissions for operation.

Failure to run correctly will raise an exception.

DynamoDB Table Class Abstractions

class autopush.db.Storage(table, metrics)[source]

Create a Storage table abstraction on top of a DynamoDB Table object

__init__(table, metrics)[source]

Create a new Storage object

Parameters:
fetch_notifications(*args, **kwargs)[source]

Fetch all notifications for a UAID

Raises:ProvisionedThroughputExceededException if dynamodb table exceeds throughput.
save_notification(*args, **kwargs)[source]

Save a notification for the UAID

Raises:ProvisionedThroughputExceededException if dynamodb table exceeds throughput.
delete_notification(uaid, chid, version=None)[source]

Delete a notification for a UAID

Returns:Whether or not the notification was able to be deleted.
class autopush.db.Router(table, metrics)[source]

Create a Router table abstraction on top of a DynamoDB Table object

__init__(table, metrics)[source]

Create a new Router object

Parameters:
get_uaid(uaid)[source]

Get the database record for the UAID

Raises:ItemNotFound if there is no record for this UAID. ProvisionedThroughputExceededException if dynamodb table exceeds throughput.
register_user(*args, **kwargs)[source]

Register this user

If a record exists with a newer connected_at, then the user will not be registered.

Returns:Whether the user was registered or not.
Raises:ProvisionedThroughputExceededException if dynamodb table exceeds throughput.
drop_user(*args, **kwargs)[source]

Drops a user record

delete_uaids(uaids)[source]

Issue a batch delete call for the given uaids

drop_old_users(months_ago=2)[source]

Drops user records that have no recent connection

Utilizes the last_connect index to locate users that haven’t connected in the given time-frame.

The caller must iterate through this generator to trigger batch delete calls. Caller should wait as appropriate to avoid exceeding table limits.

Each iteration will result in a batch delete for the currently iterated batch. This implies a set of writes equal in size to the 25 * record-size minimum.

Warning

Calling list() on this generator will likely exceed provisioned write through-put as the batch-delete calls will be made as quickly as possible.

Parameters:months_ago – how many months ago since the last connect
Returns:Iterable of how many deletes were run
update_message_month(*args, **kwargs)[source]

Update the route tables current_message_month

Note that we also update the last_connect at this point since webpush users when connecting will always call this once that month. The current_timestamp is also reset as a new month has no last read timestamp.

clear_node(*args, **kwargs)[source]

Given a router item and remove the node_id

The node_id will only be cleared if the connected_at matches up with the item’s connected_at.

Returns:Whether the node was cleared or not.
Raises:ProvisionedThroughputExceededException if dynamodb table exceeds throughput.