Amazon DynamoDB | Time to Live (TTL)

Question 1

What happens if I add a tag with a key that is same as one for an existing tag?

Time to Live (TTL)

Amazon DynamoDB | Database

Answer 1

Each DynamoDB table can only have up to one tag with the same key. If you add a tag with the same key as an existing tag, the existing tag is updated with the new value.

Question 2

What is DynamoDB Time-to-Live (TTL)?

Time to Live (TTL)

Amazon DynamoDB | Database

Answer 2

DynamoDB Time-to-Live (TTL) is a mechanism that lets you set a specific timestamp to delete expired items from your tables. Once the timestamp expires, the corresponding item is marked as expired and is subsequently deleted from the table. By using this functionality, you do not have to track expired data and delete it manually. TTL can help you reduce storage usage and reduce the cost of storing data that is no longer relevant.

Question 3

Why do I need to use TTL?

Time to Live (TTL)

Amazon DynamoDB | Database

Answer 3

There are two main scenarios where TTL can come in handy:

Deleting old data that is no longer relevant – data like event logs, usage history, session data, etc. when collected can get bloated over time and the old data though stored in the system may not be relevant any more. In such situations, you are better off clearing these stale records from the system and saving the money used for storing it.

Sometimes you may want data to be kept in DynamoDB for a specified time period in order to comply with your data retention and management policies. You might want to eventually delete this data once the obligated duration expires. Please do know however that TTL works on a best effort basis to ensure there is throughput available for other critical operations. DynamoDB will aim to delete expired items within a two-day period. The actual time taken may be longer based on the size of the data.

Question 4

How does DynamoDB TTL work?

Time to Live (TTL)

Amazon DynamoDB | Database

Answer 4

To enable TTL for a table, first ensure that there is an attribute that can store the expiration timestamp for each item in the table. This timestamp needs to be in the epoch time format. This helps avoid time zone discrepancies between clients and servers.

DynamoDB runs a background scanner that monitors all the items. If the timestamp has expired, the process will mark the item as expired and queue it for subsequent deletion.

Note: TTL requires a numeric DynamoDB table attribute populated with an epoch timestamp to specify the expiration criterion for the data. You should be careful when setting a value for the TTL attribute since a wrong value could cause premature item deletion.

Question 5

How do I specify TTL?

Time to Live (TTL)

Amazon DynamoDB | Database

Answer 5

To specify TTL, first enable the TTL setting on the table and specify the attribute to be used as the TTL value. As you add items to the table, you can specify a TTL attribute if you would like DynamoDB to automatically delete it after its expiration. This value is the expiry time, specified in epoch time format. DynamoDB takes care of the rest. TTL can be specified from the console from the overview tab for the table. Alternatively, developers can invoke the TTL API to configure TTL on the table. See our documentation and our API guide.

Question 6

Can I set TTL on existing tables?

Time to Live (TTL)

Amazon DynamoDB | Database

Answer 6

Yes. If a table is already created and has an attribute that can be used as TTL for its items, then you only need to enable TTL for the table and designate the appropriate attribute for TTL. If the table does not have an attribute that can be used for TTL, you will have to create such an attribute and update the items with values for TTL.

Question 7

Can I delete an entire table by setting TTL on the whole table?

Time to Live (TTL)

Amazon DynamoDB | Database

Answer 7

No. While you need to define an attribute to be used for TTL at the table level, the granularity for deleting data is at the item level. That is, each item in a table that needs to be deleted after expiry will need to have a value defined for the TTL attribute. There is no option to automatically delete the entire table.

Question 8

Can I set TTL only for a subset of items in the table?

Time to Live (TTL)

Amazon DynamoDB | Database

Answer 8

Yes. TTL takes affect only for those items that have a defined value in the TTL attribute. Other items in the table remain unaffected.

Question 9

What is the format for specifying TTL?

Time to Live (TTL)

Amazon DynamoDB | Database

Answer 9

The TTL value should use the epoch time format, which is number of seconds since January 1, 1970 UTC. If the value specified in the TTL attribute for an item is not in the right format, the value is ignored and the item won’t be deleted.

Question 10

How can I read the TTL value for items in my table?

Time to Live (TTL)

Amazon DynamoDB | Database

Answer 10

The TTL value is just like any attribute on an item. It can be read the same way as any other attribute. In order to make it easier to visually confirm TTL values, the DynamoDB Console allows you to hover over a TTL attribute to see its value in human-readable local and UTC time.

Question 11

Can I create an index based on the TTL values assigned to items in a table?

Time to Live (TTL)

Amazon DynamoDB | Database

Answer 11

Yes. TTL behaves like any other item attribute. You can create indexes the same as with other item attributes.

Question 12

Can the TTL attribute be projected to an index?

Time to Live (TTL)

Amazon DynamoDB | Database

Answer 12

Yes. TTL attribute can be projected onto an index just like any other attribute.

Question 13

Can I edit the TTL attribute value once it has been set for an item?

Time to Live (TTL)

Amazon DynamoDB | Database

Answer 13

Yes. You can modify the TTL attribute value just as you modify any other attribute on an item.

Question 14

Can I change the TTL attribute for a table?

Time to Live (TTL)

Amazon DynamoDB | Database

Answer 14

Yes. If a table already has TTL enabled and you want to specify a different TTL attribute, then you need to disable TTL for the table first, then you can re-enable TTL on the table with a new TTL attribute. Note that Disabling TTL can take up to one hour to apply across all partitions, and you will not be able to re-enable TTL until this action is complete.

Question 15

Can I use AWS Management Console to view and edit the TTL values?

Time to Live (TTL)

Amazon DynamoDB | Database

Answer 15

Yes. The AWS Management Console allows you to easily view, set or update the TTL value.

Question 16

Can I set an attribute within a JSON document to be the TTL attribute?

Time to Live (TTL)

Amazon DynamoDB | Database

Answer 16

No. We currently do not support specifying an attribute in a JSON document as the TTL attribute. To set TTL, you must explicitly add the TTL attribute to each item.

Question 17

Can I set TTL for a specific element in a JSON Document?

Time to Live (TTL)

Amazon DynamoDB | Database

Answer 17

No. TTL values can only be set for the whole document. We do not support deleting a specific item in a JSON document once it expires.

Question 18

What if I need to remove the TTL on specific items?

Time to Live (TTL)

Amazon DynamoDB | Database

Answer 18

Removing TTL is as simple as removing the value assigned to the TTL attribute or removing the attribute itself for an item.

Question 19

What if I set the TTL timestamp value to sometime in the past?

Time to Live (TTL)

Amazon DynamoDB | Database

Answer 19

Updating items with an older TTL values is allowed. Whenever the background process checks for expired items, it will find, mark and subsequently delete the item. However, if the value in the TTL attribute contains an epoch value for a timestamp that is over 5 years in the past, DynamoDB will ignore the timestamp and not delete the item. This is done to mitigate accidental deletion of items when really low values are stored in the TTL attribute.

Question 20

What is the delay between the TTL expiry on an item and the actual deletion of that item?

Time to Live (TTL)

Amazon DynamoDB | Database

Answer 20

TTL scans and deletes expired items using background throughput available in the system. As a result, the expired item may not be deleted from the table immediately. DynamoDB will aim to delete expired items within a two-day window on a best-effort basis, to ensure availability of system background throughput for other data operations. The exact duration within which an item truly gets deleted after expiration will be specific to the nature of the workload and the size of the table.

Question 21

What happens if I try to query or scan for items that have been expired by TTL?

Time to Live (TTL)

Amazon DynamoDB | Database

Answer 21

Given that there might be a delay between when an item expires and when it actually gets deleted by the background process, if you try to read items that have expired but haven’t yet been deleted, the returned result will include the expired items. You can filter these items out based on the TTL value if the intent is to not show expired items.

Question 22

What happens to the data in my Local Secondary Index (LSI) if it has expired?

Time to Live (TTL)

Amazon DynamoDB | Database

Answer 22

The impact is the same as any delete operation. The local secondary index is stored in the same partition as the item itself. Hence if an item is deleted it immediately gets removed from the Local Secondary Index.

Question 23

What happens to the data in my Global Secondary Index (GSI) if it has expired?

Time to Live (TTL)

Amazon DynamoDB | Database

Answer 23

The impact is the same as any delete operation. A Global Secondary Index (GSI) is eventually consistent and so while the original item that expired will be deleted it may take some time for the GSI to get updated.

Question 24

How does TTL work with DynamoDB Streams?

Time to Live (TTL)

Amazon DynamoDB | Database

Answer 24

The expiry of data in a table on account of the TTL value triggering a purge is recorded as a delete operation. Therefore, the Streams will also have the delete operation recorded in it. The delete record will have an additional qualifier so that you can distinguish between your deletes and deletes happening due to TTL. The stream entry will be written at the point of deletion, not the TTL expiration time, to reflect the actual time at which the record was deleted. See our documentation and our API guide.

Question 25

When should I use the delete operation vs TTL?

Time to Live (TTL)

Amazon DynamoDB | Database

Answer 25

TTL is ideal for removing expired records from a table. However, this is intended as a best-effort operation to help you remove unwanted data and does not provide a guarantee on the deletion timeframe. As a result, if data in your table needs to be deleted within a specific time period (often immediately), we recommend using the delete command.

Question 26

Can I control who has access to set or update the TTL value?

Time to Live (TTL)

Amazon DynamoDB | Database

Answer 26

Yes. The TTL attribute is just like any other attribute on a table. You have the ability to control access at an attribute level on a table. The TTL attribute will follow the regular access controls specified for the table.

Question 27

Is there a way to retrieve the data that has been deleted after TTL expiry?

Time to Live (TTL)

Amazon DynamoDB | Database

Answer 27

No. Expired items are not backed up before deletion. You can leverage the DynamoDB Streams to keep track of the changes on a table and restore values if needed. The delete record is available in Streams for 24 hours since the time it is deleted.

Question 28

How can I know whether TTL is enabled on a table?

Time to Live (TTL)

Amazon DynamoDB | Database

Answer 28

You can get the status of TTL at any time by invoking the DescribeTable API or viewing the table details in the DynamoDB console. See our documentation and our API guide.

Question 29

How do I track the items deleted by TTL?

Time to Live (TTL)

Amazon DynamoDB | Database

Answer 29

If you have DynamoDB streams enabled, all TTL deletes will show up in the DynamoDB Streams and will be designated as a system delete in order to differentiate it from an explicit delete done by you. You can read the items from the streams and process them as needed. They can also write a Lambda function to archive the item separately. See our documentation and our API guide.

Question 30

Do I have to pay a specific fee to enable the TTL feature for my data?

Time to Live (TTL)

Amazon DynamoDB | Database

Answer 30

No. Enabling TTL requires no additional fees.

Question 31

How will enabling TTL affect my overall provisioned throughput usage?

Time to Live (TTL)

Amazon DynamoDB | Database

Answer 31

The scan and delete operations needed for TTL are carried out by the system and does not count toward your provisioned throughput or usage.

Question 32

Will I have to pay for the scan operations to monitor TTL?

Time to Live (TTL)

Amazon DynamoDB | Database

Answer 32

No. You are not charged for the internal scan operations to monitor TTL expiry for items. Also these operations will not affect your throughput usage for the table.

Question 33

Do expired items accrue storage costs till they are deleted?

Time to Live (TTL)

Amazon DynamoDB | Database

Answer 33

Yes. After an item has expired it is added to the delete queue for subsequent deletion. However, until it has been deleted, it is just like any regular item that can be read or updated and will incur storage costs.