Messaging

Question 1

SQS Message Duration

Answer 1

Default is 4 days but range 1-14 days

attribute: MessageRetentionPeriod

Question 2

SNS Max Subscribers and Max Topics?

Answer 2

12.5 million subscribers. 100K Topics.

Question 3

Kinesis Data Expiry

Answer 3

Default 24 hours. Extended Data can be 7 days. Long Term Retention 365 days.

Question 4

Max sqs msg size

Answer 4

256kb.

Using Java extended client library to store msgs > 256kb. It stores data in S3 and just a link ref in sqs

Question 5

Max sqs msgs

Answer 5

Unlimited.

Max of 120k in flight for std queue and 20k

Question 6

SQS Visibiltiy Timeout

Answer 6

30 Seconds. Configurable up to 12 hours.

attribute: VisibilityTimeout

Question 7

SQS Delivery Delay

Answer 7

Default 0. Maximum 15 seconds.

attribute: DelaySeconds

Question 8

How to Scale Standard SQS Queue

Answer 8

Add more consumers.

Question 9

Can you change SQS Queue Type after creation ?

Answer 9

NO

Question 10

How to remove a queue and its contents ?

How to just empty the queue?

Answer 10

DeleteQueue will empty and remove the Queue. Can take 60 seconds.

PurgeQueue will just empty it.

Question 11

BeanStalk Updates - Rolling v ROlling with Additional Batches.

Answer 11

Rolling update takes down old instances in small batches, replacing it with the new version. Users may therefore sometimes point to old version and sometimes new version. No additional instances are spun up so cost remains the same but deployment will take longer and minor performance hit during the rollout.

Rolling WIth Additional batches is similar but it adds new batches (and this more Instances at more $) but ensures that performance remains the same throughout.

Question 12

in SQS, whats the best way to handle high variable traffic spikes ?

Answer 12

Use backlog per instance metric with target tracking scaling policy

Whilst you can use CloudWatch Amazon SQS metric like ApproximateNumberOfMessagesVisible, this doesn’t consider how many consumers are running so back log per instance = msgs / consumers

Question 13

What options do you have to address the issue of ProvisionedThroughputExceeded for Kinesis?

Answer 13

1) Increase the number of shards within your data streams to provide enough capacity

2) Configure the data producer to retry with an exponential backoff

Question 14

How long does kinesis keep data for ?

Answer 14

Default 24 hours. Max 365 days

Question 15

Kinesis or Kinesis firehose?

Answer 15

Kinesis data streams are used in places where an unbounded stream of data needs to worked on in real time. Data by default is stored for 24 hours but can store up to 365 days. Requires custom dev code.

Kinesis Firehose delivery streams are used when data needs to be delivered to a storage destination, such as S3

Firehose: Firehose handles loading data streams directly into AWS products for processing.

Fully managed, send to S3, Splunk, Redshift, ElasticSearch

Serverless data transformations with Lambda

Near real time (lowest buffer time is 1 minute)

Automated Scaling

No data storage

Question 16

Kinesis KPL library verses Kinesis Agents ?

Answer 16

The KPL is an easy-to-use, highly configurable library that helps you write to a Kinesis data stream. It acts as an intermediary between your producer application code and the Kinesis Data Streams API actions.

Kinesis Agent is designed to continuously monitor a set of files and send new data to your stream for example send EC2 log files to Kinesis.

Question 17

Kinesis Data STreams Standard V Enahanced Fan Out

Answer 17

Stanard for low nbr of consumers (1-3) with latency of 200ms.

Enahnced for lower latency (70ms) and handling larger consumers (5+). Costs more.

Question 19

What is the maximum number of EC2 instances that can be deployed to process 10 shards.

Answer 19

10. You can have at most one EC2 instance per shard in Kinesis for the given application.

Question 20

Sqs visibility v sqs retention

Answer 20

Retention is how long msgs stay on queue. Default is 4 days. Max is 14 days.

Visibility is how long they remain invisible to other consumers when a consumer picks up the msg. Ie how long your process will need to process the msg.

Default is 29 seconds. Max is 12 hours