SQS, and Kinesis

Question 1

What is the max message size in SQS?

Answer 1

256 KB.

Question 2

What the number of messages that can

be processed as a batch in SQS?

Answer 2

10 messages at a time.

Question 3

What is visibility timeout in SQS?

Answer 3

Once a consumer receives a message, SQS doesn’t allow any other consumers to read the same message for a period of time, called the visibility timeout.

Question 4

What is the default and max visibility timeout period in SQS?

Answer 4

• Default: 30 seconds.

- Max: 12 hours

Question 5

What is the default and max retention period in SQS?

Answer 5

• Default: 4 days

- Max: 14 days

Question 6

What is default and max per-queue delay settings in SQS?

Answer 6

• Default: 0 second

- Max: 15 minutes

Question 7

What are the two different queue types in SQS?

Answer 7

• Standard queue

- FIFO queue

Question 8

What are the pros and cons of standard queue in SQS?

Answer 8

Pro: almost unlimited throughput. Can handle 120,000 in-flight message.

Con:

• Messages may be delivered out of order
• Messages may occasionally be delivered more than once

Question 9

What are the pros and cons of FIFO queue in SQS?

Answer 9

Pro:

• Messages are delivered in order
• Messages are delivered exactly once
• Supports message grouping

Con:

• Can send 3,000 messages per second
• Can handle 20,000 in-flight messages

Question 10

Is message in SQS deleted automatically after received by client?

Answer 10

No, client will need to delete it by calling API.

Question 11

What are polling options to receive messages from SQS?

Answer 11

• Short polling (quick, but may not get all the messages)

- Long polling (slow, but get all messages)

Question 12

What is Dead-Letter Queue in SQS?

Answer 12

You can have SQS automatically pull such a message from your source queue and place it into a dead‐letter queue after it’s been received so many times.

It’s based on maxReceiveCount of the message.

One thing to keep in mind is that when a message is moved to a dead‐letter queue, the message will be deleted based on its original creation date.

Question 13

What is Kinesis?

Answer 13

Kinesis is a collection of services that let you collect, process, store, and deliver streaming data.

Kinesis can perform real‐time ingestion of gigabytes per second from thousands of sources, making it perfect for things like audio and video feeds, application logs, and telemetry data.

Question 14

What are the Kinesis offerings?

Answer 14

• Kinesis Video Streams
• Kinesis Data Streams
• Kinesis Data Firehose

Question 15

What is Kinesis Video Streams?

Answer 15

Kinesis Video Streams is designed to ingest and index almost unlimited amounts of streaming video data, such as from webcams, security cameras, and smartphones.

Kinesis Video Streams uses a producer‐consumer model.

Kinesis automatically indexes the stream by using timestamps, allowing for easy retrieval later on.

By default, Kinesis Video Streams will store a video stream for 24 hours, but you can extend this retention period up to 7 days.

Question 16

What is Kinesis Data Streams?

Answer 16

Kinesis Data Streams is a data pipeline that can aggregate, buffer, and reliably store data from producers until a consumer is ready to process it.

Kinesis Data Streams stores data in a data record, along with a partition key and sequence number to uniquely identify each record.

Multiple consumers can read from a stream concurrently, a process called fan‐out.

The maximum throughput of a stream depends on the number of shards you configure.

Each shard supports up to five read transactions per second, with a maximum data rate of 2 MB per second. For writes, you can push up to 1,000 records per second, with a data rate of 1 MB per second.

Question 17

What’s the difference between SQS and Kinesis Data Streams?

Answer 17

SQS temporarily holds a small message in queue until a single consumer processes and deletes it.

Kinesis Data Streams, on the other hand, is designed to provide durable storage and playback of large data streams—such as log files—to multiple consumers.

Question 18

What is Kinesis Data Firehose?

Answer 18

Kinesis Data Firehose can ingest streaming data and transform it before sending it to a destination. Data transformation may include cleaning data or converting it to a different format.

You can configure Kinesis Data Firehose to receive data from a Kinesis Data Stream.

Question 19

What is the difference between Kinesis Data Streams and Kinesis Data Firehose?

Answer 19

Kinesis Data Streams uses producer-consumer model, and has a retention period of 7 days.

Kinesis Data Firehose is not an open-ended solution, and you must configure one or more AWS resources as destinations. The retention period is 24 hours.