Kinesis overview and Streams

Question 1

What are the 3 types of Data Collection?

Answer 1

• Real-Time (Immediate Actions)
• Near Real-Time (Reactive Actions)
• Batch (Historical Analysis)

Question 2

What are 3 examples of Real-Time data collection?

Answer 2

• Kinesis Data Streams (KDS)
• Simple Queue Service (SQS)
• Internet of Things (IoT)

Question 3

What are 2 examples of Near Real-Time Data Collection?

Answer 3

• Kinesis Data Firehose (KDF)

- Database Migration Service (DMS)

Question 4

What are 2 examples of Batch Data Collection?

Answer 4

• Snowball

- Data Pipeline

Question 5

What are the 3 Kinesis Services?

Answer 5

• Kinesis Streams
• Kinesis Analytics
• Kinesis Firehose

Question 6

What does Kinesis Streams allow you to do?

Answer 6

Low latency streaming ingest at scale

Question 7

What does Kinesis Analytics allow you to do?

Answer 7

Perform real-time analytics on streams using SQL

Question 8

What does Kinesis Firehose allow you to do?

Answer 8

Load streams into S3, Redshift, ElasticSearch & Splunk

Question 9

How do Kinesis Streams work?

Answer 9

• Producers update data in shards/partitions

- Consumers read from partitions

Question 10

What is the data retention in Kinesis Streams?

Answer 10

• Default is 24 hours

- Can go up to 7 days

Question 11

How many applications can consume the same stream?

Answer 11

Multiple applications can consume the same stream

Question 12

Can an application update data in a Kinesis stream?

Answer 12

No, the data is immutable. It is append-only and will remain until the data retention period has been reached

Question 13

How are records ordered?

Answer 13

They are ordered per shard

Question 14

What is produced to a shard?

Answer 14

Records and Record Keys are produced to shards

Question 15

What is a record made up of?

Answer 15

A record is made up of a Data Blob

Question 16

What is a Data Blob?

Answer 16

It is the data being sent in a stream, serialized as bytes up to 1MB

Question 17

What is a Record Key?

Answer 17

It helps to group records in shards.

- Same key = dame shard

Question 18

How do you avoid a “hot partition” problem?

Answer 18

Use a highly distributed record key

Question 19

What is a Sequence Number?

Answer 19

It is the unique id given to the record by Kinesis

Question 20

How many MBs per second can a producer write per shard?

Answer 20

1MB per second

Question 21

How many messages per second can a producer write per shard?

Answer 21

1000 messages per second

Question 22

What happens if I go over the 1MB/s limit or the 1000 messages/s limit?

Answer 22

I get the “ProvisionedThroughoutException”

Question 23

What is the read data limit per shard for Classic Consumers?

Answer 23

2MBs per second across all consumers

Question 24

What is the API call limit per shard for Classic Consumers?

Answer 24

5 API calls per second per shard across all consumers

Question 25

What is the read data limit per shard for Enhanced Fan-Out Consumers?

Answer 25

2MBs per second, per shard, per Enhanced Consumer

Question 26

What is the API call limit per shard for Enhanced Fan-Out Consumers?

Answer 26

It is a push model, no API calls needed

Question 27

What is the default Data Retention for Kinesis Streams?

Answer 27

24 hours

Question 28

What can the Data Retention for Kinesis Streams be extended to?

Answer 28

7 days