AWS ML Eng Assoc - data storage 2

Question 1

Kinesis Data Streams

Answer 1

A real-time data streaming service that can ingest and process large amounts of data in real-time

Question 2

Shard

Answer 2

A unit of throughput capacity in Kinesis Data Streams. Each shard provides 1 MB/s of write capacity and 2 MB/s of read capacity

Question 3

Partition Key

Answer 3

A key used to group data by shard within a stream

Question 4

KPL (Kinesis Producer Library)

Answer 4

A library that helps you easily and reliably put data into Kinesis Data Streams

Question 5

KCL (Kinesis Client Library)

Answer 5

A library that helps you consume and process data from Kinesis Data Streams

Question 6

Enhanced Fan-Out

Answer 6

A feature that allows consumers to receive records from a stream with dedicated throughput of 2 MB/s per shard

Question 7

Kinesis Data Firehose

Answer 7

A fully managed service for delivering real-time streaming data to destinations such as S3; Redshift; Elasticsearch; and Splunk

Question 8

Kinesis Data Analytics

Answer 8

A service that allows you to process and analyze streaming data using SQL or Apache Flink

Question 9

MSK (Managed Streaming for Apache Kafka)

Answer 9

A fully managed Apache Kafka service that allows you to build and run applications that use Apache Kafka to process streaming data

Question 10

Shard Splitting

Answer 10

The process of increasing the number of shards in a Kinesis stream to increase capacity

Question 11

Shard Merging

Answer 11

The process of combining two shards in a Kinesis stream to decrease capacity

Question 12

Hot Shard

Answer 12

A shard that receives more data than others; potentially causing throughput issues

Question 13

Kinesis Agent

Answer 13

A stand-alone Java application that offers an easy way to collect and send data to Kinesis Data Streams

Question 14

Provisioned Mode

Answer 14

A capacity mode in Kinesis where you specify the number of shards for your stream

Question 15

On-Demand Mode

Answer 15

A capacity mode in Kinesis where capacity is automatically managed to accommodate your workload

Question 16

Random Cut Forest

Answer 16

An algorithm used in Kinesis Data Analytics for anomaly detection in streaming data

Question 17

Use case: Streaming ETL

Answer 17

Using Kinesis Data Analytics or MSK to perform real-time Extract; Transform; Load operations on streaming data

Question 18

Use case: Real-time Analytics

Answer 18

Using Kinesis Data Streams and Kinesis Data Analytics to process and analyze data in real-time for insights

Question 19

Use case: Log and Event Data Processing

Answer 19

Using Kinesis to ingest and process log files and event data from various sources in real-time

Question 20

Mnemonic: KPL puts; KCL gets

Answer 20

Remember that Kinesis Producer Library (KPL) is used to put data into streams; while Kinesis Client Library (KCL) is used to get data from streams

Question 21

Metaphor: Kinesis as a river

Answer 21

Think of Kinesis Data Streams as a river; producers add water (data) upstream; consumers take water out downstream; shards are the width of the river determining how much water can flow