Scaling - Kinesis

Question 1

What is Kinesis?

Answer 1

custom enabled application that eases collection, processing, analyzation of streaming data and video real time

Question 2

What problem does Kinesis Solve?

Answer 2

don’t worry about provision, deployment, maintenance of hardware and software of data streams; collect data real time such as video, audio, machine learning, analytics for further processing

Question 3

What are the kinesis components?

Answer 3

producer (any kinesis data source), kinesis video stream (transport, consumption and storage of live video data), and consumer (takes data for view, processing and analyzing)

Question 4

What is a kinesis producer?

Answer 4

any source that puts data into a kinesis data stream

Question 5

What is a kinesis video stream?

Answer 5

enable you to securly transport live video data, store it, and make it available for consumption both real time or ad hoc to AWS

Question 6

What is a kinesis consumer?

Answer 6

retrieves data such as frames and fragments from a Kinesis video stream to view, process or analyze it

Question 7

What does kinesis video streams provide?

Answer 7

APIs for creation/management of media data to and from a stream, console that supports live and on-demand playback

Question 8

What is shard?

Answer 8

the base throughput unit of a kinesis data stream; a sequence of data records and provides a fixed unit of capacity, each shard supports 1000 PUT records ps, 1MB/sec data input and 2MB/sec data output

Question 9

How long are streams stored?

Answer 9

24 hours by default, to 7 days of extended retention and 365 long-term retention

Question 10

What are Kinesis streams?

Answer 10

streams are made up of shards

Question 11

How fast are shards processed?

Answer 11

five reads per sec at max total read rate is 2 MBps/1000 writes per second at total of 1 MBps

Question 12

How can capacity be increased?

Answer 12

by increasing the number of shards;

Question 13

What is Kinesis Data Firehose?

Answer 13

capture, transform, load data streams into AWS data stores to enable real-time analytics with BI tools; no shards or streams; data can be saved to S3, Redshift

Question 14

What is Kinesis Data Analytics?

Answer 14

analyze, query and transform streamed data in real-time using standard SQL; data in ingested into Kinesis streams/data analytics and will SQL queries run on that data, data is then moved to Redshift, S3

Question 15

What are the limits to shards and streams?

Answer 15

no upper limit to shard in a stream or streams themselves, single shad can ingest up to 1Mbps (1000 shards per second, 5 reads per second, default shard limit is 500 and 200 shards for certain AWS regions

Question 16

What is Kinesis Streams?

Answer 16

capture, store streaming video and data for real time processing

Question 17

What is pricing for Kinesis Data Streams?

Answer 17

charged for each shard at an hourly rate; PUT charged per million rate; extended data retention incurs an additional rate per shard hour, Hourly Shard cost determined by the number of shards within your Amazon Kinesis data stream.
PUT Payload Unit cost determined by the number of 25KB payload units that your data producers add to your data stream

Question 18

What is pricing for Kinesis Video Streams?

Answer 18

pay only for volume of data your ingest, store and consume through the service

Question 19

What is pricing for Kinesis Data Firehouse?

Answer 19

you pay only for the volume of data you transmit through the service; billed for volume of data you ingest in Kinesis

Question 20

What are the limits to Kinesis Data Firehouse?

Answer 20

up to 50 Kinesis Data Firehouse streams per region; max record size sent to Firehouse is 1 Mbps

Question 21

What is pricing for Kinesis Data Analytics?

Answer 21

charged hourly based on average number of Kinesis Processing Units; charged a single addition KPU for Java app orchestration

Question 22

What is the order of operations for Kinesis Data Firehose?

Answer 22

1. Navigate to Kinesis Service
2. Create Delivery Stream
3. Select Destination (S3, Redshift, Elasticsearch)
4. Set Buffer
5. Create IAM Role
6. Generate Traffic from Source to Destination
   7.