Kinesis Flashcards
Makes it easy to collect, process, and analyze streaming data in real-time
Kinesis
capture, process, and store data streams
Kinesis Data Streams
load data streams into AWS data stores
Kinesis Data Firehose
analyze data streams with SQL or Apache Flink
Kinesis Data Analytics
capture, process, and store video streams
Kinesis Video Streams
Kinesis Data Streams – Capacity Modes
Provisioned vs On-demand
- You choose the number of shards provisioned, scale manually or using API
- Each shard gets 1MB/s in (or 1000 records per second)
- Each shard gets 2MB/s out (classic or enhanced fan-out consumer)
- You pay per shard provisioned per hour
Provisioned mode
Kinesis Data Streams – Capacity Modes
Provisioned vs On-demand
- No need to provision or manage the capacity
- Default capacity provisioned (4 MB/s in or 4000 records per second)
- Scales automatically based on observed throughput peak during the last 30 days
- Pay per stream per hour & data in/out per GB
On-demand mode
Kinesis Data Streams vs Firehose
- Streaming service for ingest at scale
- Write custom code (producer/consumer)
- Real-time (~200 ms)
- Manage scaling (shard splitting/merging)
- Data storage for 1 to 365 days
- Supports replay capability
Kinesis Data Firehose
Kinesis Data Streams vs Firehose
- Load streaming data into S3 / Redshift /
OpenSearch / 3rd party/custom HTTP - Fully managed
- Near real-time (buffer time min. 60 sec)
- Automatic scaling
- No data storage
- Doesn’t support replay capability
Kinesis Data Firehose
Used when migrating to the cloud from on-premises that may use open protocols, instead of re-engineering the application to use SQS and SNS.
Amazon MQ