Kinesis Data Firehose Flashcards
Kinesis Data Firehose
Kinesis Data Firehose is the easiest way to load streaming data into data stores and analytics tools.
Captures, transforms, and loads streaming data.
Firehose can batch, compress, and encrypt data before loading it.
Can Kinesis Data Streams act as a source for Firehose
YES
What is the Maximum Record Size
The maximum size of a record (before Base64-encoding) is 1000 KB.
Firehose Destinations include:
- Amazon S3.
- Amazon Redshift.
- Amazon Elasticsearch Service.
- Splunk.
Can encrypt data with…
Can encrypt data with an existing AWS Key Management Service (KMS) key.
Server-side-encryption can be used if Kinesis Streams is used as the data source.
Firehose Transformation Pattern ?
Firehose can invoke an AWS Lambda function to transform incoming data before delivering it to a destination.
Kinesis Firehouse Backup Pattern
For Amazon Redshift destinations, streaming data is delivered to your S3 bucket first.
Kinesis Data Firehose then issues an Amazon Redshift COPY command to load data from your S3 bucket to your Amazon Redshift cluster.
If data transformation is enabled, you can optionally back up source data to another Amazon S3 bucket:
ElasticSearch and Splunk Backup Pattern
For Amazon Elasticsearch destinations, streaming data is delivered to your Amazon ES cluster, and it can optionally be backed up to your S3 bucket concurrently: