Amazon Kinesis Data Analytics | Key Kinesis Data Analytics Concepts Flashcards
What are the limits of Kinesis Data Analytics?
Key Kinesis Data Analytics Concepts
Amazon Kinesis Data Analytics | Analytics
Kinesis Data Analytics elastically scales your application to accommodate for the data throughput of your source stream and your query complexity for most scenarios. For detailed information on service limits, see Limits in the Amazon Kinesis Data Analytics Developer Guide.
What is a Kinesis Data Analytics application?
Key Kinesis Data Analytics Concepts
Amazon Kinesis Data Analytics | Analytics
An application is the Kinesis Data Analytics entity that you work with. Kinesis Data Analytics applications continuously read and process streaming data in real-time. You write application code using SQL to process the incoming streaming data and produce output. Then, Kinesis Data Analytics writes the output to a configured destination.
Each application consists of three primary components:
Input – The streaming source for your application. In the input configuration, you map the streaming source to an in-application input stream. The in-application stream is like a continuously updating table upon which you can perform SELECT and INSERT SQL operations. Each input record has an associated schema, which is applied as part of inserting the record into the in-application stream.
Application code – A series of SQL statements that process input and produce output. In its simplest form, application code can be a single SQL statement that selects from a streaming input and inserts results into a streaming output. It can also be a series of SQL statements where the output of one feeds into the input of the next SQL statement. Further, you can write application code to split an input stream into multiple streams and then apply additional queries to process these separate streams.
Output – You can create one or more in-application streams to hold intermediate results. You can then optionally configure an application output to persist data from specific in-application streams to an external destination.
What is an in-application stream?
Key Kinesis Data Analytics Concepts
Amazon Kinesis Data Analytics | Analytics
An in-application stream is an entity that continuously stores data in your application for you to perform the SELECT and INSERT SQL operations. You interact with an in-application stream in the same way that you would a SQL table. However, a stream differs from a table in that data is continuously updated. In your application code, you can create additional in-application streams to store intermediate query results. Finally, both your configured input and output represent themselves in your application as in-applications streams.
What inputs are supported in a Kinesis Data Analytics application?
Key Kinesis Data Analytics Concepts
Amazon Kinesis Data Analytics | Analytics
Kinesis Data Analytics supports two types of inputs: streaming data sources and reference data sources. A streaming data source is continuously generated data that is read into your application for processing. A reference data source is static data that your application uses to enrich data coming in from streaming sources. Each application can have no more than one streaming data source and no more than one reference data source. An application continuously reads and processes new data from streaming data sources, including Amazon Kinesis Data Streams or Amazon Kinesis Data Firehose. An application reads a reference data source, including Amazon S3, in its entirety for use in enriching the streaming data source through SQL JOINs.
What is a reference data source?
Key Kinesis Data Analytics Concepts
Amazon Kinesis Data Analytics | Analytics
A reference data source is static data that your application uses to enrich data coming in from streaming sources. You store reference data as an object in your S3 bucket. When the application starts, Kinesis Data Analytics reads the S3 object and creates an in-application SQL table to store the reference data. Your application code can then join it with an in-application stream. You can update the data in the SQL table by calling the UpdateApplication API.
What application code is supported?
Key Kinesis Data Analytics Concepts
Amazon Kinesis Data Analytics | Analytics
Kinesis Data Analytics supports the ANSI SQL with some extensions to the SQL standard to make it easier to work with streaming data. Additionally, Kinesis Data Analytics provides several machine learning algorithms that are exposed as SQL functions including anomaly detection, approximate top-K, and approximate distinct items.
