08 Analytics

Question 1

Describe Kinesis and its primary function.

Answer 1

Kinesis is a highly-available scalable streaming service that allows producers to send data into streams for processing and analysis.

Question 2

How does Kinesis ensure scalability?

Answer 2

Kinesis uses sharding to provide scalability, with each shard supporting 1 MB/s of ingestion and 2 MB/s of read throughput.

Question 3

Define the data retention period in Kinesis by default.

Answer 3

Data in Kinesis is retained for 24 hours by default, but this can be increased to 365 days.

Question 4

What is the role of Kinesis Data Firehose?

Answer 4

Kinesis Data Firehose delivers records from Kinesis streams to supported destinations such as S3, OpenSearch, or Redshift.

Question 5

Explain the near-realtime aspect of Kinesis Data Firehose.

Answer 5

Kinesis Data Firehose operates in near-realtime, but there is a delay due to batching and other processing.

Question 6

How can data be transformed in Kinesis Data Firehose?

Answer 6

Data can be transformed in-flight using AWS Lambda functions.

Question 7

Describe the function of Kinesis Data Analytics.

Answer 7

Kinesis Data Analytics uses SQL to analyze data in real-time, with results sent to other Kinesis streams or Kinesis Firehose.

Question 8

What type of data can Kinesis Data Analytics reference?

Answer 8

Kinesis Data Analytics can reference static data from an S3 bucket.

Question 9

Define Elastic Map Reduce and its purpose.

Answer 9

Elastic Map Reduce is a managed implementation of Apache Hadoop, designed to process large amounts of data using the Hadoop ecosystem.

Question 10

What additional elements are included in Elastic Map Reduce?

Answer 10

Elastic Map Reduce includes other elements of the Apache ecosystem alongside Hadoop.

Question 11

Describe the architecture of an EMR cluster.

Answer 11

An EMR cluster consists of a master node, core nodes that provide HDFS storage and run jobs, and task nodes that only run jobs.

Question 12

How does EMRFS enhance durability compared to HDFS?

Answer 12

EMRFS is backed by S3, making it more durable than HDFS, which is tied to instances in a single Availability Zone (AZ).

Question 13

Define Redshift and its primary use case.

Answer 13

Redshift is a petabyte-scale data warehouse optimized for Online Analytical Processing (OLAP) and uses a columnar data format.

Question 14

Explain the node structure of a Redshift cluster.

Answer 14

A Redshift cluster has a single leader node and multiple compute nodes.

Question 15

What is the significance of Enhanced VPC routing in Redshift?

Answer 15

Enhanced VPC routing causes traffic in the Redshift cluster to use the VPC’s routing settings instead of public endpoints.

Question 16

How often are backups taken in Redshift?

Answer 16

Backups in Redshift are taken automatically every approximately 8 hours or when more than 5GB of data has changed.

Question 17

Discuss the cost efficiency strategy for EMR clusters.

Answer 17

For cost efficiency, use on-demand instances for the master and core nodes, and spot instances for the task nodes.

Question 18

Identify the limitations of both EMR and Redshift regarding availability.

Answer 18

Both EMR and Redshift run in a single Availability Zone (AZ), which means they are not highly available.

Question 19

What happens to snapshots in Redshift?

Answer 19

Snapshots in Redshift are automatically taken and stored in S3.

Question 20

Describe the default retention period for data in Redshift.

Answer 20

Data is retained for 1 day by default, but a retention period of up to 35 days can be selected.

Question 21

How does Redshift Spectrum function with data in S3?

Answer 21

Redshift Spectrum directly queries data in S3 without loading it into Redshift first.

Question 22

Define the requirement for using Redshift Spectrum.

Answer 22

Redshift Spectrum is not serverless; a Redshift cluster is still required.

Question 23

What is the purpose of Federated Query in Redshift?

Answer 23

Federated Query allows Redshift to query data stored in other databases without loading it into Redshift first.

Question 24

How can one mitigate the single-AZ risk in Redshift?

Answer 24

To overcome the single-AZ risk, take regular snapshots and restore to a new cluster.

Question 25

Describe Amazon MQ.

Answer 25

Amazon MQ is a managed implementation of Apache ActiveMQ, providing functionality similar to SNS and SQS.

Question 26

What messaging protocols does Amazon MQ support?

Answer 26

Amazon MQ supports JSM, AMQP, MQTT, OpenWire, and STOMP protocols.

Question 27

How does Amazon MQ handle messaging?

Answer 27

Amazon MQ allows for one-to-one and one-to-many messaging.

Question 28

Where does Amazon MQ run?

Answer 28

Amazon MQ runs in a VPC either as a single instance or as a high availability (HA) pair (active/standby).