Kafka

Question 1

Kafka difference # 1

Answer 1

It works as a modern distributed system that runs as a cluster and can scale to handle all the applications in even the most massive of companies. Rather than running dozens of individual messaging brokers, hand wired to different apps, this lets you have a central platform that can scale elastically to handle all the streams of data in a company.

Question 2

Kafka difference # 2

Answer 2

Kafka is a true storage system built to store data for as long as you might like. This has huge advantages in using it as a connecting layer as it provides real delivery guarantees—its data is replicated, persistent, and can be kept around as long as you like.

Question 3

Kafka difference # 3

Answer 3

The world of stream processing raises the level of abstraction quite significantly. Messaging systems mostly just hand out messages. The stream processing capabilities in Kafka let you compute derived streams and datasets dynamically off of your streams with far less code.

Question 4

What does DAG stand for?

Answer 4

Directed Acyclic Graph

Question 5

What does DAG stand for?

Answer 5

Directed Acyclic Graph

Question 6

What is a Source Processor?

Answer 6

Sources are where information flows into the Kafka Streams application. Data is read from a Kafka topic and sent into one or more stream processors.

Question 7

What is a Stream Processor?

Answer 7

These processors are responsible for applying data processing/transformation logic on the input stream. In the high-level DSL, these processors are defined ising a set of built-in operators that are exposed by the Kafka Streams library.

Examples of operators are filter, map, flatmap, and join.

Question 8

What is a Sink processor?

Answer 8

Sinks are where enriched, transformed, filtered, or otherwise processed records are written back to Kafka, either to be handled by another stream processing application or to be sent to a downstream data store via something like Kafka Connect. Like source processors, sink processors are connected to a Kafka topic.

Question 9

3 kinds of processors in Kafka

Answer 9

1. Source processors
2. Stream processors
3. Sink processors

Question 10

Parallelise

Answer 10

Adapt for running on a parallel processing system.