Components of a Data Platform

Question 1

What are the common components of a data platform?

Answer 1

The following are common layers of a Data Platform in order:

Data Source Layer
Ingestion Layer
Storage Layer
Processing Layer
Consumption Layer

Question 2

What are some components of Data Source Layer?

Answer 2

You can find various data sources in the Data Source Layer. They can come from structured, semi structured, unstructured data.
Examples:
Exposed Web API’s
Web Applications
Mobile Applications
Database’s

Question 3

What are some components of the Ingestion Layer?

Answer 3

Data platforms need mechanisms for ingesting data from the various data sources into the storage layer.

Examples:
Batch Ingestion
Steam Ingestion

Question 4

What are some components of the Storage Layer?

Answer 4

The goal of the storage layer, as it’s name suggest is to store the data, while handling the various type that arrived from the different sources.

Examples:
SQL Databases
NoSQL Databases
Object Storage ETC
Data Lake
Data Warehouse

Question 5

What are some components of the Processing Layer?

Answer 5

Once data is stored (or interlocked with ingestion), it often needs to be processed and transformed before it can be analyzed or consumed by applications. This can be done with the following as examples:

ETL
ELT

Question 6

What are some components of the Consumption Layer?

Answer 6

In this layer, mechanisms for querying, analyzing, and visualizing data to derive insights or to make data-driven decisions are provided.

Examples:
BI Tools
Machine Learning Frameworks
Data Analytics Tools