Lecture 5: Big Data Architecture

Question 1

Core Principles of Big Data Architecture

Answer 1

• Scalability
• Fault-tolerance
• Optimization
• Modularity

Question 2

Lambda Architecture

Answer 2

• Batch layer: processes data in batches
• Speed layer: processes data in real-time
• Serving layer: merges outputs from batch and speed layer

Question 3

Kappa Architecture

Answer 3

• Stream processing layer: handles all data processing in real-time without a separate batch layer
• Storage layer: stores processed data for querying and analytics

Question 4

Data Lakehouse

Answer 4

• Combines the flexibility of data lakes with the performance and structure of data warehouses
• Unified storage layer: structured and unstructured data
• Query engine: allows for querying raw data

Question 5

Data Mesh

Answer 5

• Domain-oriented decentralization: each domain team is responsible for its data
• Self-service data infrastructure: provides tools and frameworks that allow teams to manage and use data independently

Question 6

Medallion Architecture

Answer 6

• Bronze layer: raw data
• Silver layer: filtered, cleaned, augmented data
• Gold: business-level data