Delta

Question 1

What is it?

Answer 1

Delta Lake enables organizations to build Data Lakehouses, which enable data warehousing and machine learning directly on the data lake leverage super cheap cloud storage

Question 2

What format does Delta use and what file does it work on top of?

Answer 2

Table Format, Parquet files

Question 3

What problem does Delta solve?

Answer 3

Data Lake RELIABILITY (Corrupted tables , failed jobs)
Data Inconsistency
DATA LAKE PERFORMANCE - Slow queries

Question 4

Why will customers care about Delta?

Answer 4

RELIABILITY AND PERFORMANCE - Delta always guarantees a success view of data and responsive queries at scale.
OPEN FORMAT - No vendor lock-in (cloud agnostic)
COST SAVINGS - Same ability as DW only cheaper and opensource

Question 5

When should you position Delta with Customers?

Answer 5

1/ If a customer is building a lakehouse and want low latency
2/ If you are talking to data engineers tasked with data management , concerns of performance and reliability with data storage

Question 6

How does Delta Work?

Answer 6

Creates log file (transaction logs) with parque files, enables ability to track commits/change the dataset. Can understand what happens with dataset

Question 7

What are the Key underlying capabilities of Delta?

Answer 7

1/ACID transactions for object storage (Parquet) 
2/Delta logs as “source of truth”
3/Time travel 
4/Easy rollback
5/Metadata management at scale
6/Schema Enforcement

Question 8

What are common misconceptions of delta?

Answer 8

OpenSource vs Databricks Delta: Delta Lakes enhanced capabilities are only within Databricks and not in open source Delta. NOT TRUE anymore, we are open sourcing all of Delta with delta 2.0

Question 9

What are red flags when pitching Delta to customers?

Answer 9

1/Data warehouse users - Touch on openness of Delta and cost savings, flexibility, collaboration with Delta sharing, unstructured,structured.
2/Small datasets - Delta shines better in large volume datasets

Question 10

Who are Delta main competitors

Answer 10

Snowflake,BigQuery, Hudi,Redshift

Question 11

What are some questions to ask around competition?

Answer 11

1/How long do your current queries/jobs take to complete? Do you have SLAs to meet?
2/Do you own your data?
3/Do you have a storage layer thats optimal for BI and ML at the same time?

Question 12

How much does Delta Cost?

Answer 12

1/No extra Databricks costs outside of DBUs (compute usage)

2/Customers pay for storage on their accounts (S3)