E10 Flashcards
1
Q
The main idea of the Lambda Architecture is…
A
To build Big Data systems as a series of layers
Each layer satisfies a subset of the desired properties and builds upon the functionality provided by the layers above it
2
Q
Batch Layer
A
- Manages the master dataset – an immutable, append-only set of raw data
- Pre-computes arbitrary query functions – called batch views
- Runs in a loop and continuously recomputes the batch views from scratch
- Very simple to use and understand
- Scales by adding new machines.
3
Q
Speed Layer
A
- Accommodates all requests that are subject to low latency requirements
- Does incremental computation instead of the recomputation done in the batch layer
4
Q
Speed Layer Goal
A
To have updated information on what happened since the last batch view was generated
5
Q
Serving Layer Goal
A
To merge views created by the batch layers with views created by the speed layer
6
Q
Difference Batch and Speed Layer
A
One big difference is that the speed layer only looks at recent data, whereas the batch layer looks at all the data at once
7
Q
Serving Layer
A
- Indexes batch views so that they can be queried with low latency
- The serving layer is a specialized distributed database that loads in a batch view and makes it possible to do random reads on it
- When new batch views are available, the serving layer automatically swaps those in so that more up-to-date results are available
- It does not need to support specific record updates
- > This is a very important point, as random writes cause most of the complexity in databases
8
Q
Storing data in raw format has many advantages:
A
- Data is always true (or correct): all records are always correct; no need to go back and re-write existing records; you can simply append new data
- You can always go back to the data and perform queries you did not anticipate when building the system
9
Q
Data should be stored in raw format, should be
A
- Immutable
- Kept forever
