Chapter 6

Question 1

Parallel data processing

Answer 1

simultaneous execution of multiple sub-tasks that
collectively comprise a larger task. The goal is to reduce the execution time by dividing a
single larger task into multiple smaller tasks that run concurrently

Question 2

Distributed Data Processing

Answer 2

closely related to parallel data processing in that the same principle of “divide-and-conquer” is applied
distributed data processing is
always achieved through physically separate machines that are networked together as a cluster

Question 3

Hadoop

Answer 3

an open-source framework for large-scale data storage and data processing that is compatible with commodity hardware

Question 4

processing workload in Big Data

Answer 4

defined as the amount and nature of data that is

processed within a certain amount of time

Question 5

Workloads are usually divided into two types

Answer 5

• batch

* transactional

Question 6

Batch processing

Answer 6

involves processing data in batches
and usually imposes delays, which in turn results in high-latency responses. Batch
workloads typically involve large quantities of data with sequential read/writes and
–comprise of groups of read or write queries.
Queries can be complex and involve multiple joins. OLAP systems
Transaction workloads involve small amounts of data
with random reads and writes

Question 7

Transactional processing

Answer 7

known as online processing
data is processed interactively without delay,
resulting in low-latency responses.
OLTP and operational systems

Question 8

clusters

Answer 8

provide support to create horizontally scalable storage solutions

Question 9

benefit of clusters

Answer 9

comprised of low-cost commodity nodes that collectively provide increased processing capacity
–they provide inherent redundancy and fault tolerance, as they consist of physically separate nodes

Question 10

MapReduce

Answer 10

It divides a big problem into a collection of

smaller problems that can each be solved quickly

Question 11

Map tasks

Answer 11

• map
• combine (optional)
• partition

Question 12

Reduce tasks

Answer 12

• shuffle and sort

* reduce

Question 13

map stage

Answer 13

during which the dataset file is divided
into multiple smaller splits. Each split is parsed into its constituent records as a key-value
pair. The key is usually the ordinal position of the record, and the value is the actual
record.

Question 14

Processing in Realtime Mode

Answer 14

data is processed in-memory as it is captured before being persisted to the disk.

• -Response time generally ranges from a sub-second to under a minute.
• -Realtime mode addresses the velocity characteristic of Big Data datasets.

Question 15

Whereas the CAP theorem is primarily related to _____, the SCV principle is related to ______

Answer 15

distributed data storage; distributed data processing.

Question 16

Speed

Answer 16

Speed refers to how quickly the data can be processed once it is generated.
In the case of realtime analytics, data is processed comparatively faster than batch
analytics.

Question 17

Consistency

Answer 17

Consistency refers to the accuracy and the precision of the results.
Results are deemed accurate if they are close to the correct value and precise if close to each other.

Question 18

Volume

Answer 18

Volume refers to the amount of data that can be processed. Big Data’s velocity characteristic results in fast growing datasets leading to huge volumes of
data that need to be processed in a distributed manner

Question 19

Event Stream Processing (ESP)

Answer 19

incoming stream of events, generally from a single source and ordered by time, is continuously analyzed. The analysis can occur via simple queries or the
application of algorithms that are mostly formula-based

Question 20

Complex Event Processing (CEP)

Answer 20

a number of realtime events often coming from disparate sources and arriving at different time intervals are analyzed simultaneously for the detection of
patterns and initiation of action

Question 21

cannot use map reduce with realtime processing because

Answer 21

MapReduce cannot process data incrementally and can only process complete datasets.

Question 22

combine step

Answer 22

combiner looks for similar nodes and combines it

Question 23

partition

Answer 23

During the partition stage, if more than one reducer is involved, a partitioner divides the
output from the mapper or combiner
into partitions between reducer instances.

Question 24

shuffling and sort

Answer 24

shuffling -output from all partitioners is copied across the network to the nodes running the reduce task
sort - MapReduce engine automatically groups and sorts the key-value pairs according
to the keys so that the output contains a sorted list of all input keys and their values with
the same keys appearing together

Question 25

Reduce

Answer 25

reducer will either further summarize its input or will

emit the output without making any changes

Question 26

Task Parallelism

Answer 26

parallelization of data processing
by dividing a task into sub-tasks and running each sub-task on a separate processor,
generally on a separate node in a cluster

Question 27

Data Parallelism

Answer 27

parallelization of data processing
by dividing a dataset into multiple datasets and processing each sub-dataset in
parallel