Introduction to Big Data

Question 1

What is Big Data?

Answer 1

Extremely large volumes of data.

Question 2

What are the three factors to consider when designating something as Big Data?

Answer 2

Its volume, variety and velocity (the rate of change of data growth).

Question 3

What is variety?

Answer 3

Data that is considered unstructured, usually derived from unstructured digital content.

Question 4

What are the problems with Big Data?

Answer 4

Storage, Computational Efficiency, Data loss, Cost.

Question 5

What are the traditional solutions for Big Data?

Answer 5

Relational Database Management systems (RDBMS), Grid Computing, RAID Systems.

Question 6

What is RDBMS and what issues does it have?

Answer 6

Relational Database Management Systems (MySQL, PL/SQL, etc.). It has scalability issues (as the data gets bigger, so does the computational time). They are also designed to handle structured data.

**RDBMS are not horizontally scalable (you cannot improve performance by adding more computing power)

Question 7

What is grid computing and what are some of its drawbacks?

Answer 7

Putting computers in parallel and have a program run do computations on each data. While good for low volume, it is intensive on computational tasks.

** also requires experience in low level programming languages (not suitable for mainstream)

Question 8

What is RAID and what are some of the drawbacks?

Answer 8

Redundant Array of Independent Disks (RAID) systems were not designed to scale. As volume increases, so does cost, and though they have tried to be sold as scalable systems, their efforts have largely failed.

Question 9

What is Hadoop?

Answer 9

A framework for distributed computing.

Question 10

What are the two main components of Hadoop?

Answer 10

Hadoop Distributed File System (HDFS) -> storage solution
MapReduce -> Computation Solution

Question 11

What does the Hadoop Distributed File System (HDFS) do?

Answer 11

Takes care of all your distributed storage complexities.

• Splitting your data into blocks
• Replicating each block to more than one node
• Keep track of which block is stored in which node

Question 12

What is MapReduce?

Answer 12

A programming model implemented by Hadoop that takes care of all the computational complexities.

Question 13

What was Hadoop built to work on?

Answer 13

Commodity hardware.

**needs a machine that has a processor, hard disk and RAM.

Question 14

What was Hadoop built to work on?

Answer 14

Commodity hardware.

**needs a machine that has a processor, hard disk and RAM.

Question 15

What do you need to deal with Big Data?

Answer 15

Distributed computing platform.