Big Data self study

Question 1

MapReduce simplification apps?

Answer 1

Pig, Hive

Question 2

MapReduce data ingestion apps?

Answer 2

Flume, Sqoop

Question 3

MapReduce direct query apps?

Answer 3

HBase, Impala

Question 4

What is Hive?

Answer 4

• Hive is a data warehousing system
• Works with non-relational data
• HiveQL is used.
• Batch processing is done

Question 5

What is Pig?

Answer 5

• compiling a high-level scripting language, named Pig Latin, into MapReduce jobs for executing in Hadoop
• Pig Latin is scripting, therefore, procedural

Question 6

What is Flume?

Answer 6

• ingest data to HDFS
• harvesting large sets of data from server log files, like clickstream data from web server logs.
• can transform data while being harvested

Question 7

What is Sqoop?

Answer 7

• converting data back and forth between a relational database and the HDFS

Question 8

What is HBase?

Answer 8

• column-oriented NoSQL database
• highly distributed, designed to scale out
• Avoids the delays caused by batch processing
• interact with lower-level langs like Java
• used by FB for messaging system

Question 9

What is Impala?

Answer 9

• write SQL queries directly against the data while it is still in HDFS.
• heavy use of in-memory caching on data nodes

Question 10

What is YARN?

Answer 10

• Hadoop version 2
• separation of cluster resource management from jobs management.
• suitable for multitenancy