Mapreduce

Question 1

What is the MapReduce Programming Model’s Data type?

Answer 1

Key-value records

Question 2

What is the MapReduce Programming Model’s map function?

Answer 2

(Kin, Vin)–>list(Kinter, Vinter)

Question 3

What is the MapReduce Programming Model’s reduce function?

Answer 3

(Kinter, list(Vinter))–>list(Kout, Vout)

Question 4

Check notes for wordcount example

Answer 4

Check notes for wordcount example

Question 5

What is Apache Hadoop MapReduce?

Answer 5

An open source implementation of Google’s MapReduce framework.
There are two ways to write jobs:
–Java
–Hadoop Streaming(for Python, Perl, etc)

Question 6

Explain Google PageRank.

Answer 6

The PageRank algorithm rates linked documents (web pages)
It is the basis of the Google search engine for the ranking of web pages.
Principle: The numerical weight (PageRank) PRp of a web page p depends on the number and the numerical weight of the web pages, which link to p.

Question 7

What is the PageRank algorithm?

Answer 7

Go to notes

Question 8

Parallelisation improves computation efficiency but what are some challenges it has faced?

Answer 8

Managing multiple servers is difficult because servers need coordination and server failure should not affect job execution.
Achieving parallelisation is difficult because tasks must be made fully independent. It’s hardware to rationale program irregularities (debugging).