Week 6B (large scale data analysis using MR)

Question 1

at the abstract level, how do we modify the calculation of pagerank to avoid spider traps

Answer 1

allow each rand surfer a small possibility of teleporting to random page. taxation

Question 2

what is this equation used to do

Answer 2

compute new vector estimate of pageranks v’ from the current pagerank estimate v and the transition matrix M

it is the taxation equation

Question 3

in this equation what is B usually chosen as and what is it

Answer 3

0.8-0.9

the probability that the random surfer decides to follow an outlink from their present page

Question 4

in this equation, what is e

Answer 4

a vector for all 1’s with the approprtiate number of components

Question 5

in this equation, what is n

Answer 5

num of nodes in web graph

Question 6

in this equation, what case does the term BMv represebt

Answer 6

with probability B, the random surfer decides to follow an outlink from their present page

Question 7

in this equation, what data structure does (1-B)e /n evaluate to

Answer 7

a vector

Question 8

in this equation, what does (1-B)e /n represent

Answer 8

the introduction with probability of 1-B of a new random surfer at a random page

Question 9

in a graph what is a clique

Answer 9

set of nodes with all possible arcs from one to another

Question 10

is there a dead end or spider trap

Answer 10

dead end: E

spider trap: No, having a clique does not mean there is spider trap

Question 11

we do recursive deletion, what would be the pagerank assigned to each of the nodes

Answer 11

Question 12

2 basic steps for search engine when generating their final results

Answer 12

select candidate pages from query

among qualified pages, Page rank score computed for each

Question 13

how do we reduce the amount of data that must be passed from the Map tasks to Reduce tasks during pagerank calulation

Answer 13

M is a very sparse matrix, represent it by only its nonzero elements

Question 14

what is a spam farm

Answer 14

collection of pages whose purposes is to increase pagerank of certain page or pages

Question 15

what does the graph of a spam farm look like

Answer 15