Week 5

Question 1

Is the input to every reducer sorted?

Answer 1

yes

Question 2

What is the Jaccard coeffecient measuring

Answer 2

Question 3

what does a higher jaccard similarity mean

Answer 3

higher fraction of data is shared between P and Q

Question 4

Answer 4

Question 5

what is a simple deffeciency of jaccard similarity

Answer 5

Question 6

in cosine similarity, what are v and w

Answer 6

Question 7

how do you measure the weight of a token

Answer 7

term frequency - inverse document frequency. (tf-idf)

Question 8

term freqency equation

Answer 8

Question 9

Answer 9

1

Question 10

Answer 10

Question 11

what 2 things can cause the tf-idf to go up for a word

Answer 11

increases in the num of occurances within a document

if the word is more rare in the collection of documents

Question 12

Answer 12

Question 13

what is a safe n number for n grams in research

Answer 13

n=9

Question 14

What is an appliaction of near neighbor search

Answer 14

Question 15

what is a minhash of a set

Answer 15

Question 16

jaccard similarity of two sets in english

Answer 16

the size of their intersection divided by the size of their union

Question 17

how is minhash related to jaccard similarity

Answer 17

the prob(over all permutations of the rows) that h(c1) = h(c2) is the same as jaccard_sim(c1,c2)