Exam stuff

Question 1

What is the worst method for initializing the factors in matrix factorization?

Answer 1

Setting all the factors to the same amount. The gradient of the error function will then be the same for all factors, effectively meaning we train the model with a single factor.

Question 2

What is regularization?

Answer 2

A term added to prevent overfitting. Preventing the network to memorize the training data. The more data we have the lower the risk of overfitting

Question 3

How is the similarity vs prob of detection plot changed when the number of bands b is changed but b*r stays constant? When only r changes?

Answer 3

The central point s_0 moves to the left the more b increases and p = 1 is reached at lower similarity. The central point s_0 moves more to the right when only r is increased.

Question 4

Let us assume that a Matrix Factorization model with K factors has been trained for N users and M items. How much memory is needed to store the model parameters assuming each parameter is stored with P bytes? How large is P?

Answer 4

P with normal precision is 4 bytes (8 with double precision). The parameters need P(NK + MK) in memory.

Question 5

How much memory is taken by the signature matrix?

Answer 5

Each element of sig matrix is -1 or 1. So each entry is 1 bit. So M = random projections * no of documents.