Multi-Choice: Regression

Question 1

Q1 (Basics & Linear Regression). A sequence of points (x1,y1),…,(xN,yN) is described by yi = m xi + εi, εi ~ N(0, σ²). Which of the following is the maximum likelihood solution for m? (One correct choice)
1) mMLE = (Σyixi)/(Σxᵢ²)
2) mMLE = (Σyixi)/(Σxi)
3) mMLE = log[(Σyixi)/(Σxᵢ²)]
4) mMLE = sqrt[(Σyixi)/(Σxᵢ²)]

Answer 1

Correct item: 1. Explanation: Differentiating the log-likelihood leads to (Σxᵢyᵢ)/(Σxᵢ²).

Question 2

Q2 (Basics & Linear Regression). Suppose we have a prediction problem for an angle target t. We use L(y, t) = 1 - cos(y - t). The true angle is 3π/4, predictions y = wᵀx + b. Which are correct? (Two correct choices)
1) Max loss at y=5π/4
2) Max loss at y=7π/4
3) Min loss at y=3π/4
4) Gradient wrt b is x sin(wᵀx + b - t)

Answer 2

Correct items: 2 and 3. Explanation: The max occurs when y - t = π (i.e., 7π/4) and the min when y = 3π/4.

Question 3

Q3 (Basics & Linear Regression). We have two models g(x)=ax+b and h(x)=cx²+dx+f. Their losses are sums of squared errors. Which is correct? (One correct choice)
1) dL(g)/da=0 if a=(xy - x̄ȳ)/(x² - x̄²)
2) dL(g)/db=0 if b=(xy - x̄ȳ)/(x² - x̄²)
3) dL(h)/df=0 if f=ȳ - cx̄²
4) dL(h)/dd=0 if d=xy

Answer 3

Correct item: 1. Explanation: Solving dL(g)/da=0 yields a=(xy - x̄ȳ)/(x² - x̄²).

Question 4

Q4 (Basics & Linear Regression). We fit a model yi ~ N(log(w xi), 1). Which is true? (Two correct choices)
1) The likelihood is (2π)^(-N/2)*Π exp(-(yi - log(w xi))²/2)
2) The log-likelihood is (N/2) log(2π)+(1/2) Σ(yi - log(w xi))²
3) Adding -α||w||² to the log-likelihood and maximizing => w→0 as α→∞
4) Adding -α||w||² to the log-likelihood and minimizing => w→0 as α→∞

Answer 4

Correct items: 1 and 3. Explanation: i.i.d. normal form leads to the given likelihood; maximizing LL - α||w||² makes w shrink to 0 as α→∞.

Question 5

Q5 (Basics & Linear Regression). Which is true about linear regression? (Two correct choices)
1) Bayesian vs ML differ in how they model parameters
2) Maximizing the posterior with Gaussian prior & likelihood => regularized least squares
3) ML with Gaussian likelihood => regularized least squares
4) ML approach can use priors of parameters

Answer 5

Correct items: 1 and 2. Explanation: ML and Bayesian differ in parameter modeling; the MAP with Gaussian prior is equivalent to a regularized least squares.

Question 6

Q6 (Basics & Linear Regression). We have 10 coin tosses (heads/tails). H ~ Binomial(10, θ). The prior on θ is Beta(α, β). Which is true? (One correct choice)
1) Posterior is binomial
2) Posterior is Beta
3) Posterior is same as prior distribution

Answer 6

Correct item: 2. Explanation: Beta is conjugate to the binomial, so the posterior remains Beta.

Question 7

Q7 (Basics & Linear Regression). In Bayesian linear regression with Gaussian priors and Gaussian likelihood, which are correct? (Two correct choices)
1) Posterior of parameters is Gaussian
2) Always possible to compute posterior for any prior
3) We can approximate the posterior using Laplace approximation

Answer 7

Correct items: 1 and 3. Explanation: Gaussian prior + Gaussian likelihood => Gaussian posterior; or approximate using Laplace if no conjugacy.

Question 8

Q8 (Basics & Linear Regression). About the choice of priors in Bayesian learning, which are correct? (Three correct choices)
1) Posterior is independent of prior
2) Prior choice can matter computationally
3) Prior can encode prior knowledge
4) With few data, prior is not important
5) With few data, prior is more important

Answer 8

Correct items: 2, 3, and 5. Explanation: Prior choice affects computational complexity, can reflect prior knowledge, and is especially critical with sparse data.

Question 9

Q9 (Basics & Linear Regression). Leave-one-out cross-validation with 1000 data points is done for 3 different regression models. How many trainings total? (One correct choice)
1) 9000
2) 1000
3) 2997
4) 3000
5) 999
6) 997

Answer 9

Correct item: 4. Explanation: Each model is trained 1000 times; 3 models => 3000 trainings.

Question 10

Q10 (Basics & Linear Regression). The process of training a model with provided target variables is called? (One choice) 1) Clustering
2) Unsupervised learning
3) Supervised learning
4) Parameter selection
5) Model prediction

Answer 10

Correct item: 3. Explanation: Training with given targets is supervised learning.