Oefententamen

Question 1

What is the purpose of regularization in linear regression?

Answer 1

Regularization penalizes the size of weights to reduce overfitting by discouraging large weights. This results in a model that generalizes better to unseen data.

Question 2

Which regularization terms are commonly used in linear regression?

Answer 2

L1 regularization (Lasso): lambda * SUM |w_i| and L2 regularization (Ridge): lambda * SUM (w_i)^2. L1 encourages sparsity (some weights zero), while L2 encourages small weights.

Question 3

How can we find an optimal regularization weight in a regularized model?

Answer 3

Cross-validation techniques, like 10-fold or leave-one-out cross-validation, are used to determine the best regularization weight by measuring performance across different subsets of data.

Question 4

When should you increase or decrease the regularization parameter lambda in linear regression?

Answer 4

If the model is overfitting (too closely fits the data), increase lambda. If it is underfitting (too smooth, missing patterns), decrease lambda

Question 5

How can you determine independence of variables in a multivariate Gaussian distribution?

Answer 5

Variables X and Y are independent if their covariance Cov(X, Y) is zero. This is represented in the covariance matrix by having zero off-diagonal elements.

Question 6

What is an unbiased estimator?

Answer 6

An estimator “theta hat” is unbiased if its expected value equals the true parameter value: E[theta hat] = theta. This means it does not systematically overestimate or underestimate the parameter.

Question 7

What assumptions are made about the noise in linear regression?

Answer 7

Noise assumptions include: constant variance (homoscedasticity), independence of errors, and often, Gaussian (normally distributed) errors.

Question 8

Define conjugate prior in Bayesian inference.

Answer 8

A prior is conjugate if the posterior distribution is in the same family as the prior. This simplifies computation as the functional form of the distribution remains consistent.

Question 9

Explain Maximum Likelihood Estimation (MLE) in simple terms.

Answer 9

MLE finds parameter values that maximize the likelihood of observed data. It’s achieved by differentiating the likelihood with respect to the parameter, setting it to zero, and solving.

Question 10

How is uncertainty reflected in the Bayesian posterior distribution?

Answer 10

The Bayesian posterior reflects uncertainty due to unknown parameters and potential noise in both training and unseen data.