Revision Qs

Question 1

If a linear regression model was fitted to a logistic regression dataset, what could be the problems w.r.t. the Gauss-Markov properties?

Answer 1

• Linearity Assumption Violated
  Logistic regression assumes a sigmoidal (S-shaped) relationship, but linear regression assumes a straight-line relationship.
  As a result, predictions might be outside the [0,1] range, making them meaningless for classification.
• Homoscedasticity Violated
  In logistic regression, the variance of errors depends on the predicted probability–> heteroscedasticity.
• Normality of Errors Violated
  Linear regression assumes normally distributed residuals, but in logistic regression, residuals follow a Bernoulli distribution–>affects inference and statistical tests.

Question 2

Impact of a node in NNs

Answer 2

• Effect on Convergence
  More nodes → More parameters to train → Slower convergence due to increased complexity.
  Too few nodes → Underfitting, leading to faster but suboptimal convergence.
  Activation functions (e.g., ReLU, sigmoid) also influence how gradients propagate and whether the network suffers from vanishing or exploding gradients.

*Effect on Generalization
More nodes can improve the network’s capacity to learn complex patterns but may lead to overfitting.
Fewer nodes may result in underfitting, where the model fails to capture important relationships in the data.
Regularization techniques (dropout, L2 regularization) help balance complexity and generalization.