Jupyter Notebook 2.1 - PolyReg_learning_curves_bias-variance Flashcards
What is polynomial regression, and how is it related to learning curves and the bias-variance tradeoff?
Polynomial regression is an extension of linear regression that fits a curve to data that doesn’t follow a straight line. It introduces polynomial features (e.g., x², x³) to capture more complex patterns in the data.
Learning Curves: Used to check if your model is overfitting or underfitting by comparing training and validation performance as the model learns.
Bias-Variance Tradeoff: Balancing between a model being too simple (high bias) and too complex (high variance) is crucial for building models that generalize well to new data.
How does polynomial regression extend linear regression to fit complex data?
Polynomial regression allows us to fit a curve to data by adding polynomial features to the linear regression model. Instead of fitting a straight line, we transform our features into higher dimensions using polynomial terms. For example, instead of the simple linear equation:What does the polynomial regression look like?
This allows the linear regression model to fit more complex patterns in the data by considering higher-degree terms.
What are bias and variance in the context of machine learning models?
Bias and variance are two sources of error in machine learning models:
Bias: This refers to the error introduced by the simplifying assumptions made by the model. A high bias model may fail to capture the underlying patterns in the data, leading to underfitting (e.g., assuming linearity when the true relationship is quadratic).
Variance: This measures how sensitive the model is to fluctuations in the training dataset. A model with high variance may fit the training data very well but will not generalize to new data, resulting in overfitting. For example, a high-degree polynomial can adapt too closely to the training data, leading to high variance.
What is the bias-variance tradeoff in machine learning?
The bias-variance tradeoff is a fundamental concept in machine learning that helps optimize model performance:
High Bias: Oversimplified models tend to have high bias, leading to underfitting in both training and test data. As model complexity increases, bias decreases.
High Variance: More complex models tend to have high variance, fitting the training data well but failing to generalize to new data, leading to overfitting.
The goal is to find the sweet spot where the total error (bias, variance, and irreducible error) is minimized. To achieve this, one must balance bias and variance to minimize generalization error, typically estimated using validation data, since test set performance cannot be used for model selection.
