Lecture 12 Flashcards
What is model class selection in machine learning?
The process of choosing the appropriate model type for a given problem, such as classification, regression, or clustering.
What is the purpose of a hyperplane in classification?
A hyperplane separates different classes in a feature space, with its dimension being one less than the number of features.
What is the maximum margin hyperplane?
A hyperplane that maximizes the margin between different classes, used in Support Vector Machines.
Why is feature selection important in machine learning?
Choosing relevant features improves model accuracy and reduces overfitting.
What is overfitting?
When a model is too complex and fits the training data too well, leading to poor generalization on new data.
What is underfitting?
When a model is too simple and fails to capture patterns in the training data, leading to poor performance.
What is a design matrix?
A matrix representing all features of a dataset, allowing transformations for improved model performance.
What is the role of mean squared error (MSE) in regression?
MSE measures the average squared difference between predicted and actual values, serving as a loss function.
What is Occam’s Razor in model selection?
The principle that the simplest model that fits the data should be preferred.
Why do we use high-dimensional feature spaces?
Transforming data into higher dimensions can make it linearly separable, improving classification performance.
What is gradient descent?
An optimization algorithm used to find the minimum of a function by iteratively updating parameters.
What is the difference between local and global minima?
A local minimum is a point where the function is lower than nearby points, while a global minimum is the lowest possible value.
How does stochastic gradient descent (SGD) differ from regular gradient descent?
SGD updates parameters using a single random data point or a small batch, making it faster but less stable.
What is convex optimization?
A type of optimization problem where the loss function has a single global minimum, making it easier to solve.
How do we prevent a model from getting stuck in local minima?
By using techniques like momentum, learning rate adjustments, or trying different starting points in gradient descent.
What is the difference between classification and regression?
Classification predicts categorical labels, while regression predicts continuous values.
How do we evaluate a regression model?
Using metrics like mean squared error (MSE) to measure prediction accuracy.
What is the purpose of a tangent line in optimization?
It represents the slope of a function at a point, helping determine the direction for optimization.
Why is feature engineering important in machine learning?
It helps create informative features that improve model accuracy and efficiency.
What is the bias-variance tradeoff?
The balance between underfitting (high bias) and overfitting (high variance) in a model.
What does it mean for a model to be consistent?
A model is consistent if it accurately represents the observed data and generalizes well to new examples.
What is a loss function in machine learning?
A function that measures how far a model’s predictions are from actual values, guiding optimization.
How can increasing the number of features affect model performance?
It can improve accuracy but may also lead to overfitting if not managed properly.
What is the main advantage of using higher-dimensional spaces in classification?
It can make previously inseparable data linearly separable.
What is the fundamental goal of machine learning model selection?
To find the simplest model that generalizes well to unseen data while maintaining accuracy.
