Machine_Learning_Flash_Cards

Question 1

Keyword/Topic

Answer 1

Definition

Question 2

Encoding (One-Hot, Bag of Words)

Answer 2

Encoding techniques used to convert categorical data or text into numerical format. One-hot encoding represents each category as a binary vector. Bag of words represents text as a count of word occurrences.

Question 3

Colour Palettes

Answer 3

Diverging Color Palettes:
Used to show data that has a meaningful midpoint (e.g., positive vs. negative values). Colors transition from one hue to a neutral midpoint and then to another hue.

Sequential Color Palettes:
Designed for ordered data, where colors transition gradually from light to dark (or vice versa) to represent increasing values.

Categorical Color Palettes:
Used for distinct, non-ordered categories. Each category is represented by a unique color to ensure clear differentiation.

Question 4

CSV and HDF5

Answer 4

CSV is a plain-text format storing tabular data, while HDF5 is a binary format designed for large datasets, supporting hierarchical data structures.

Question 5

Stevens’ Scales

Answer 5

Nominal:
Categories with no inherent order
Ordinal:
Categories with a meaningful order, but intervals
Interval:
Ordered categories with equal intervals, but no true zero point (e.g., temperature in Celsius or Fahrenheit).
Ratio:
Ordered categories with equal intervals and a true zero point, allowing for meaningful ratios (e.g., height, weight, age).

Question 6

Spearman vs Pearson

Answer 6

Spearman measures rank correlation (monotonic relationships). Pearson measures linear correlation between two continuous variables.

Question 7

Data Wrangling

Answer 7

The process of cleaning and transforming raw data into a usable format for analysis, including handling missing values, filtering, and merging datasets.

Question 8

MAR, MCAR, MNAR

Answer 8

MCAR (Missing Completely At Random):
Missing data is random and unrelated to any variables.

MAR (Missing At Random):
Missing data depends on observed variables.

MNAR (Missing Not At Random):
Missing data depends on the missing values themselves.

Question 9

Misclassification

Answer 9

A classification error where a data point is assigned the wrong class label.

Question 10

Variance

Answer 10

A measure of the spread of data points. High variance indicates a model is sensitive to fluctuations in the training data.

Question 11

Bagging

Answer 11

An ensemble method that reduces variance by training multiple models on bootstrapped datasets and averaging their predictions.

Question 12

K-Means (Elbow Method)

Answer 12

A clustering algorithm. The Elbow Method helps determine the optimal number of clusters by plotting the sum of squared errors for different k values.

Question 13

Boosting

Answer 13

1.Initialize the model with equal weights for all data points.
2.Train a weak learner (e.g., decision tree).
3.Calculate errors and assign higher weights to misclassified points.
4.Train the next weak learner on the weighted data.
5.Combine all weak learners to form a strong model.

Question 14

Agglomerative Clustering

Answer 14

A bottom-up hierarchical clustering method where clusters are merged iteratively based on similarity.

Question 15

Cluster Assumption

Answer 15

Assumes data points within the same cluster share similar properties, foundational in unsupervised learning.
Single
Complete
Average

Question 16

Maximum Likelihood Estimator (MLE)

Answer 16

A method to estimate parameters by maximizing the likelihood function, representing the probability of observing the data given the model.

Question 17

Regression vs Classification

Answer 17

Regression predicts continuous values; classification predicts discrete class labels.

Question 18

Mean Square Error (MSE)

Answer 18

A loss function for regression that calculates the average squared difference between predicted and actual values.

Question 19

K-Fold Cross-Validation

Answer 19

A model validation method that splits data into k subsets, training on k-1 and validating on the remaining fold iteratively.

Question 20

Gradient Descent & SGD

Answer 20

Optimization algorithms: Gradient Descent minimizes loss by adjusting weights iteratively. Stochastic Gradient Descent updates weights using single examples.

Question 21

L1 and L2 Regularization

Answer 21

L1 adds a penalty proportional to the absolute value of weights (sparse solutions). L2 penalizes the square of weights (weight decay).

Question 22

K-Nearest Neighbors (KNN)

Answer 22

A classification/regression algorithm that predicts based on the majority vote or average of the k closest data points.

Question 23

Logistic Regression (Softmax)

Answer 23

A classification algorithm. Softmax generalizes logistic regression for multi-class classification.

Question 24

No Free Lunch Theorem

Answer 24

No algorithm is universally best for all problems. Performance depends on the specific dataset.

Question 25

Bias

Answer 25

Bias is the error introduced by approximating real-world problems with simplified models. High bias leads to underfitting.

Question 26

Overfitting

Answer 26

A model captures noise in the training data, performing poorly on new data. Regularization and validation sets help mitigate this.

Question 27

Validation Set

Answer 27

A subset of data used to tune model hyperparameters and assess performance during training.

Question 28

Testing Set

Answer 28

A hold-out dataset used to evaluate the final performance of a trained model.

Question 29

Reduce Variance in Trees

Answer 29

Techniques like bagging, random forests, and pruning reduce overfitting and variance in decision trees.