Fundamentals of ML

Question 1

What is ML?

Answer 1

• teach a computer model to make predictions and draw conclusions from data
• building computer systems that learn from data
• ML algorithms are trained to find relationships and patterns in data
• intersection of Data Science and Software Engineering
• Data Scientist: explore and prepare data, train ML model
• Software Engineer: integrate models in applications

Question 2

ML as a function

Answer 2

• A ML model is a software application that encapsulates a function to calculate an output value based on one or more input values
• Training = defining functions
• Inferencing = predict new values

Question 3

Steps of Training and Inference

Answer 3

1. Data = past observations
   - x = observed attributes / features
   - y = known value of prediction / label
   - x can be a vector of multiple features
2. Algorithm is applied to determine relationship between x and y
3. Result of algorithm is a model that encapsulates a calculation on x to calculate y
   - calculation is a function y = f(x)
4. Trained model can be used for inference
   - predictions are ŷ (y-hat)
   - rained models are used to draw conclusions from new data

Question 4

Types of ML

Answer 4

1. Supervised ML
   a) Regression
   b) Classification
   ba) binary classification
   bb) multiclass classification
2. Unsupervised ML
   a) Clustering

Question 5

Supervised ML

Answer 5

Training data with known features and values (= labeled dataset)
- most common type
- label can be anything from a category label to a real-valued number
- model learns a mapping between the input (features) and the output (label) during the training process
- once trained, model can predict the output for new, unseen data

Question 6

Common Examples for supervised ML

Answer 6

• linear regression for regression problems
• logistic regression for binary classification
• decision trees
• support vector machines for classification problems

Question 7

Unsupervised ML

Answer 7

Only features no known labels (= unlabeled dataset)
- Model finds patterns and relationships between features

Question 8

Common Examples of unsupervised ML

Answer 8

• Clustering (grouping similar data points together)
• Dimensionality reduction (reducing the number of random variables under consideration by obtaining a set of principal variables)
• k-means for clustering problems
• Principal Component Analysis (PCA) for dimensionality reduction problems

Question 9

Regression

Answer 9

Models are trained to predict numeric label values based on training data that includes both features and known labels
e.g. predicting ice-cream sales (y) based on temperature (x)

Question 10

Regression elements of training process

Answer 10

1. Split training data randomly (train and validate subset)
2. Use algorithm to fit data to a model
3. Validating by predicting values
4. compare actual labels to predictions
   / aggregate differences to calculate metric of accuracy

Question 11

Regression Evaluation Metrics

Answer 11

• MAE (Mean Absolute Error)
• MSE (Mean Squared Error)
• RMSE (Root Mean Squared Error)
• R2 (Coefficient of Determination)

Question 12

MAE

Answer 12

Mean absolute error
- Variance (by how many was each prediction wrong)
- doesn’t matter if + or -

Question 13

MSE

Answer 13

Mean squared error
- amplifies larger errors
- no longer represents quantity
- better to have a model that’s consistently slightly wrong than fewer but larger errors

Question 14

RMSE

Answer 14

Root mean squared error
- to represent quantity with squared error

Question 15

R2

Answer 15

Coefficient of determination
- proportion of variance in validation results
- natural random variance opposed to anomalous aspect
R2 = 1- ∑(y-ŷ)2 ÷ ∑(y-ȳ)2
ȳ = mean of actual value labels
- result between 0 and 1
- the closer to 1 the better the model is fitting the validation data

Question 16

Binary Classification

Answer 16

Calculates probability values for class assignments
observed item is or isn’t an instance of a specific class
e.g. predicting Diabetes yes or no

Question 17

Steps of Binary Classification

Answer 17

1. Algorithm calculates probability values for class assignments
2. Evaluation metrics compare predicted to actual classes
3. Probability is measured as a value between 0.0 and 1.0
4. Function describes probability of y being true (=1) for a given value x (f(x) = P (y=1|x)

Question 18

Classification evaluation metrics

Answer 18

• Confusion matrix
• Accuracy
• Recall (TPR true positive rate)
• Precision
• F1-Score
• AUC (area under the curve)

Question 19

Confusion Matrix

Answer 19

Matrix of number of correct and incorrect predictions for each possible class label.
columns = ŷ ( 0 and 1)
rows = y (0 and 1)
TN (true negative): ŷ=0 and y=0
FP (false positive): ŷ=1 and y=0
FN (false negative): ŷ=0 and y=1
TP (true positive): ŷ=1 and y=1
The arrangement of the confusion matrix is such that correct (true) predictions are shown in a diagonal line from top-left to bottom-right

Question 20

Accuracy

Answer 20

Proportion of right predictions
(TN+TP) ÷ (TN+FN+FP+TP)

Question 21

Recall

Answer 21

TPR true positive rate
Measures proportion of positive cases identified correctly
TP ÷ (TP+FN)
e.g. compared to patients with Diabetes, how many were identified correctly

Question 22

Precision

Answer 22

Proportion of predicted positive cases where label is actually positive
TP ÷ (TP+FP)
e.g. what proportion of predicted positive cases actually have diabetes

Question 23

F1- Score

Answer 23

Combines recall and precision
(2 x Precision x Recall) ÷ (Precision + Recall)

Question 24

AUC (Area under the curve)

Answer 24

Plotting an ROC (received operator curve)
Shows all TPRs and FPRs (true/ false positive rate) for all thresholds (decision point on yes or no)
Where straight line goes into curve = AUC
e.g. if AUC is 0.875 –> works better than random guessing (over 0.5)

Question 25

Multiclass Classification

Answer 25

Label represents one of multiple possible classes
Mostly mutually exclusive (e.g. penguin species) but multilabel classifications are possible (e.g. movie genres)

Question 26

Training-algorithms for a multiclass classification model

Answer 26

-OVR (One vs Rest)
-Multinominal algorithms

Question 27

OVR

Answer 27

One vs Rest
Trains binary classification for each class
Calculating the probability that the observation is an example of the target class

Question 28

Multinominal algorithms

Answer 28

Single function that returns a multi valued output
Output = Vector with probability distribution for all classes (all add up to 1)

Question 29

Clustering

Answer 29

Identify similarities between observations
No known cluster labels
Can be used to create classes for multiclass Classification

Question 30

Training Clustering Model

Answer 30

K-Means (most common)
1. Feature values vectorized to define n-dimensional coordinates
2. decide on k (= how many clusters), plot k at random coordinates (=centroids)
3. each data point is assigned to its nearest centroid
4. centroids moved to center of datapoints, based on mean distance between points
5. reassign data points to closest centroid
6. repeat 4. and 5. until stable or predetermined maximum iterations

Question 31

Evaluating a Clustering Model

Answer 31

How well are clusters separated?
- average distance to centroid
- average distance to other centroids
- maximum distance to centroid
- silhouette (ratio of distance between points in same cluster and points in different clusters, value between -1 and 1. The closer to 1 the better the cluster separation)

Question 32

Iterative Training

Answer 32

Vary:
- feature selection and preparation
- algorithm selection
- algorithm parameters

Question 33

Deep Learning

Answer 33

• tries to emulate how the brain learns
• creation of artificial neural network that simulates electrochemical activity in biological neurons by using mathematical functions
• each neuron = function, operates with input x and weight w
• multiple layers of neurons (DNN deep neural network, deeply nested functions)
• change w in iterations to reduce loss (ŷ compared to y)
• Data is batched into Matrices and processed using linear algebraic calculations -> good GPUs (Graphics processing unit) needed

Question 34

How does a neural network learn?

Answer 34

1. Training features are fed into the input layer (vector x)
2. Randomly assigned w , function combines x and w
3. If threshold is reached, activation function passes it on
4. each neuron is connected with each neuron in next layer (fully connected network)
5. Output layer produces a vector containing the calculated values for ŷ
6. Loss function to compare the predicted ŷ values to the known y values and aggregate the difference
7. Optimization function: use differential calculus to evaluate the influence of each weight on the loss, gradient descent to improve (each weight is in- or decreased)
8. New weights are backpropagated to the layers in the network, replacing the previously used values.
9. Multiple iterations (known as epochs) until the loss is minimized and the model predicts acceptably accurately

Question 35

Azure Machine Learning

Answer 35

Cloud service for training, deploying, and managing machine learning models
- Exploring data and preparing it for modeling.
- Training and evaluating machine learning models.
- Registering and managing trained models.
- Deploying trained models for use by applications and services.
-Reviewing and applying responsible AI principles and practices.