Machine Learning Basics

Question 1

What is Machine Learning?

Answer 1

The art and science of giving computers the ability to learn to make decisions from data (improve at a task based on experience) without being explicitly programmed.

Question 2

Name three types of Machine Learning categories

Answer 2

Supervised Learning
Unsupervised Learning
Reinforcement Learning

Question 3

What is Unsupervised Learning?

Answer 3

Uncovering hidden patterns from unlabelled data e.g. grouping customer into distinct categories (clusters) that were unknown before and are hopefully meaningful

Question 4

What is Reinforcement Learning

Answer 4

Software agents interact with an environment and try to find the most efficient pathway to a goal, learn how to optimise behaviour.

Given a system of rewards and punishments. Sucessful routes get reward, failures are restarted.

Question 5

What is Supervised Learning?

Answer 5

The Machine Learning model trains on sets (variables/features) of labelled training data (target variable) then predicts the labels (target variable) of subsequent testing datasets for unseen, often through multiple iterations.
Also called Predictive Data Analytics.

Question 6

What is Exploratory Data Analysis (EDA)?

Answer 6

Data analysis which performs initial explore of data, using mostly graphical techniques, to gain insight into nature of data and structure. What are important variables and outliers.

Question 7

Who codified EDA practice?

Answer 7

John Tukey in 1970s

Question 8

Complete the following …“Science does not begin with a tidy question …” (EDA)

Answer 8

“… nor does it end with a tidy answer”

Question 9

What did Tukey refer to EDA as?

Answer 9

Detective work

Question 10

Name some EDA techniques

Answer 10

Plot data norms: level of distribution, measures of central tendency: mode, median, mean

Range of spread of distribution: Standard Deviations, Percentiles, Quartiles

Relationships between variables/features in datasets/observations

Investigate trends for variables over time.

Question 11

Describe Data Wrangling

Answer 11

A process that occurs during the Data Preparation stage.

Take messy, incomplete data or data that is too complex and simplify and/or clean it so that it’s useable for analysis

Remove or impute missing values
Convert categorical to numeric
Standardise/Normalise data
Clean data
Join data together
Generate new fields

Overlap with Feature Engineering

Question 12

What is Feature Engineering?

Answer 12

Taking whatever information you have about your problem and turning it into a usable numeric format that you can use to build your feature matrix.

Question 13

How does Machine Learning work?

Answer 13

Use data to form a hypothesis, new data exposed errors in your hypothesis so the error gap is measured and hypothesis is adjusted to fit. Aim to get the error gap as low as possible.

Question 14

Name some types of Feature Engineering.

Answer 14

Converting Categorical features to numeric - could use one-hot encoding
Encode images to pixel representation
Impute missing data - fill Nan with mean of column
Build Feature Pipeline to chain together above tasks

Question 15

What do Machine Learning Algorithms do?

Answer 15

Algorithms learn a pattern inherent in existing data. These patterns can be used to make predictions about data that has not yet been analysed. This pattern, or model, is much smaller than the training data.

Question 16

Describe the Machine Learning lifecycle.

Answer 16

Derive pattern/data model using training data and algorithm
Check model using test data
Use formal process to check accuracy of model
Apply model to new data

Question 17

What is Dimensional Reduction in terms of Feature Engineering?

Answer 17

No of dimension too high = too long to process data/produce model
Some dimensions may not be of use

Can either just throw away dimension - use intuition
Employ Dimension Reduction techniques: Decision Trees, Principal Component Analysis (PCA)

Question 18

What is Principal Component Analysis (PCA)?

Answer 18

PCA is a feature extraction technique for reducing the dimension of a feature space (curse of dimensionality), so that there are fewer relationships between features to consider, and => less likely to overfit model.

Question 19

What is Clustering in terms of Unsupervised learning?

Answer 19

Finding islands of similarity in complex data sets.
Uniting singular points into distinct groups or clusters.
Examining data and assembling data points into sluters based on a measure of distance.

Question 20

Describe the K-Means Clustering algorithm

Answer 20

Unsupervised method utilising clustering.

1. Choose No of clusters (K) to be used by algorithm (Scree plot)
2. Randomly plot K cluster centre points as start position
3. Assign each point to nearest centriod
4. Update position of centriods to reflect new centre/average location of data points
5. Repeat 3+4 until no new data assignment occurs.

Question 21

Describe Association Rules algorithm?

Answer 21

Unsupervised method which uncovers how items are associated with each other e.g. shopping patterns. parent buys healthly ingredients for family; single males buy beer and crisps. Understanding patterns helps to increase sales.

Question 22

What are the two main types of Supervised Learning?

Answer 22

Regression - target (predicted) variable is continuous

Classification - target (predicted) variable consists of categories

Question 23

Describe K-Nearest Neighbours (K-NN) algorithm

Answer 23

Supervised Classification method that classifies a data point based on the classification of its neighbours. K indicates the No of nearest neighbour data points to include in majority voting process.
Choosing K is parameter tuning

Question 24

Describe Linear Regression algorithm

Answer 24

*** Supervised Regression method based on Linear Algebra. Line of best fit.

Use to predict continuous values
Regression implies that one of the variables is dependent on the other => different to correlation

Question 25

Describe Support Vector Machine (SVM) algorithm

Answer 25

Supervised Classification method to derive optimal boundaries separating groups.
Identify peripheral data points (support vectors) located closest to points from other group, then optimal boundary drawn down middle.
Fast method - less time required for computation as other support vectors used to derive boundary.
However, sensitive to position of support vectors

Question 26

What is Parameter Tuning in context of Machine Learning?

Answer 26

Parameters are options used to tweak an algorithm’s settings. Model’s accuracy suffers when not sufficiently tuned.
Overfitting - mistake random variations as persistent pattern
Underfitting - overlooks underlying patterns

Question 27

Describe Decision Tree algorithm

Answer 27

Supervised method that can be used for both Classification and Regression. => sometime referred to as CART (Class and Regress Tree).

A decision tree would predict survival chance through a series of binary questions (Yes/No type questions). Start at top question (root node), and move through tree branches as guided by response until you reach leaf node that indicates survival chance.

Question 28

Why is Evaluation required in Machine Learning?

Answer 28

To ensure that any model generated can do the job it has been built for.

1. Determine a model built for a particular task is most suited to that task
2. Estimate how the model will perform when deployed
3. Convince the business for whom model is being developed that it will meet their needs

Question 29

Is there more to evaluation than just measuring model performance?

Answer 29

Yes. For a model to be successfully deployed must consider issues such as:

how quickly model makes predictions
how easy it is for human analysts to understand predictions
how easy is it to retrain a model should it go stale

Question 30

How should we define ‘best’ performance when evaluating a model?

Answer 30

It depends on the context.
No model will ever be 100% perfect, but there are a range of ways in which models can be incorrect. For medical diagnosis we require very accurate results, and don;t predict a sick patient as healthy.
A model predicting which customers would response to an online ad may not be held to the same strict criteria.

Question 31

When evaluating models what is the most important rule?

Answer 31

Don’t use the same data sample to both train a predictive model and then to evaluate/test it

Question 32

When designing evaluation experiments what is hold-out sampling?

Answer 32

A hold out set is used to evaluate performance of a model, ensuring it was not used in training the model. Data is randomly allocated to a train and test set, and the performance measured on the test set should be a good indicator of how model will perform on future unseen data. Generally 70/30 split.

Question 33

When designing evaluation experiments what is K-Fold Cross Validation?

Answer 33

K-Fold validation is a re-sampling procedure used for limited data samples. Available data is split into K equal sized folds, and K separate evaluation experiments are performed. 1st time, data in the 1st fold is test set and remaining folds are for training. 2nd time, 2nd fold is test and so on.
The performance metrics from each run are aggregated to give a final score.
K=10 has been found to be a good value.

Question 34

What is a Confusion Matrix?

Answer 34

A table used to understand performance of a classification model (classifier) on a set of test data for which true values are known. Used as the basis for calculating performance measures.

It calculates frequency of each possible model prediction outcome from a test set.
For prediction problem with binary target feature there are four possible outcomes: TP, TN, FP, FN

Question 35

What is a True Positive?

Answer 35

Instance in test set that had a positive target feature value, and predicted to have a positive target feature value.

Question 36

What is a True Negative?

Answer 36

Instance in test set that had a negative target feature value, and predicted to have a negative target feature value.

Question 37

What is a False Positive?

Answer 37

Instance in test set that had a negative target feature value, and predicted to have a positive target feature value.

Question 38

What is a False Negative?

Answer 38

Instance in test set that had a positive target feature value, and predicted to have a negative target feature value.

Question 39

What is the order of a Confusion Matrix?

Answer 39

Predicted target value
pos neg
Actual Target pos TP FN
neg FP TN

FP = Type 1 error (False Predicted Positive)
FN = Type 2 error (False Predicted Negative)

Question 40

In Evaluation what are accuracy and misclassification rates (Error Rate)?

Answer 40

Accuracy: overall, how often is the classifier correct
Misclassification: overall, how often is it wrong - this is also called the Error Rate.

Accuracy calculated from (TP/TN)/ Total
Misclass = 1 - Accuracy

Question 41

In Evaluation what are problems with replying on accuracy rate?

Answer 41

The accuracy paradox. Sometimes accuracy is not a good measure for classifiers in predictive analytics. If there are far more instances of one category over another, say 99%, then predicting every element is in that category will have an accuracy of 99%.

This is known as class imbalance. If used for fraud prediction and only 1% of transactions are fraud, then accuracy of 99% correctly predicting non-fraud is useless.

Question 42

In Evaluation what is the Precision performance measure?

Answer 42

% calculated from confusion matrix, and tells us how confident that an instance predicted to have positive target level actually has a positive target level. - when it predicts positive how often is it correct.

Higher % indicates better performance.

TP / (TP + FP)

Question 43

In Evaluation what is the Recall performance measure?

Answer 43

% calculated from confusion matrix, and tells us how confident that all instances with a positive target level have been found by the model - when it’s actually positive, how often does it predict positive.

Also called TPR or sensitivity

Higher % indicates better performance.

TP / (TP + FN)

Question 44

In Evaluation what is the F1 performance measure?

Answer 44

Useful alternative to simple accuracy rate. Precision and Recall collapsed into single performance measure - harmonic mean of both measures.

Question 45

What is a classifier?

Answer 45

Another name for a classification model used in Supervised Learning

Question 46

Should you use terms True Positive/True Negative etc when more than two class in classifier?

Answer 46

Not recommended as it can be confusing.

Multi-class (> 2) problems can have confusion matrices and performance metrics, just better not to use TP/TN labels.

Question 47

In Evaluation what is the True Negative Rate (TNR)?

Answer 47

Tells us how confident that all instances with a negative target level have been found by the model - when it’s actually negative, how often does it predict negative.

TN / (TN + FP)

Question 48

What is a ROC curve??

Answer 48

Receiver Operating Characteristic Curve - ROC Curve.
Commonly used way to visualise performance of a binary classifier.

The curve is plotted using the prediction measurements of TPR (y axis) and FPR (x axis). Plotting performing for each value of the threshold used to separate classes.

Question 49

What constitutes a good ROC curve?

Answer 49

A curve that hugs the upper left hand corner.

Imagine a diagonal line bisecting the plot. A curve close to this line is poor and no better than random guessing.

Question 50

What is AUC?

Answer 50

Area under the curve (AUC) is a metric (single value) that can measures the area under the ROC curve and can be used to assess models. High value is good.

Question 51

What is the threshold used in classifier predictions?

Answer 51

Different classification models produce a prediction score for their target predictions, and a threshold is used to convert into the predicted classification target value. This threshold can be altered to impact the results of the prediction