Classification

Question 1

What is supervised learning

Answer 1

Supervised learning is an approach to machine learning where a computer algorithm is trained on labelled input data to predict an output or target variable. The algorithm learns from the labelled data to identify underlying patterns and relationships between the input and output variables

Question 2

What is unsupervised learning

Answer 2

Unsupervised learning is a machine learning approach that involves analysing and clustering unlabelled datasets. The algorithm used in unsupervised learning attempt to identify patterns and relationships in the data without the need for human intervention. Unsupervised learning can help to reveal hidden data grouping or patterns

Question 3

What is reinforcement learning

Answer 3

Reinforcement learning is a feedback-based machine learning technique in which an agent learns to make decisions by interacting with its environment. The agent receives feed back in the form of rewards or penalties for its actions and adjust its behaviour to maximise the rewards it receives. Through trial and error, the agent learns to make better decisions and achieve better outcomes over time

Question 4

What are some common challenges associated with unsupervised learning

Answer 4

One of the main challenges associated with unsupervised learning is that the absence of labelled data makes it difficult to evaluate the performance of the algorithm. Another challenge is the potential for the algorithm to identify spurious patterns (occurs when two factors appear causally related to one another but are not), which can lead to incorrect conclusions. Researchers attempt to overcome these challenges by using various techniques, such as clustering validation metrics and visualisation to evaluate the algorithm’s performance and identifying and removing outliers in the data

Question 5

What are the two types of supervised learning

Answer 5

Regression and Classification

Question 6

What is a classification problem in machine learning

Answer 6

A classification problem is a type of supervised learning problem in which the goal is to predict a categorical label for the output variable based on input variables

Question 7

What are some characteristics of a classification problem

Answer 7

The output variable is categorical, meaning it can take on a limited number of possible values. The input variables do not need to be categorical and can be a combination of text and numeric features

Question 8

What are some common algorithms used in classification problems

Answer 8

See common algorithms used in classification problems include decision trees, logistic regression, support vector machines (SVMs), and neural networks

Question 9

How do you evaluate the performance of a classification model

Answer 9

The performance of a classification model can be evaluated using metrics such as accuracy, precision, recall, and F1 score. These metrics measure how well the model correctly predicts the different classes in the dataset

Question 10

What are some challenges associated with classification problems

Answer 10

Some common challenges with classification problems include imbalanced datasets, where one class has significantly more data than the other, and overfitting, where the model fits the training data too closely and performs poorly on new, unseen data

Question 11

What is the objective of a linear regressor, and how does it identify the intercept and slope of a linear equation

Answer 11

The objective of a linear regressor is to find the best-fitting line that can represent the relationship between an input variable and an output variable. It does so by minimizing the average squared error between the predicted and actual output values. The intercept and slope of the linear equation can be derived from the parameters that minimize the error

Question 12

How does a perceptron or a single neuron work, and how it related to a linear model

Answer 12

A perceptron or a single neuron is a basic building block of a neural network that takes one or more inputs, applies weights to them, and produces an output based on the weighted sum. In a binary classification problem like the one described in the text, the output of the perceptron can be thresholded to produce a binary decision. A perceptron is a linear model in the sense that the output is a linear function of the inputs and weights, although it can be combined with non-linear activation functions to learn more complex patterns

Question 13

What are the steps to apply classification to a linear regressor model (using example from notes)

Answer 13

Step 1: We can fit a linear regressor with output labels being 0 and 1. This is a straightforward approach where we treat the classification problem as a regression problem. Here, the intercept is c = -0.74 and m = 0.33
Step 2: Use thresholding to classify the inputs into different categories. For example, if we decide that if our prediction is below the green line (at 0.5) then our predicted label would be purple, i.e. lightweight. On the other hand, if the prediction is above this threshold, then we would label the associated output as yellow, or heavyweight
Step 3: We need to evaluate the performance of our model. We can use metrics like accuracy, precision, recall, F1-score, tec, to evaluate the performance of our model
Step 4: We can tune the model by adjusting the threshold value, changing the regularisation parameter, using different optimisation techniques, or selecting different features. This step is to improve the performance of the model
Step 5: We can use our trained model to predict the labels for new instances. We can use the same threshold value we have during the training phase to classify the new instances into different categories

Question 14

Why is it important to evaluate the performance of a machine learning model

Answer 14

It is important to evaluate the performance of a machine learning model to determine how well it is able to generalise to new, unseen data. This helps in identifying and addressing potential issues with the model, improving its accuracy and reliability

Question 15

What is overfitting in machine learning

Answer 15

Overfitting is a common problem in machine learning where a model is trained to fit the training data too closely, resulting in poor generalisation to new, unseen data. This can happen when the model is too complex or when there is insufficient data to train the model

Question 16

What are hyperparameters in machine learning

Answer 16

Hyperparameters are parameters that are set by the user before the model is trained, and they control the behaviour of the learning algorithm. Examples of hyperparameters include the learning rate, regularisation strength, number of hidden layers in a neural network, etc.

Question 17

What is cross-validation in machine learning

Answer 17

Cross-validation is a technique used to evaluate the performance of a machine learning model. It involves dividing the data into multiple folds, and then training the model on one fold and testing it on another, and repeating this process for each fold. This helps in obtaining a more reliable estimate of the model’s performance on new, unseen data

Question 18

What is the Sigmoid function and how is it useful in prediction models

Answer 18

The Sigmoid function is a mathematical function that takes an input and limits its output between the range of [0,1]. It is useful in prediction models because it can convert any input into a probability score, which can be interpreted as the likelihood of a particular outcome

Question 19

How does the Sigmoid function relate to logistic regression

Answer 19

Logistic regression is a type of statistical analysis that is used to predict the probability of a binary outcome (e.g. yes/no, pass/fail). It is based on the Sigmoid function, which maps any input value to a probability score between 0 and 1

Question 20

What is the Sigmoid function equation

Answer 20

f(x) = 1 / (1 + e^(-t)), where t is the input to the function (t = mx + c), and e is the mathematical constant e, which is approximately equal to 2.71828.

Question 21

What happens when we replace t with a linear equation in the Sigmoid function

Answer 21

When we replace t with a linear equation in the Sigmoid function, we get a logistical function. The logistical function also has an S-shaped curve, but it has an upper and lower limit, which makes it more suitable for binary classification tasks

Question 22

What is Logistic Regression

Answer 22

Logistic regression is a statistical technique used to analyse and model the relationship between a categorical dependent variable and one or more independent variables

Question 23

What is the purpose of the transformation function in Logistical Regression

Answer 23

The transformation function in Logistic Regression is used to map the output of the linear regression to a probability value between 0 and 1

Question 24

How is the error calculated in Logistical Regression

Answer 24

The error in logistical regression is calculated using the difference between the predicted probability and the actual outcome of the dependent variable for each data point

Question 25

How is the error minimised in logistic regression

Answer 25

The error in logistic regression is minimised by adjusting the parameters of the regression model, such as the slope and intercept , to reduce the overall difference between the predicted probabilities and the actual outcomes for all data points

Question 26

What is the typical threshold used in Logistic Regression

Answer 26

The typical threshold used in Logistic Regression is 0.5, which has a probabilistic meaning and is used to determine the predicted class based on the predicted probability

Question 27

What is multi-input linear equation in Logistic Regression

Answer 27

Multi-input linear equation in Logistic Regression refers to the use of more than one independent variable in the regression model to predict the probability of the dependent variable

Question 28

What does the Logistic Regression Model look like

Answer 28

Check Notes

Question 29

What does a single perceptron classifier look like

Answer 29

Check notes

Question 30

What are Activation Functions

Answer 30

Activation Functions are mathematical functions applied to the output of a regression model, such as neural network, to introduce non-linearity into the model and produce a desired output

Question 31

What is the purpose of Activation Functions

Answer 31

The purpose of Activation Function is to introduce non-linearity into the model, which is necessary for the model to learn complex patterns and relationships in the data. Activation Functions also moderate the output from the regressor to produce a desired output

Question 32

What are some examples of Activation Functions?

Answer 32

Some examples of Activation Functions include Sigmoid, ReLU (Rectified Linear Unit), Tanh (Hyperbolic Tangent), Softmax, and Leaky ReLU.

Question 33

What does a Multi-Layer Perceptron look like

Answer 33

Check Notes

Question 34

What is a confusion matrix

Answer 34

A confusion matrix is a table used to evaluate the performance of a classification model by presenting the number of correct and incorrect predictions in a tabular form

Question 35

What is the purpose of a confusion matrix

Answer 35

The purpose of a confusion matrix is to provide a clear representation of the performance of a classification model on a given dataset by presenting the true positives, true negatives, false positives, and false negatives

Question 36

What are the components of a confusion matrix

Answer 36

The components of a confusion matrix include true positive, false positive, true negative, false negative

Question 37

How is a confusion matrix used to compute different performance metrics

Answer 37

A confusion matrix is used to calculate various performance metrics such as accuracy, precision, recall, F1 score, and AUC(Area Under the Curve) by analysing the different components of the matrix. The metrics help to evaluate the performance of a classification model accurately

Question 38

What is a True Positive

Answer 38

It represents the number of correct positive predictions

Question 39

What is a False Positive

Answer 39

It represents the number of incorrect positive

Question 40

What is a True Negative

Answer 40

It represents the number of correct negative predictions

Question 41

What is a False Negative

Answer 41

It represents the number of incorrect negative predictions

Question 42

What is a False Positive Rate (FPR)

Answer 42

FPR measures the proportion of actual negative instances that are predicted as positive. It is calculated as FP/(FP + TN)

Question 43

What is a False Negative Rate (FNR)

Answer 43

FNR measures the proportion of actual positive instances that are predicted as negative. It is calculated as FN/(FN + TP)

Question 44

What is Recall

Answer 44

Recall measures the proportion of positive instances that are correctly identified out of all positive instances. It is calculated as TP/(TP + FN)

Question 45

What is Precision

Answer 45

Precision measures the proportion of positive instances that are correctly identified. It is calculated as TP/(TP+FP)

Question 46

What is F1 score

Answer 46

F1 score is the harmonic mean of precision and recall, and it is a measure of the trade-off between precision and recall. It is calculated as 2 * [ (precision * recall)/(precision + recall) ]

Question 47

What is an imbalanced dataset

Answer 47

An imbalanced dataset is a type of dataset where the number of examples in each class is not equal or roughly equal. One or more classes have significantly fewer samples than other classes in the dataset

Question 48

How does an imbalanced dataset affect accuracy

Answer 48

When dealing with an imbalanced dataset, accuracy may not be an appropriate measure of model performance. For instance if there are 200k examples of the purple class and only 20 examples of the yellow class, a model that predicts all examples as purple will have a high accuracy of 99.99%, but it will fail to identify any yellow examples. In other words, the model’s performance is highly biased towards the majority class

Question 49

What are the consequences of making large errors in the minority class in an imbalanced dataset

Answer 49

In an imbalanced dataset, making large errors in the minority class has little impact on the overall accuracy of the model because the minority class contributes very little to the total number of examples. However, it can have serious consequences in real-world scenarios. For example, in a medical diagnosis system, if the model fails to identify a rare disease it can have life-threatening consequences for the patient

Question 50

What are some alternative metrics that can be used to evaluate model performance in an imbalanced dataset

Answer 50

To evaluate model performance in an imbalanced dataset, some alternative metrics that can be used include precision, recall, F1-score, AUC-ROC (Area Under the Receiver Operating Characteristic Curve), and PR-AUC (Precision-Recall Area Under the Curve). These metrics provide a better understanding of the model’s performance, especially in identifying examples of the minority class

Question 51

What are Performance Metrics in Machine Learning

Answer 51

Performance Metrics are quantitative measures used to evaluate the performance of a model in terms of its accuracy , precision, recall and other metrics. These metrics are used to determine how well a model is able to predict outcomes based on the input data

Question 52

What is Accuracy as a Performance Metric

Answer 52

Accuracy is used to measure the overall correctness of a model’s predictions, regardless of whether the predictions are positive or negative. It is calculated by dividing the number of correct predictions by the total number of predictions

Question 53

What is Sensitivity as Performance Metric

Answer 53

Sensitivity is used to measure the proportion of positive cases that are correctly identified by the model. It is also known as the True Positive Rate and is calculated by dividing the number of true positives predictions by the total number of actual positive cases

Question 54

What is Specificity as a Performance Metric

Answer 54

Specificity is used to measure the proportion of negative cases that are correctly identified by the model. It is also known as True Negative Rate (TNR) and is calculated by dividing the number of true negative predictions by the total number of actual negative cases

Question 55

What is False Positive Rate (FPR) as a Performance Metric

Answer 55

False Positive Rates is a performance metric used to measure the proportion of negative cases that are incorrectly identified as positive by the model. It is calculated by dividing the number of fake positive predictions by the total number of actual negative cases, and is given by 1- TNR

Question 56

What is Informedness as a Performance Metric

Answer 56

Informedness is used to evaluate a model’s performance on both balanced and imbalanced data. It is defined as the sum of the True Positive Rate (TPR) and True Negative Rate (TNR) minus 1, and ranges from -1 to 1, with a higher value indicating better performance. 1 = TPR + TNR - 1

Question 57

What happens if the threshold is varied in a Machine Learning model

Answer 57

Varying the threshold can have an impact on its performance. The threshold determines the point at which the model makes a positive prediction, and changing it can affect the model’s accuracy, sensitivity and specificity

Question 58

How does changing the threshold affect the model’s performance

Answer 58

It can affect its performance in different ways depending on the task and the data. For example, if the threshold is lowered, the model may predict more positive cases, resulting in a higher sensitivity but lower specificity. Conversely , if the threshold is raised, the model may predict fewer positive cases, resulting in a higher specificity but lower sensitivity

Question 59

How can we track the impact of changing the threshold on a model’s performance

Answer 59

It can be tracked by plotting the TPR against the FPR for different threshold values. This is known as the Receiver Operating Characteristic (ROC) curve. The area under the ROC curve can be used as a performance metric, with a higher AUC indicating better performance. By analysing the ROC curve, we can determine the optimal threshold for the model based on the specific trade-offs between sensitivity and specificity

Question 60

What is the Receiver Operating Characteristic (ROC) curve

Answer 60

The Receiver Operating Characteristic (ROC) curve is a graphical representation of the performance of a binary classification model, which shows the trade-off between the TPR and FPR for different thresholds

Question 61

What is a Decision Tree

Answer 61

A Decision Tree is a tree-like model used for making decisions and predicting outcomes based on a series of conditions or features. It is a machine learning algorithm that recursively partitions the data based on the values of its input features, and makes a series of binary decisions untiol a prediction is made

Question 62

How does a Decision Tree work

Answer 62

A decision tree starts at the root node, which represents the entire dataset. It then recursively splits data into smaller subsets based on the values of the input features, such that each split maximally separates the data into different classes. At each split, the algorithm selects the feature that best separates the data into different classes, based on a certain criterion (e.g. information gain, Gini impurity). This process continues until a stopping criterion is met, such as a minimum number of samples at a leaf node or maximum depth of the tree

Question 63

What are the advantages of using a Decision Tree

Answer 63

Some advantages include its simplicity, interpretability, and ability to handle both categorical and numerical data. Decision Trees can also handle missing values and outliers, and are relatively fast and scalable for larger datasets. Additionally, they can be used for both classification and regression tasks, and can be used in ensemble methods such as Random Forests to improve their accuracy. Requires little data preparation, The cost of using the tree is logarithmic in the number of data points used to train the tree, able to handle multi-output problems.

Question 64

What are the limitations of using a Decision Tree

Answer 64

Some limitations of using Decision Trees include its tendency to overfit the data, especially when the tree is deep or the dataset is noisy. Decision Trees are also sensitive to the choice of hyperparameters and the order of the input features. Additionally, Decision Trees may not perform well on datasets with complex interdependencies among the input features or when the classes are imbalanced

Question 65

How to create a decision tree from a dataset

Answer 65

Step 1: For each feature, build a sub-tree by sorting the feature values and computing the average of every consecutive pair. For each average, build a sub-tree using it as a threshold and computer the “Gini” impurity for each resulting leaf. They compute the weight of each leaf and total Gini impurity using a weighted sum. Finally select the sub-tree with the minimum Gini impurity
Step 2: Select the sub-tree with the lowest Gini impurity as the head of the decision tree
Step 3: If there are impure leaves, replace them with one of the other sub-trees after updating the Gini impurity within the subset. Continue until all leaves are pure, or the maximum depth has been reached

Question 66

What is the Gini impurity

Answer 66

The measure of the degree of impurity or randomness in a set of categorical outcomes or classifications. It ranges from 0 (when all the elements in a set belong to the same category) to 1 (when the elements in a set are evenly distributed across all categories). In decision tree algorithms, the Gini impurity is used to evaluate the quality of a split in the data by measuring the degree of homogeneity of the resulting subsets after the split. The lower the Gini impurity, the better the split

Question 67

What is the Gini impurity Formula

Answer 67

I = 1 - P(+ve)^2 - P(-ve)^2, where P(+ve) and P(-ve) are the proportions of positive and negative instances in the dataset, respectively

Question 68

What is the formula to compute the weight of a leaf

Answer 68

Weight of a leaf = (Number of instances belonging to the leaf) / (Total number of instances in the dataset)