10. Machine Learning Flashcards
Machine Learning and Artificial Intelligence (Definition Machine Learning; ML, DL and AI)
Machine Learning means to use learning algorithms on (big) data to make accurate predictions and detect previously unknown patterns.
Machine Learning, Deep Learning and AI have different meanings. Not every system categorized as an intelligent system uses machine learning.
Types of Machine Learning Algorithms (3)
- Supervised Learning
- Unsupervised Learning
- Reinforced Learning
ML-Algorithm: Supervised Learning (Definition, Examples)
Supervised Learning = having a lot of data, where each dataset has a specific label (= target class/Target value)
Examples: Email Spam Detection, Handwriting Recognition, Medical Diagnosis,..
ML-Algorithm: Unsupervised Learning (Definition, Examples)
Unsupervised Learning = having a dataset but no labeled classes -> with ML we want to learn about the raw data.
Examples: Clustering, Recommender Systems, Risk Factor Analysis
ML-Algorithm: Reinforced Learning
Reinforced Learning = the algorithm tries something random and waits for feedback -> if feedback is positive, the algorithm learns that it was doing the right thing
=> incremental learning by iteratively trying different actions and processing the feedback.
Examples: Games, Traffic Light Control, Robotics
Supervised Learning: Regression
Make quantitative (real valued) predictions on the basis of a (vector of) features or attributes => reveals causal relationship between the independent variables (input) and dependent variables (output)
Example: Bundesliga. Estimating the position of a team based on the scored goals.
Supervised Learning: Classification (Goal)
Goal: Use training data to build a classification model that predicts the correct category (label) for previously unknown data with high accuracy.
Supervised Learning: Decision Trees (Method, How to build one)
Create a model that predicts the value of an output target variable at the leaf nodes of the tree, based on several input variables at the root and interior nodes of that tree.
How to build a decision tree:
- Trees are built from the root to the leaves. In each iteration, one further attribute is defined. The decision tree learning algorithm selects the attribute which provides the most information gain (IG)
- IG is a measure on how well an attribute will split up the remaining data into disjunct groups
- The algorithm prefers an attribute with an high IG over attributes with lower IGs
- the IG for each attribute must be computed
Supervised Learning: k-Nearest Neighbor
Goal: Use labeled data to predict the class of a previously unknown instance based on its similarity to other data points. The idea is to identify the closest neighbor(s) and classify the new instance similarity. If the closest neighbors have different classes, perform a majority vote. This approach does not require a training phase and no model is built.
Training and Test Data (Rule, Trade-off)
In supervised learning algorithms, we have labeled data that we can use to train our models -> however, we need some data to test our trained model!
-> Rule: Never train on test data
Instead, we split up our data set into training data and test data and keep them separate:
- the larger we choose the training set, the better our
model becomes
- the larger we choose our testing set, the more
confidence we can have in our results
The Results of Classification: The Confusion Matrix
Confusion Matrix = represents relation between predicted outcome and the actual outcome = a table layout that allows visualization of the performance of an algorithm (supervised learning). Each row of the matrix represents the instances in a predicted class while each column represents the instances in an actual class -> shows us how many times our algorithm was correct/wrong
True vs. False and Positive vs. Negative (Definitions (4))
From the confusion matrix of the two or more classes, we can derive the the confusion matrix for each individual class.
Definitions:
- Positive: Instance is the predicted object
- Negative: Instance is not the predicted object
- True: Prediction is correct
- False: Prediction is wrong
Thus, the values in a Confusion Matrix can be TP, TN, FP or FN.
Evaluating Classification Models: Accuracy
Accuracy = metric for evaluating models. Loosely speaking, it’s the the fraction of predictions our model got right: (TP+TN) / (TP+TN+FP+FN)
Evaluating Classification Models: Precision and Recall
Precision = What proportion of positive identifications was actually correct? TP / (TP+FP)
Recall = What proportion of actual positives was identified correctly? TP / (TP+FN)
Accuracy vs. Precision vs. Recall (When do you need what metric?)
A high accuracy can be first evidence but should be treated with caution.
A high precision is important when false alarms are costly.
A high recall is important when it’s vital to detect every single positive instance.
