ML Fundamentals

Question 1

What is machine learning?

Answer 1

Machine Learning is the science (and art) of programming computers so they can learn from data.

Question 2

Supervised/Unsupervised learning - what are they?

Answer 2

Supervised - the training set you feed to the algorithm includes the desired solutions, typically seen in classification or regression

Unsupervised - Opposite of supervised, the training data is unlabeled.

Question 3

Supervised learning algorithms

Answer 3

k-nearest neighbors, linear regression, logistic regression, support vector machines (SVMs), Decision trees and random forests, neural networks

Question 4

Unsupervised learning algorithms

Answer 4

Clustering (k-means, DBSCAN, Hierarchical cluster analysis)

Anomaly and novelty detection (one-class SVM, isolation forest)

Visualization and Dimensionality reduction (principal component analysis [PCA], kernal PCA, locally linear embedding (LLE), t-distributed stochastic neighbor embedding (t-SNE))

Association ruling (Apriori, eclat)

Question 5

Dimensionality reduction

Answer 5

to simplify the data without losing too much information. One way to do this would be to merge several correlated features into one. For example merging a cars age and mileage into one ‘wear and tear’ feature. Good idea to reduce the dimension you are training data on.

Question 6

Feature extraction

Answer 6

Form of dimensionality reduction where you merge two correlated features into one. For example merging a cars age and mileage into one ‘wear and tear’ feature

Question 7

Semi-supervised learning

Answer 7

Some of the data is labeled. Ex: Photos app names the people

Question 8

Reinforcement learning

Answer 8

The learning system, called an agent in this context, can observe the environment, select and perform actions, and get rewards in return (or penalties in the form of negative rewards, as shown in Figure 1-12). It must then learn by itself what is the best strategy, called a policy, to get the most reward over time. A policy defines what action the agent should choose when it is in a given situation.

Question 9

Batch learning

Answer 9

the system is incapable of learning incrementally, it must be trained using all the available data. Done offline.

Question 10

Online learning

Answer 10

You train the system incrementally by feeding it data instances sequentially, either individually or in small groups

Question 11

learning rate

Answer 11

how fast learning system adapts to changing data. If you set a high learning rate, then your system will rapidly adapt to new data

Question 12

Generalizing

Answer 12

Given a number of training examples, the system needs to be able to make good predictions for (generalize to) examples it has never seen before

Question 13

Instance-based learning

Answer 13

system learns examples by heart, then generalizes to new cases by using a similarity measure to compare them to the learned examples

Question 14

model-based learning

Answer 14

build a model of these examples and then use the model to make predictions

Question 15

utility function (fitness function)

Answer 15

measures how good your model is

Question 16

cost function

Answer 16

measures how bad your model is

Question 17

Steps of ML

Answer 17

1. Study the data 2. Select a model 3. train the data 4. apply the model to make predictions on new cases

Question 18

Challenges for ML

Answer 18

1. Insufficient quantity of data. 2. Non-representative training data 3. Poor data quality 4. Irrelevant features 5. Overfitting the training data

Question 19

regularization

Answer 19

constraining a model to make it simpler and reduce the risk of overfitting

Question 20

hyperparameter

Answer 20

used to apply regularization to avoid overfitting.

Question 21

out of core learning

Answer 21

Online learning algorithms can also be used to train systems on huge datasets that cannot fit in one machine’s main memory . The algorithm loads part of the data, runs a training step on that data, and repeats the process until it has run on all of the data

Question 22

fitting or training

Answer 22

capturing patterns from data

Question 23

training data

Answer 23

data used to fit or train the model

Question 24

leaf

Answer 24

point of a decision tree where you make a prediction

Question 25

standard deviation

Answer 25

measures how spread out values are from the mean

Question 26

feature

Answer 26

inputs into our model later used to make predictions of the prediction target

Question 27

mean absolute error

Answer 27

error = |actual - predicted|

mean absolute error is the mean of the abs value of each of the errors

Question 28

overfitting

Answer 28

model matches the training data perfectly, but matches poorly in validation on new data

Question 29

underfitting

Answer 29

model fails to capture important distinctions and patterns in the data

Question 30

random forest

Answer 30

made of many decision trees, making a prediction by averaging the predictions of each component tree