Which machine learning algorithm should I use?

Question 1

Dimension reduction

Answer 1

Reducing the number of variables under consideration. In many applications, the raw data have very high dimensional features and some features are redundant or irrelevant to the task. Reducing the dimensionality helps to find the true, latent relationship.

Question 2

Supervised learning

Answer 2

Supervised learning algorithms make predictions based on a set of examples.

• Classification
• Regression
• Forecasting

Question 3

PCA

Answer 3

An unsupervised clustering method which maps the original data space into a lower dimensional space while preserving as much information as possible. The PCA basically finds a subspace that most preserves the data variance, with the subspace defined by the dominant eigenvectors of the data’s covariance matrix.

Question 4

CheatSheet

Answer 4

Question 5

Linear SVM and kernel SVM

Answer 5

When the classes are not linearly separable, a kernel trick can be used to map a non-linearly separable space into a higher dimension linearly separable space.

When most dependent variables are numeric, logistic regression and SVM should be the first try for classification.

Question 6

Unsupervised: Clustering

Answer 6

Question 7

Factors to consider in ML algorithm

Answer 7

• The size, quality, and nature of data.
• The available computational time.
• The urgency of the task.
• What you want to do with the data.

Question 8

Supervised: Classification

Answer 8

Question 9

SVD

Answer 9

• SVD is also widely used as a topic modeling tool, known as latent semantic analysis, in natural language processing (NLP).
• SVD of a user-versus-movie matrix is able to extract the user profiles and movie profiles which can be used in a recommendation system

Question 10

Classification

Answer 10

When the data are being used to predict a categorical variable

Question 11

DBSCAN

Answer 11

When the number of clusters k is not given, DBSCAN (density-based spatial clustering) can be used by connecting samples through density diffusion.

Question 12

Regression

Answer 12

When predicting continuous values

Question 13

Hierarchical result

Answer 13

use hierarchical clustering

Question 14

Semi-supervised learning

Answer 14

Use unlabeled examples with a small amount of labeled data to improve the learning accuracy.

Question 15

When trying to solve a new ML problem what are the three steps?

Answer 15

​​

1. Define the problem. What problems do you want to solve?
2. Start simple. Be familiar with the data and the baseline results.
3. Then try something more complicated.

Question 16

Why we need PCA, SVD and LDA

Answer 16

We generally do not want to feed a large number of features directly into a machine learning algorithm since some features may be irrelevant or the “intrinsic” dimensionality may be smaller than the number of features

Question 17

Supervised: Regression

Answer 17

Question 18

Neural networks and deep learning

Answer 18

• A neural network consists of three parts: an input layer, hidden layers, and an output layer.
• The number of hidden layers defines the model complexity and modeling capacity.
• Output layer is a categorical variable, then the neural network is a way to address classification problems.
• Output layer is a continuous variable, then the network can be used to do regression.
• Output layer is the same as the input layer, the network can be used to extract intrinsic features.

Question 19

What are [1], [2], [3], and [4]?

Answer 19

[1] Unsupervised: Dimensionality Reduction)

[2] Unsupervised: Clustering

[3] Supervised: Regression

[4] Supervised: Classification

Question 20

Considerations when choosing an algorithm

Answer 20

• Accuracy (Phase III)
• Training time (Phase II)
• Ease of use (Phase I)

Question 21

What are PCA, SVD and LDA

Answer 21

Principal component analysis (PCA)

Singular value decomposition (SVD)

Latent Dirichlet allocation (LDA)

Question 22

Hierarchical clustering

Answer 22

Hierarchical partitions can be visualized using a tree structure (a dendrogram). It does not need the number of clusters as an input and the partitions can be viewed at different levels of granularities (i.e., can refine/coarsen clusters) using different K.

Question 23

Perform dimension reduction​

Answer 23

Principal component analysis

Question 24

k-means, k-modes, and GMM (Gaussian mixture model) clustering

Answer 24

• Clustering aims to partition n observations into k clusters.
• K-means define hard assignment: the samples are to be and only to be associated with one cluster.
• GMM : define soft assignment: Each sample has a probability to be associated with each cluster.
• Both algorithms are simple and fast enough for clustering when the number of clusters k is given.

Question 25

Linear regression

and

Logistic regression

Answer 25

Question 26

LDA, GMM, and NLP

Answer 26

• LDA is a probabilistic topic model and it decomposes documents into topics in a similar way as a Gaussian mixture model (GMM) decomposes continuous data into Gaussian densities.
• • Differently from the GMM, an LDA models discrete data (words in documents) and it constrains that the topics are a prior distributed according to a Dirichlet distribution

Question 27

Forecasting

Answer 27

Making predictions about the future based on the past and present data.

Question 28

Reinforcement learning

Answer 28

Reinforcement learning analyzes and optimizes the behavior of an agent based on the feedback from the environment. Machines try different scenarios to discover which actions yield the greatest reward, rather than being told which actions to take. Trial-and-error and delayed reward distinguishes reinforcement learning from other techniques.

Question 29

Unsupervised learning

Answer 29

The machine is presented with totally unlabeled data. It is asked to discover the intrinsic patterns that underlies the data, such as:

• a clustering structure,
• a low-dimensional manifold,
• a sparse tree and graph.

Question 30

Clustering

Answer 30

Grouping a set of data examples so that examples in one group (or one cluster) are more similar (according to some criteria) than those in other groups.

Question 31

Unsupervised: Dimensionality reduction

Answer 31

Question 32

Trees and ensemble trees

Answer 32

• Subdivide the feature space into regions with mostly the same label
• Random Forrest and gradient boosting are two popular ways to use tree algorithms to achieve good accuracy as well as overcoming the over-fitting problem

Question 33

Numeric prediction quickly​

Answer 33

​​

• Decision trees
• Logistic regression