ML-Midterm

Question 1

What is Machine Learning?

Answer 1

machine learning is about modeling based on a specific hypothesis function, ideally in the form of a density function that describes the data.

Question 2

What is supervised learning and unsupervised learning ?

Answer 2

One has label the other one does not

Question 3

What is a model ?

Answer 3

model is an approximation of a system

and

predict the behaviour of it.

Question 4

gradient descent

Answer 4

gradient descent can find parameters to minimize the loss of the training data

Question 5

Explain the k-fold cross validation procedure and explain what it is used for ?

Answer 5

Procedure: divide data in k even sets choose 1 set as validation set and k-1 as the training set, then alter the validation set and repeat the operation k times.

It is use for validate the accuracy of trained model.

Question 6

What is Training set, testing set and validation set ?

Answer 6

Training Set: this data set is used to adjust the weights on the neural network.

Validation Set: this data set is used to minimize overfitting. Then you can use this information to turn your hyper parameter.

Testing Set: this data set is used only for testing the final solution in order to confirm the actual predictive power of the network.

Question 7

What is hyper parameter?

Answer 7

parameter of the training algorithm, such as learning rates, momentum, or maximum number of iterations

Question 8

What is information leakage or information contamination ?

Answer 8

you use the test data to train the model and use them to test your model.

Question 9

What is support vector machine ?

Answer 9

Maximum margin classifier

Question 10

What is hyperplane in SVM ?

Answer 10

hyperplane is the dividing or separating plane between the two classes

Question 11

Why SVM called support vectors ?

Answer 11

????

Question 12

What is Soft margin classifier ?

Answer 12

Allow some overlap of the data points.

Question 13

Why use kernel ? kernel trick?

Answer 13

Transform the data to higher dimension so it is linearly separable.

To avoid calculate the dot product in the feature space.

Question 14

What is the purpose of regularization? give example and why it helps overfitting.

Answer 14

Prevent overfitting
Ridge regression we add L2 regularization
to control the complexity of model. It penalizes the features with less influence on the model.

Question 15

What is Batch gradient decent ?

Answer 15

batch gradient decent use all the data to train.

Question 16

What is Stochastic gradient decent ?

Answer 16

Stochastic gradient decent use only single data at a time.

Question 17

How does momentum helps in linear regression.

Answer 17

Momentum helps to overcome the local minimum.

Question 18

What is Ridge regression ?

Answer 18

We add L2 regularization as a penalty term when updating the weights. Weight decay.

Question 19

What is a random variable ?

Answer 19

Different every time.

Follows a specific probability density function

Question 20

Different distribution, uniform, bimodel, multimodel distribution?

Answer 20

one peak , two peak and multiple peak.

Question 21

What is Bayes theorem ?

Answer 21

p(x|y) = p(y|x)p(x)/p(y)

prior knowledge p(x)

evidence called the likelihood p(y|x) (observed)

posterior distribution p(x|y)

Question 22

Explain what is maximum likelihood principle ?

Answer 22

Given a parameterize hypothesis function p(y|x; w), we will chose as parameters the values which make the data y most likely under this assumption

Question 23

Likelihood function ? Maximum (log) likelihood

Answer 23

maximizing the log-likelihood function is equivalent to minimizing a quadratic error term

Question 24

Why LMS regression is equivalent to MLE for Gaussian data ?

Answer 24

Because the linear dependence of the mean and constant variance.

Question 25

What estimation MLE and MAP give us ?

Answer 25

Point estimate.

Question 26

Difference between MLE and MAP ? What is it used for? Related?

Answer 26

MAP maximize the posteriori MLE maximize the likelihood. Eg, babes rule.

They used to obtain a point estimate of an unobserved quantity based on training data.

Related if the prior is constant(uniform prior)MAP=MLE.

Question 27

What estimation MLE and MAP give us ?

Answer 27

Point estimate of a distribution.

Question 28

What is generative model ?

Answer 28

Generative model is to ‘generate’ examples of the class objects

Question 29

What is generative model ? What it is used to?

Answer 29

Generative model is to ‘generate’ examples of the class objects. Use to solve classification task.

Question 30

What is discriminative model ?

Answer 30

A model that discriminates between classes based on the feature values.

Question 31

What is K-means clustering, what is k-medios.

Answer 31

k-mean is to find the means of the data and k-medios is to find the central of the data.

Question 32

What is Gaussian Mixture Model ?

Answer 32

we have k Gaussian classes, where each class is chosen randomly from a multinominal distribution

Question 33

What is expectation-maximization (EM) algorithm ?

Answer 33

E-step find labels of the data based on a assumption of distribution.

We make assumptions of training labels from the current model (expectation step)

M-step to update the parameters of the model to maximize the probability of the observations (maximization step).

Question 34

What is Causal model ?

Answer 34

???

Question 35

What is naive babes good to ? And Why ?

Answer 35

Text classification, It is efficient that you do not have to calculate the probability of the words not in the dictionary.

Question 36

Graphical representation of Naive Bayes ?

Answer 36

class
/ | \
x1 x2 x3

Question 37

Assumption in Naive Bayes ?

Answer 37

the probabilities are conditionally independent.