ML-Final

Question 1

What is threshold logical unit

Answer 1

Simple model of a neuron

Each input value is multiplied with the corresponding weight value, and these weighted values are then summed.

If the weighted summed input is larger than a certain threshold value, then the output is set to one, and zero otherwise

Question 2

What is weight parameter

Answer 2

representing the ‘strength’ of a connection

Question 3

What is a perceptron

Answer 3

where the output is calculated from the weighted summed input with a activation function(gain function, transfer function, output function, activation function. )

Question 4

Give examples of gain function, transfer function, output function, activation function.

Answer 4

??? sigmoid, tanh

Question 5

Why we add bias term to perceptron

Answer 5

A bias allows a perceptron to shift the prediction to better fit.

Question 6

Similarities of SVM and perceptron ?

Answer 6

Linear SVM is a special case of a perceptron

Question 7

What is the difference between Deep Learning and SVM?

Answer 7

SVM solve the optimization problem with specific transformations of the feature space.

Deep learning will aim at learning the appropriate transformations.

Question 8

What is delta term(delta rule?)

Answer 8

δ = (y(i) − y)y(1 − y)

Delta rule is a gradient descent learning rule for updating the weights of the inputs

Question 9

Why a multilayer feedforward network is a universal function approximator

Answer 9

there is guaranteed to be a neural network so that for every possible input, (x) the value f(x) is output from the network

given enough hidden nodes, any functions can be approximated with arbitrary precision by these networks

Question 10

What is error-back-propagation or backpropagation

Answer 10

calculation of the gradient proceeds backwards through the network, gradient of the final layer of weights being calculated first and the gradient of the first layer of weights being calculated last

Question 11

Batch, mini-batch and online

Answer 11

online avoiding local minima, mini-batch large datasets. Batch high memory space.

Question 12

No free lunch” theorem

Answer 12

no one model that works best for every problem.

The assumptions of a great model for one problem may not hold for another problem

Question 13

What is cross entropy? (The negative log probability) what is it used for ?

Answer 13

The negative log probability of the given label times the current model(probability distribution)

H(p,q) = − sum[ p(y) log q(y) ]

q: true nature of data
p: The neural network model represents the probability p(y|x; w)

Derive learning rule

Question 14

KL-divergence, what is it equivalent to? What are they related to? What are they used in neural network ?

Answer 14

related to cross entropy
H(p, q) = H(p) + KL(p||q)

minimizing the cross entropy is equivalent to minimizing the KL-divergence

both are closely related to the maximum (log) likelihood principle

use to generate learning rule?

Question 15

What is softmax function, why and where does it used in neural network?

Answer 15

Softmax function is a generalization of the logistic function that “squeeze” the output in the range (0, 1)

It is used to highlight the largest values and suppress values which are significantly below the maximum value in a neural network.

final layer of a neural network

Question 16

How neural networks are related to probabilistic regression?

Answer 16

cross entropy, KL-divergence

Question 17

What is the relationship of maximize the log probability and cross entropy

Answer 17

That is, we want to maximize the log probability of the data given the labels. Since the cross entropy is the negative of this, maximizing the log probability of the data given the labels is equivalent of minimizing the cross entropy

Question 18

What is deep learning?

Answer 18

Deep learning basically refer to neural networks with many layers

Question 19

What is a filter in CNN?

Answer 19

It is a vector describing a pattern

Question 20

What is convolution?

Answer 20

Convolution is the operation of multiplying and adding while shifting the filter

Question 21

What is a stride in CNN?

Answer 21

It is how many steps you shift the filter in each iteration of the convolution

Question 22

What is a pooling operation in convolutional neural networks and why is this operation important?

Answer 22

Pooling is taking the average or the maximum of the previous output in a certain area of the filtered image.

It compress down the image and high-level representation.

This is usually called downsampling, this operation is important is because it reduce the dimensionality of features and computational cost .

Also helps to prevent overfitting.

Question 23

Briefly explain `dropout’ and why it is used in deep networks. 


Answer 23

Dropout: Randomly (e.g. p=0.5) ignoring hidden node for a specific input during learning. temporarily turned off

The reason we use it is that it is a regularization technique that helps to prevent overfitting.

Question 24

Sparse representation, comprised representation and fully distributed representation.

Answer 24

???

Question 25

What is autoencoder? Why it is useful?

Answer 25

An autoencoder is a neural network that tries to reconstruct its input.

it is a feature extraction algorithm it helps us find a representation for our data

Question 26

What is the relation between Ridge regression and a Gaussian prior?

Answer 26

Ridge regression use the L2 regularization, and the L2 regularization is equivalent to a Gaussian prior.

Question 27

What is batch normalization ?

Answer 27

Batch normalization: normalize the input to each hidden layer over each mini-batch

Question 28

What is skip connections in neural network?

Answer 28

the process to skip the convolutional layers in the network

Question 29

What is Recurrent Neural Networks and Where is the term ‘recurrent’ comes from?? What is used for ?

Answer 29

Recurrent Neural Networks perform the same task for every element of a sequence.

It used for sequence processing eg, for machine translation

Question 30

Explain what is backpropagation-though-time in RNN?

Answer 30

????

Question 31

What is Gated Neural Network?

Answer 31

A gated recurrent network has an extra memory state(namely gated) that will be carried from the current step to the next step.

A forgetting gate and a write gate can modify its value.

An example of such neural network would be LSTM (Long Short Term Memory ) or a gated recurrent unit (GRU) s

Question 32

What is Boltzmann machine ? What is the challenge of it?

Answer 32

Special form of recurrent network that the connections between nodes are symmetric

The challenge is finding practical training rules

Question 33

What is reinforcement learning (RL)?

Answer 33

A learning system with action and reward.

Question 34

In reinforcement learning, what is a policy? 


Answer 34

A policy in reinforcement learning is use to determine the action to take in each state.

Question 35

What are the RL challenges ?

Answer 35

1. Credit assignment

2. Exploration versus exploitation trade-off.

Question 36

What is Markov condition ? or the Markov Decision Process? (same as transition function in RL)

Answer 36

transition function only depend on the previous state and the intended action from the previous state

Question 37

What is Reward function in RL?

Answer 37

rt+1 = ρ(st, at)

returns the value of reward when the agent is entering state st+1 by taking action at from state st

Question 38

What is Policy in RL?

Answer 38

A policy in reinforcement learning is use to determine the action to take in each state. Policy: at = π(st)

Question 39

Value function and Optimal Value function

Answer 39

Reward and disconnect reward
this functions tells us how good is action a in state s

Value function (state-action): Qπ(s, a) 
Value function (state): V π (s) = Qπ (s, π(s))
Optimal Value function: V ∗(s) = max Q∗(s, a),

Question 40

Optimal policy

Answer 40

Optimal policy: π∗(s) = arg max Q∗(s, a).

Question 41

What is Model-based Reinforcement Learning ?

Answer 41

we assume that the agent has a model of the environment and its behaviour by knowing the reward function ρ(s, a) and the transfer functions τ (s, a).

Question 42

What is Model-free Reinforcement Learning ?

Answer 42

???

Question 43

SARSA

Answer 43

??

Question 44

Q-learning

Answer 44

??

Question 45

Explain the difference between the SARSA and Q-Learning algorithm. 


Answer 45

SARSA is an on-policy approach of RL.

in the part where γ Q (st+1, at+1) we know that its use the previous policy to generate the next policy.

Namely State-Action-Reward-State-Action.

Q-learning is an off-policy approach in RL.

γ m a xaQ (si+1, a′) is the part that is different than ASRSA.
Here, we do not limit the how the next action is selected which means the policy generated in Q-learning is not depends on the previous policy.

Question 46

epsilon-greedy policy ?

Answer 46

??

Question 47

What is the difference between on-policy and off policy?

Answer 47

????

Question 48

basic Bellman equation?

Answer 48

???

Question 49

What can we learn about SARSA and Q-Learning ?

Answer 49

SARSA will avoid the mistake due to exploration, and Q- learning still have the ability to learn with different exploring policy.

Question 50

What is reward function in RL? What is transfer function ?

Answer 50

reward function ρ(s, a) and the transfer functions τ (s, a).

Question 51

What is non-Markovian condition ?

Answer 51

non-Markovian condition would be the case in which the next state depends on a series of previous states and actions

Question 52

Temporal difference ?

Answer 52

ggggg