Supervised Learning

Question 1

Bias error

Answer 1

Model does/can not correctly represent the concept (underfit)

Question 2

Variance error

Answer 2

Model specializes in training set (overfit)
Regularization (favoring smoother
functions, output varies slowly with input) helps to mitigate the variance error

Question 3

Multilinear Regression assumes

Answer 3

• Relation between xi and y is linear
• All variables (x) have Normal distributions
• Variables are independent and residual / error is constant

Question 4

The input of an artificial
neuron:

Answer 4

Comes from all neurons of the previous layer or it is an external
input

Question 5

The output of an artificial neuron

Answer 5

Is sent to all neurons of
the next layer or is (part of) the network output

Question 6

Backpropagation

Answer 6

1. Present each example (x(i),d(i))
2. Calculate network response x(i) : f(x(i))
3. Propagate error backwards (iteratively building error
   derivative at each layer)
4. Save partial derivatives
5. After all examples processed, update weights

Question 7

Artificial Neural Networks are

Answer 7

• Robust to noise and approximations
• Based in a simplified model of a neuron
• Support incremental training
• Compress information of many examples in a small model

Question 8

Deep Learning

Answer 8

Alternating prediction layers with feature detection and decorrelation

Question 9

Deep Learning - network structure

Answer 9

• Convolutional layers: apply convolutions to get the feature maps
• Pooling (sub-sampling) layers: reduce feature maps’ dimensions (combine features and/or decorrelate)
• Dense layers – similar to the “hidden” layers on a classical neuronal network

Question 10

kNN problems

Answer 10

• Define distance
• Define class selection
• Non-linear problems

Question 11

A set has the largest entropy if

Answer 11

each of its elements belongs to a
different class

Question 12

PlayTennis(no/yes) - entropy(S)

Answer 12

− ( P(no) x log2 (P(no)) + P(yes) x log2 (P(yes)) )

Question 13

Decision Tree - the best split

Answer 13

The best split is the split that results in the largest entropy reduction, that is, the largest information
gain (IG)

Question 14

Decision Tree - C4.5 / C5.0

Answer 14

Similar to ID3, but. . .
* Support for continuous attributes - discretizes continuous attributes
* Allows missing values - examples not used when calculating entropy
* Allows different costs for attributes
* Prunning

Question 15

Learning ensembles

Answer 15

Boosting (Kearns 88)
* Can a set of weak learners create a single strong
learner?
* Classification combines the results of all the subtrees
* Misclassified examples become more important for
the error in each iteration
* New trees are trained to fit the residual error

Bagging - Bootstrap aggregating: (Breiman 96)
* Selects randomly the subsets
* Trains several learners,
* Classification by voting, regression by averaging

Question 16

XGBoost (eXtreme Gradient
Boosting)

Answer 16

• An optimized Gradient Boosting Machine
• Uses many small trees
• Classifies an example by joining the scores of each of the various trees
• Train by adding trees that improve the result or pruning
• Trees are fitted to predict the residual error