Session 4

Question 1

what is information gain?

Answer 1

it is the most common splitting criterion and is based on entropy
-> it measures how much a split reduces entropy (measures the change in entropy before and after splitting)

Question 2

what does disorder correspond to?

Answer 2

how mixed the segment is with respect to the values of attribute of interest

Question 3

what is entropy?

Answer 3

it is a measure of disorder in the data

Question 4

how can you calculate entropy?

Answer 4

Question 5

what is a parent set?

Answer 5

original set of examples (data points before splitting)

Question 6

what is a children set?

Answer 6

an attribute (e.g., age) can segment the parent set into k children sets (subsets)

Question 7

When is an attribute chosen for splitting?

Answer 7

The attribute that reduces entropy the most (= has the highest information gain) is chosen for the split

Question 8

what is the formula for information gain?

Answer 8

Question 9

what are disadvantages of ID3 decision trees?

Answer 9

• tends to prefer splits that result in large number of partition each beaing pure but small (we get a very wide decision tree)
• overfitting with less generealization capability (will try to fit in every outlier -> will make a segment for Musk in ranking of CEO salary, even if he is the only one so high up)
• cannot handle missing value

Question 10

what are the application possibilities of ANN (artificial neural networks)?

Answer 10

• spam detection
• time series prediction
• pattern recognition (how does van gogh paint)
• computer games

Question 11

how does ANN function?

Answer 11

it functions like human neurons -> learning by making interneuron connections

Question 12

what is a single perceptron algorithm ANN?

Answer 12

uses no hidden layer and mimics biology

Question 13

how does an ANN work?

Answer 13

inputs go into a propagation function that calculations the net input, then a transform cuntion calculates an activation level, then we reieeve an output

Question 14

what is the propagation function?

Answer 14

where inputs are independent variables, such as # of amenities

Question 15

what is the activation function?

Answer 15

function/ level that determines whether a neuron (whether the whole process starts) produces an output or not

Question 16

how does learning work in ANNs?

Answer 16

• comparing computed (predicted) outputs to desired (true target values) outputs of historical cases
• is defined as a change of weights between units

Question 17

what are the three tasks in the process of learning in ANNs?

Answer 17

1. compute temporary outputs
2. compare outputs with dired targets
3. adjust the weights and repeat process

Question 18

when is a data set linearly separable?

Answer 18

if there exists a straight line (in 2D) or a hyperplane (in highe dimensions) that can perfectly separate all data points of one class from those of another class without any errors

Question 19

when do we need multilayer perceptron?

Answer 19

Question 20

what are the three layers in multilayer perceptrons?

Answer 20

1. input layer: includes single attributes
2. hidden layers: the middle layer of ANN which has three or more layers - each layer increases the training effort exponentially
3. output layer: the layer containing the solution of the problem

Question 21

how does the development process of an ANN look like?

Answer 21

Question 22

what is the activiation function in MLPs?

Answer 22

• relation between the internal activation level and the output
• can be linear or non-linear
• differentiability means if we can build derivates of the cuntion
• there are different types

Question 23

what are the different types of activation functions?

Answer 23

Question 24

what are the four types of learning?

Answer 24

1. supervised learning
2. unsupervised learning
3. reinforcement learning (you don’t tell correct output, just say if correct or incorrect)
4. direct design methods

Question 25

what are the two times of learning?

Answer 25

1. incremental training (you adapt model step by step, by adding new data incrementally)
2. batch training (you train a model only using a subsample of data at a time)

Question 26

what are the 5 learning rules in ANN?

Answer 26

1. delta rule
2. gradient descent
3. back propagation
4. hebbian rule
5. competitive leaning

Question 27

what is back propagation?

Answer 27

• similar to delta rule, but also calculates weight changes for hidden layers

Question 28

what is gradient descent?

Answer 28

• finding combinations of all weights so that the sum of the squared errors F is minimized
• but required high computational complexity in high dimensional spaces