C3

Question 1

classification types

Answer 1

• binary
• multi-class: each instance has 1 label/class
• multi-label: each instance has 0 or more labels

Question 2

vector space models

Answer 2

(eg. support vector machine)
- each instance is modelled as a vector in a space
- each item in the vector is the weight for one feature

contiguity hypothesis: data points in the same class form a contiguous region (closeness of two points represents similarity)

Question 3

support vector machine

Answer 3

• representation of the examples in a vector space
• divide the examples of the separate categories by a margin as large as possible
• separating boundary = hyperplane (in an n-dimensional space, the hyperplane has n-1 dimensions)

what if the data is not linearly separable? –> use Kernel Trick: add extra dimension z = x^2 + y^2
learn line k that separates the classes and transform it back to the original space: x^2 + y^2 = k

Question 4

neural networks

Answer 4

more powerful classifier than logistic regression:

• multiple nodes –> multiple functions –> non-linearity
• multiple layers –> multiple abstractions over the input data

weights are learned automatically (supervised learning using true labels)

Question 5

feedforward networks

Answer 5

multilayer network in which the units are connected with no cycles (no outputs are passed back to lower layers)
each layer is fully connected, each hidden unit sums over all the input units

output layer gives probability distribution:

• binary classification: single output node, y is the probability of positive output
• multi-class classification: one output node for each category, y is the probability of that category

Question 6

training neural networks

Answer 6

learn parameters W and b for each layer that make y_estimate for each training observation as close as possible to the true y

1. loss function to model distance between y_estimate and y
2. find parameters to minimize this loss function