Chapter 4: Machine Learning and Optimisation

Question 1

what datatypes do we have in machine learning

Answer 1

objects and pairwise relations

Question 2

what data containers do we have and what do they do

Answer 2

hold data types

scalar, matrix, vector, tensor

Question 3

what data structures do we use. what do they do

Answer 3

set, tree, graph

they allow us to create a set of containers

Question 4

what is a parameter

Answer 4

they control model behaviour. They are set during training.

a non parametric model has none

Question 5

give an example of a non parametric model

Answer 5

KNN

Question 6

what is an objective function

Answer 6

it is either maximised or minimised by a technique:
by example- supervised
by reward- reinforcement
by exploration- unsupervised

also called loss/cost

Question 7

describe the three approaches for decision making

Answer 7

1. direct- map from input to output
   2/3, inference- give the probability
   2- calculate probability directly
   3- calculate probability indirectly i.e. using a base theory

Question 8

give the three approaches for decision making

Answer 8

1 discriminant function
2 direct model
3 bayes theorem

Question 9

within approach 1, give the different approaches

Answer 9

linear model
linear basis function
kernel method

Question 10

within approach 2, give the different approaches

Answer 10

for classification- logistic regression

for regression- Bayesian regression

Question 11

what is the approach for approach 3

Answer 11

naïve bayes

Question 12

describe a linear model

Answer 12

the output is the weighted sum of the inputs

Question 13

how is the linear model used for classification

Answer 13

threshold function. this creates a hyperplane which is a classification/division/separating boundary

Question 14

how can we create a ROC curve in a binary model

Answer 14

change the threshold value

Question 15

how do we handle non linear data patterns

Answer 15

map to linear within a new feature space. We then have an input space and a feature space

Question 16

what are the two basis functions described in this course

Answer 16

gaussian basis function and polynomial basis function

Question 17

what does gaussian basis function use as parameters

Answer 17

mean and variance

Question 18

what does a polynomial basis function use as parameters

Answer 18

integers

Question 19

describe the kernel method

Answer 19

like linear basis function, in that it creates a non linear separating boundary, but directly defines the mapping function rather than relying on user set values

uses the inner products

Question 20

give example kernel function

Answer 20

linear, polynomial, gaussian, hyperbolic tangent

Question 21

describe logistic regression

Answer 21

gives the probability of a sample belonging to a class.

apply SoftMax to a linear model to obtain class posterior.

Question 22

describe gaussian Bayesian linear regression

Answer 22

for regression

assumes the output follows a gaussian distribution

Tread w and o^2 as random

Question 23

how do we infer p(c | x)

Answer 23

bayes theorem

Question 24

what is approach 3

Answer 24

Instead of calculating probability directly, we apply a base theory for probability.

Question 25

give p(c | x)
( bayes theorem )

Answer 25

p(x)