Lecture 1

Question 1

Define artificial intelligence (AI)

Answer 1

Methods where a computer mimics human (or other animal) behaviour

The theory and development of computer systems able to perform tasks normally requiring human intelligence, such as visual perception, speech recognition, decision-making, and translation between languages.

Question 2

Define machine learning.

Answer 2

A subfield of artificial intelligence, using statistical models that let machines get better at tasks with experience.

Question 3

Define deep learning.

What are its features?

Answer 3

Machine learning using multilayer [deep] neural networks.

• Highly flexible and non-linear
• Capable of representing any functional mapping

A type of machine learning based on artificial neural networks in which multiple layers of processing are used to extract progressively higher level features from data.

Question 4

What are the different types of machine learning?

Answer 4

Supervised learning and unsupervised learning.

Question 5

Describe supervised learning.

Answer 5

Training a model by showing it inputs and outputs. When you show it a new input, it will predict an output.

Question 6

Describe unsupervised learning.

Answer 6

Discovering patterns in the data that were not known before.

Question 7

What are the two types of supervised learning?

Answer 7

Classification and regression.

Question 8

Describe classification.

Answer 8

Given an input, we assign it to a particular class.

Question 9

Describe regression.

Answer 9

Given an input, we assign to it a number (or set of numbers).

Question 10

Briefly, how do we conduct machine learning?

Answer 10

• Start off with a training set - for each piece of training data we know the right answer (ie what we want to predict)
• Build a model for the process that generated the data
• Use this model to make predictions about data we have not seen before

Question 11

What are some examples of machine learning?

Answer 11

• Face recognition
• Text recognition
• Voice assistance
• Autonomous driving
• Drug discovery
• Quantum chemistry

Question 12

Regardless of the machine learning algorithm we are interested in or the objects we want to make predictions about, what kind of data do we need?

Answer 12

The object must be converted to numerical data.

Question 13

What is the data represented by?

Answer 13

A set of numbers, x

Question 14

What is each xi referred to as?

Answer 14

A feature. Therefore x is a feature vector.

Question 15

In image recognition, what could the vectors represent?

Answer 15

We get a vector xi for each image i, containing the darkness of each pixel.

Question 16

Why do we need to select features?

Answer 16

Not every piece of data we have relates to the target variable.

Some may not change at all with the target variable or some may be poor predictors.

Question 17

What is the step function?

Answer 17

H(x)

A function that increases or decreases abruptly from one constant value to another. Used in the case of classification to distinguish between two classes.

Question 18

How can we rewrite our mathematical learning problems?

Answer 18

Rewrite in terms of neurons.

Question 19

What are neurons?

Answer 19

In machine learning, a “neuron” refers to the basic processing unit within an artificial neural network, essentially a mathematical model that receives input signals, performs calculations based on assigned weights, and produces an output signal.

Question 20

What do neurons do?

Answer 20

Take inputs, transform them and give output (m). We can apply a function H(m) to this, to decide whether we are in class 1 or 2.

Question 21

What is a perceptron?

Answer 21

In machine learning, a “perceptron” is a simple model of a biological neuron, considered the most basic form of an artificial neural network. It is used for supervised learning of binary classification tasks, where it takes multiple weighted inputs and produces a single binary output (either 0 or 1) based on a linear decision boundary.

• Perceptron takes inputs
• These are combined to give the activation m
• An activation function f(.) is applied to m
• The output is z = f(m)

Question 22

How do we find the weights and bias of the activation m?

Answer 22

• We have a training set with N data points
• Each data point has an input xi and target output (ti)
• We fix the weights of the perceptron by tuning them so that we get the right answer for our inputs

ie for the network, we are finding the best vector of weights

Question 23

For a network, we want to find the best vector of weights. What does this mean?

Answer 23

Mostly, this means the set of weights that results in the fewest wrong answers.

We start with some set of weights w(0) and improve them to get w(1), w(2)…. Each step changes the weights to decrease the error.

Question 24

How do we get from w(i) to w(i+1)?

Answer 24

Decrease the errors.

Question 25

When do we get errors?

Answer 25

When we misclassify.

Question 26

What is the formula for the probability of a wrong answer?

Answer 26

[See flashcard]

Question 27

How many possibilities are there for each input?

Answer 27

Four

Question 28

What does the error, E represent?

Answer 28

The probability of getting a wrong answer

Question 29

If answers are always correct, what is the error?

Answer 29

E = 0

Question 30

Why is it difficult to derive an algorithm for updating the weights?

Answer 30

The error is not a continuous function of the weights.

Question 31

What algorithm do we use for updating the weights?

Answer 31

Perceptron learning algorithm.

Question 32

What steps are followed to update the weights?

Answer 32

[See flashcard]

Question 33

In the formula for perceptron learning, what is v?

Answer 33

v is the learning rate, it controls how quickly the weights change

Question 34

When are the weights updated?

Answer 34

Each time we select a training point

Question 35

What is batch learning?

Answer 35

When all training points are used in order.

Question 36

What is on-line learning?

Answer 36

At every update, choose a new training point at random.

Question 37

What is a popular example for the perceptron?

Answer 37

Logic gates. These perform some operation on a pair of binary inputs.

Question 38

What does a logic gate do?

Answer 38

Performs some operation on a pair of binary inputs.

Question 39

What are possibilities for the definition of the error function?

Answer 39

Mean squared error [see flashcard for formula]
Sum-of-squares error [see flashcard for formula]

Question 40

Which error function allows models with different number of training points to be compared?

Answer 40

Mean squared error.

Question 41

Which error function is often used in the literature?

Answer 41

Sum-of-squares error

Question 42

For classification problems, what error do we use?

Answer 42

The cross-entropy error
[See flashcard for formula]

Question 43

What is a linearly separable problem?

Answer 43

When we can draw a straight line in 2D separating two classes.

Question 44

If a problem is not linearly separable, what can you try to do?

Answer 44

Transform the problem so it is linear, by choosing better features.

Or problems may become solvable when we choose MORE features.

Question 45

How does the perceptron make decisions?

Answer 45

Based on the activation, zi = f(m)

Question 46

How do we get a different type of decision boundary (not linear)?

Answer 46

The activation must be a nonlinear function of the input features.

This leads to the multilayer perceptron (neural network)

Question 47

What is an alternative to the sigmoid function that is nowadays quite popular?

Answer 47

Rectified linear unit (ReLU)

Question 48

What does overfitting indicate?

Answer 48

The model is too complex.

It may be fitted to the training data perfectly but is unable to generalise.

Question 49

Pick up around p20 - but incorporate anything given from lectures