L17 & 18 - Applying Neural Networks

Question 1

When dealing with binary input, what does it mean if the output overestimates or underestimates? What action should be taken regarding weight update?

Answer 1

Overestimate -> Output is 1 when should be 0 -> Weight should be subtracted for next iteration

Underestimate -> Output is 0 instead of 1 -> Weight should be added for next iteration

Question 2

What is meant by Symbolic AI?

Answer 2

Term for the collection of all methods in AI research that are based on high-level symbolic (human readable) representation of problems, logic and search.

Question 3

What is meant by Connectionism?

Answer 3

Branch of AI that deals with that deals with the design and implementation of neural networks. Based on the humans neurological processes.

Question 4

What is meant by Explainable AI?

Answer 4

When models aren’t black box, and outcomes can be explained through analysing the logic of the model.

Question 5

What are the 3 core Neural Network properties?

Answer 5

1. Able to learn to relate input variables to required output
2. Able to generalise between samples
3. Shows grateful degradation

Question 6

What is meant by Graceful Degradation?

Answer 6

1. A property of neural networks
   2. The fact that removal of components from the module reduces performance as opposed to causing performance failure.

Question 7

What is a property that Neural Networks have that Symbolic AI doesn’t?

Answer 7

1. Graceful degradation

Question 8

Do Symbolic AI systems have good Generalisation capabilities?

Answer 8

1. No since they are programmed rather than learned
2. usually an expert model in a specific field
3. When taken out of the models field, they can’t generalise.

Question 9

Do Neural Networks (Connectionism) have good generalisation capabilities?

Answer 9

1. Neural Networks are noise tolerant
2. Learns underlying patterns for generalisation

Question 10

What are the 3 best data types to use as input for Neural Networks?

Answer 10

1. Continuous data -> Floating numbers
2. Integers
3. Discrete (Categorical)

Question 11

What are some data representation issues in Neural Networks?

Answer 11

1. Continuous data -> Can require normalisation
2. Integers -> Numbers should be ordered
3. Discrete -> Order bias can be implied if values don’t have good separate representation

Question 12

Why are missing values an issue with Neural Network input?

Answer 12

1. Occur frequently in the real world
2. Can’t be entered directly into the network

Question 13

What is a response to missing values in input data?

Answer 13

Compute an estimation for the missing value

Question 14

What are the two common types of output of Neural Networks?

Answer 14

1. Continuous
   
   2. Discrete

Question 15

How can we prevent overfitting of a Neural Network?

Answer 15

Stop the training of the neural network at an earlier point

Question 16

What is Early Stopping?

Answer 16

Technique used in NN’s to detect and prevent overfitting and improve generalisation

Question 17

How do we split our dataset in Early Stopping?

Answer 17

1. Training data
2. Validation data
3. Test data

Question 18

Looking at the relationship between validation data and training data, at what point do we know we are overfitting?

Answer 18

When validation is going up and training error is going down

Question 19

What is a problem with early stopping?

Answer 19

We are removing some of our training data

Question 20

In NLP, what is distributional semantics?

Answer 20

The notion that words certain sets of words appear in the same context E.g bus, car, taxi

Question 21

If a word has 4 meanings, how many dimensions are required in the training data?

Answer 21

4 dimensions, one for each meaning