Lecture 1 - Luke Flashcards
What is the brain initiative?
The BRAIN Initiative seeks to deepen understanding of the inner workings of the human mind and to improve how we treat, prevent, and cure disorders of the brain
Three types of output functions for neural networks
Linear
Step
Sigmoid
What is the output of the bias unit always equal to?
-1
What is Hebb’s rule?
deltaW_{ij} = x_iy_j
with weight update:
w_{ij}(t)=w_{ij}(t-1)+etadeltaW_{ij}
What does Hebb’s rule suffer from?
Self-amplification
What is the use of the synaptic weights in an NN?
Reduce error between output y and its desired output
What is the delta rule?
delta = t-y
where t is the teaching input and y is the output
Could you create a perceptron that can separate XOR inputs?
No, as they are not linearly separable
Brief description of an ANN
An ANN communicates with the environments through input and output units.
Units are linked by uni-directional connections, characterised by a weight and sign that transforms the signal.
What does a multi-layer nn do to solve problems that are not linearly separable?
Re-map input space into a space which can be linearly separated by output units.
What kind of neural network should you use on time series data?
Recurrent neural network
Why should a multilayer nn not use a linear output function?
A linear transformation of a linear data set remains linear
What is backpropagation of error?
To propagate the error of the units backwards to the hidden units through the connection weights; once we have the error for the hidden units, we can change the lower layer of connection weights in the same manner as the upper layer
How can too many weights in the nn affect performance?
Overfitting
How to overcome overfitting?
Use a validation set; divide the available data into a training set (for weight update) and validation set (for error monitoring).
