lecture 12: Neural Networks Flashcards
what is a neural network(NN)
it is in simple terms a nested function, the output of one layer becomes the input for the next layer. it is made up of nodes, also referred to as neurons
what is multilayer perception(MLP)
it is a type of neural network called a feedforward neural network (FNN), it is one of the most basic structures of a NN
different activation functions can be used for neural networks, what are the form of these function and what are the 4 kinds mentioned?
all activation functions are non-linear
sigmoid, gaussian, ReLU(rectified linear unit), softplus
what does σ(a) represent
the activation function, can be any of the 4
why is MLP also referred to as a fully connected network
because the layers are fully connected to each other, every neuron in one layer is connected to every neuron in the next layer
what has to be added to the output at each layer to enable it to become input in the next layer
the bias column
just like linear/polynomial regression or classification, MLP involves training and testing. What is the training method called?
Backpropagation
what are the 2 steps in backpropagation training
pass forward: to compute network response and errors at each output
back propagate: to pass back the error from the output to the hidden layers, to update all weight to optimise the network
what is the testing method
pass novel input data going forward, using fixed weights that were optimised during backpropagation.
estimate network response and predict output labels given novel inputs
what is the concept used to derive a better weight w during each iteration
gradient descent
what is a convolutional neural network(CNN)
a simplified variant of multi layer perception, still well connected but not fully
what is the purpose of a CNN
to significantly reduce the number of parameters, basically simplify the model without losing too much quality
how do CNNs make the computation easier?
they take advantage of repeated hierarchical structure in images
what are some examples of low, mid and high level structures
low: lines, curves
mid: shapes
high: groups of shapes that make up objects
