classification (neural nets) Flashcards
What is a neural network?
A set of matrix multiplies to apply X to get Y
Why is it called feed-forward?
None of the weights cycle back to an input unit
What does fully connected mean?
Each unit provides input to each unit in the next forward layer
Give a general summary of what feed forward neural networks do.
Each output unit takes a weighted sum of the outputs from units in the previous layer, then applies an activation function to the weighted input
What happens at the input layer?
Inputs are fed into the input layer, weighted, and fed simultaneously to the hidden layer
What is the hidden layer?
Second layer of “neuronlike” units. Outputs of the hidden layer can be input to another hidden layer
What is the output layer?
Weighted outputs of last hidden layer. It emits the network’s prediction for given tuples
What can you add to a 2 layer neural network to make it a logistic model?
A sigmoid function
What does adding bias do?
It’s like adding a constant “hot” feature to your instance. It acts as an additional node at every layer except for the output layer.
What is backpropagation?
An algorithm for iteratively improving the prediction of the model by updating the model’s weights based on prediction error (loss)
What makes backpropagation “backwards”?
Modifications are made in the backwards direction through each layer down to the first hidden layer
What are the 4 steps of backpropagation?
1) Initialize the weights
2) Propagate inputs forward
3) Backpropagate the error
4) Terminating condition
What is an epoch?
After all data has been included in a batch, this is called an epoch
What is stopping criteria?
Decides when number of epochs is sufficient
