ANN, CNN Flashcards
softmax loss role in logistic regression
- translate the linear predictive value into category probability
-Imagine Zi = Wi*x + Bi is the result of linear prediction, Softmax can make Zi nonnegative by letting them become exponential, then the sum of all items is normalized, now each Oi = σi (Z) can be interpreted as the probability of data x belong to the category i, or the likelihood
Consider a dataset that has 200 samples. These samples take 1000 epochs or 1000 turns for the dataset to pass through the model.
What is the batch size?
How many batches are there?
How many times is the model updates
- batch size of 5.
This means that the model weights are updated when each of the 40 batches containing five samples passes through. Hence the model will be updated 40 times.
what is ANN
artificial neural network
- interconnected group of nodes
operation of single neuron
- each neuron on performs a simple operation on its input
1. computing
2. apply to some non-linear function to z ( y=f(z) )
what is CNN
- a special type of ANN for images
- heavily used for vision applications
- extract features automatically from images
- no need for hand engineered feature design
- high level of generalisation for different tasks (transfer learning)
what are convolution layers
- CNN is a sequence of convolution layers followed by an activation function
- output of one layer becomes input to the next layer
- allowing the network to learn more complex representations of the input.
what do conv layer require
- number of filters K
- filter size F
- stride S
- padding
what is max pooling layer
- used to make representation smaller and tractable
- operates over each activation map independently
- takes max over filter’s view
advantages of max pooling layer
- robustness to noise
- encodes the idea “did i find any match for the filter in the search area?”
