10 + 11 Flashcards
What are the two main approaches to training artificial neural networks?
Deep belief nets and convolutional neural networks.
How does a deep network work?
Encodes complex functions using multiple hidden layers, the first layer learns simple patterns, the next learns patterns from this. Visual object classification is good for this.
How are images processed in a feed forward network?
Image treated as array of pixels, each neuron in a layer is connected to every neuron in previous layer.
How does a convolutional neural network process images?
A neuron takes input from a subimage(specified by user), another neuron with same wights takes input from same size subimage in different location(specified by user). This forms a 2D map of neurons sensitive to the same feature. This is done with many neurons to create more feature maps. Each feature map is like its own image. Pooling then occurs for each feature map via the same process to build a sub sampled map. A feature map can then be formed from this(treated as an image with colours = to number of feature maps). Second last will be the fully connected layer, in which each neuron connects to each neuron of every final feature map. This then does a standard feedforward to softmax neural network layer.
What is pooling?
Max pooling takes a subset of neighbouring neurons from a feature map and only retains the one with the highest output.
What is used to train a convolutional neural network?
Backpropagation. It is analogous to an MLP with lots of neurons but few weights. Note that the pooling layer has no trainable parameters, but can pass error “blame” backwards.
What are the typical activation functions used in a convolutional neural network?
Rectifier linear unit for convolutional layers, sigmoid function for fully connected MLP, and softmax for output.
What parts of a convolutional neural network must be specified by the user?
Convolutional layer: number of filters, size of filters, step size, padding.
Pooling layer:Size of pooling window, step size, type of pooling(max or average)
What are the biggest problems with convnets?
As long as a combination of features is found the actual order is typically ignored. State of the art networks are trained on huge training sets for this reason.
What are the basics of a genetic algorithm?
A species has some definition of fitness for each organism in it, each individual has variations in genes, giving genetic diversity(in combination with mutations). The most successful individuals will pass on their genes.
A genetic algorithm is essentially an optimisation problem.
What is a chromosome?
A sequence of characters from some alphabet.
What is the genetic algorithm steps?
- create population with random chromosomes.
- Evaluate fitness of each member
- pick two parents randomly as function of fitness.
- create new individual by mixing chromosomes(and mutations).
What are the two most common parent selection methods?
Roulette wheel: Sum fitness of all individuals in population, normalise to sum to 1. Express fitness of each individual as range in interval. Choose random number from 0,1. Pick individual whose range includes random number.
Tournament selection: Pick subset of n individuals from population at random, two fittest individuals are chosen as parents.
How are chromosomes mixed?
Crossover: choose a crossover point in chromosome. One side is the first parent’s chromosome, the other is the second parent’s chromosome. Next, using a low probability introduce a mutation in the child.
What are the parameters of a genetic algorithm?
How the chromosome encodes possible solutions, population size, selection method, mutation chance, crossover probabilities, elitism(retaining fittest individuals for next generation).
