Lecture 08 - Artificial neural networks Flashcards
Artificial neural networks
What is the name of this function?
Step function
Artificial neural networks
What is the name of this function?
Sigmoid
Artificial neural networks
What is the name of this function?
Hyperbolic tangent (tanh)
Artificial neural networks
What is the name of this simple network?
Perceptron
Artificial neural networks
What is the name of this function?
Rectified Linear Unit (ReLU)
Artificial neural networks
What is the name of this function?
Leaky ReLU
Artificial neural networks
Why are nonlinear activation functions used?
Without nonlinear activation functions, the neural network would only be able to model linear functions
Artificial neural networks
What is a feed-forward neural network?
A network in which each neuron is passed information only one time.
Artificial neural networks
What is a fully connected layer?
A layer where each neuron is connected to all neurons in the next layer.
Artificial neural networks
What is a layer where each neuron is connected to all neurons in the next layer called?
A fully connected layer.
Artificial neural networks
What is deep learning?
Deep learning is the research field that works with deep neural networks
Artificial neural networks
What is a deep neural network?
Deep neural networks have three or more hidden layers
Artificial neural networks
What is the name for a neural network with 3+ layers?
Deep neural network
Artificial neural networks
What is a recurrent neural network (RNN)?
A network where information can pass more than one time to the same neuron.
Artificial neural networks
What is a network where information can pass more than one time to the same neuron called?
Recurrent neural network (RNN)
Artificial neural networks
What does it mean to unfold a recurrent neural network?
Diagrams of RNNs are often written in compressed form.
Unfolding means untangling the diagram to show each time step being processed.
Artificial neural networks
What is the image an example of?
Unfolding a recurrent neural net (RNN)
Artificial neural networks
What is a bidirectional RNN?
An RNN with an additional layer for input in the reverse order
Artificial neural networks
What’s depicted in the image?
A bidirectional RNN.
Artificial neural networks
What types of data are RNNs good at processing? (SSTV)
- Serial data
- Sequential data
- Time-series data
- Variable-length data
Artificial neural networks
What are some common applications for RNNS? (TVNS)
- Time-series prediction
- Video analysis
- Natural Language Processing (NLP)
- Speech recognition
Artificial neural networks
What is this image an example of?
A convolutional layer
Artificial neural networks
What is a kernel in an convolutional layer?
Tensors that slide across input data, performing calculations on patches of the inputs.
E.g. a 3x3 kernel slides across an image to blur it.
Artificial neural networks
What is this called (see image)?
Convolution
Artificial neural networks
What is this called (see image)?
Pooling (Max-pooling)
Artificial neural networks
What is max-pooling?
Reducing the size of an input tensor by selecting the max value within a window.
Artificial neural networks
What are some common applications of convolutional neural networks (CNNs)? (IVG)
- Image classification
- Video analysis
- Image generation
Artificial neural networks
What is CNN short for?
Convolutional neural network
Artificial neural networks
What is RNN short for?
Recurrent neural network
Artificial neural networks
What is an autoencoder?
A network with a “bottleneck”
Artificial neural networks
What is this an example of?
An autoencoder
Artificial neural networks
How does an autoencoder work?
Autoencoder has a bottleneck that downscales (compresses) an input to its most important representation, then tries to upscale that representation to recreate the original input.
Artificial neural networks
What are some common applications for autoencoders? (DCOGS)
- Image denoising
- Data compression
- Object detection
- Image generation
- Super-resolution
Artificial neural networks
What is GAN short for?
Generative adversarial network
Artificial neural networks
What is the image an example of?
A generative adversarial network (GAN)
Artificial neural networks
What parts does a GAN consist of? (2)
- The generator
- The discriminator
Artificial neural networks
In a GAN, what does a generator do?
It creates data that resembles the real ones
Artificial neural networks
In a GAN, what does a discriminator do?
It’s a binary classifier which identifies if the data is real or created by the Generator
Artificial neural networks
What are some applications of GANs? (1)
- Image generation
Artificial neural networks
What is a LIF short for?
Leaky-integrate-and-fire
Artificial neural networks
What does a leaky integrate-and-fire model do?
- It takes input and integrates (sums) them.
- The output is usually a 0.
- If the model has summed past a threshold, the sum is reset (or reduced by some fixed amount) and a 1 is sent as output.
- The leakiness means the value decays over time.
Artificial neural networks
What is the image an example of?
A leaky-integrate-and-fire (LIF) neuron.
Artificial neural networks
What purpose does the weight in a LIF model have?
Weight or synaptic strength defines how much an incoming spike affects the membrane potential or neuron state of the postsynaptic neuron
Artificial neural networks
Which two types of neurotransmitters can spiking neurons release?
Excitatory and Inhibitory
Artificial neural networks
What do Excitatory neurotransmitters do? (2)
- Increases the membrane potential.
- increases the probability of producing a spike.
Artificial neural networks
What do Inhibitory neurotransmitters do? (2)
- Reduces the membrane potential.
- Reduces the probability of producing a spike.
Artificial neural networks
What is a central pattern generator?
A generator that produces rhythmic outputs in the absence of rhythmic inputs.
Artificial neural networks
What is this image an example of?
Outputs from a central pattern generator
Artificial neural networks
Who proposed sound localization with spike-coincidence neurons?
Jeffress
Artificial neural networks
When was sound localization with spike-coincidence neurons proposed?
1948
What is Sound localization with spike-coincidence neurons?
Detecting the location (direction) of incoming sound by producing output spikes when signals arrive at neurons close in time.
