Exam 3 Flashcards
From the matlab training video, what are 2 examples of deep learning applications
- radiology reviews, preliminary diagonoses
- self driving algorthms
Deep learning performs…
end to end learning. It receives raw data and learns to perform the task defined by itself.
Shallow learning is…
supervised learning using an ANN with low complexity
What distinguishes deep learning from shallow learning?
- higher complexity (will have multiple networks within a network, numerous hidden layers)
- input information is also more complex
A common application of deep learning is…
image recognition
DCNN stands for
Deep Convolution Neural Network
DCNNs are commonly used for
image recognition and audio processing (computer vision!)
What is a convolution layer?
A thin layer that extracts a reduced set of information, similar to filtering in signal processing
A layer which takes detailed raw data you have an performs feature engineering is an example of a
convolution layer
A pooling layer performs
dimensional reduction
A fully connected layer…
restores global connections
A dropout layer can help…
avoid overfitting and slowness
What does a dropout layer do?
Randomly drops neurons and their connections in the network.
Simplifies the NN, analogous to pruning trees
What is a deep feed-forward layer?
All linear left to right flow of data and calculations. No circular data or feedback loops.
What can feedback loops be particularly useful for?
Helping the neural network learn from time dependent information.
What are some features of a recurrent neural network?
- has circular connections
- good for dynamic and time dependent data
What makes a network a “deep convolution” NN?
It has at least one or more convolution hidden layers
A _____ performs the opposite action of a convolution neural network
a Deconvolution NN
What is a Generative Adversarial Network?
- Generative = generates new content (images, texts, audio)
- Combination of DCNNs and FF
What is Transfer Learning?
Using an already trained neural network for a new NN and a similar data set
When performing transfer learning, you add…
Add a few new hidden layers on the right, and train on only those new layesr
What is a tensor:
a higher dimensional matrix
A _____ NN is used to classify and reduce dimensionality
Auto Encoder
classify = encode
Shallow learning tends to have ___ hidden layers
one or two
Deep learning is unsupervised or supervised
Supervised
A color photo with 1000x1000 pixels would make what dimension matrix?
1000x1000x3
3 for the colors (assuming RGB combo)
Gradient descent optimization gets very complex with DNNS, greater chance of getting stuck in a local minimum because there are….
Many many more weight terms to solve for (could be millions!)
One approach to improve gradient based optimization for deep NNs is a…
stochastic approach (mentioned in lecture)
A convolution layer extracts…
A subset of important information out of all the data you fed it (filtering)
One practical purpose of a sequence of convolution and pooling layers is to…
reduce the information down to the important inputs needed to actually run the calculations in subsequent layers that have the main learning algorithm
Recurrent NNs take more or less time to train than a typical FF NN?
More
