Deep Learning-based Deep Neural Networks and Applications Flashcards
What is DL/DNN?
a subfield of machine learning that involves training neural networks to learn from data and make predicitons
What fields are DNNs used in?
computer vision, NLP, speech recognition, etc
What are deep learning algorithms designed to do?
automatically learn features from raw data by stacking multiple layers of nonlinear transformations
What does the term “deep” refer to?
The number of layers which can range from a few to hundreds to thousands
What can NN based on deep learning be viewed as?
a special case of an already existing multi-layer neural network
What was the beginning of deep learning?
Neocognitron which had 7 layers
By who and when was Neocognitron developed?
Dr. Fukushina in early 1980s
What was Neocognitron used for?
handwritten japanese character recognition and pattern recognition
What did Neocognitron inspire?
CNNs (it just didn’t use backpropagation)
What did Hubel and Wiesel win the 1981 Nobel Prize in Physiology or Medicine for?
research on the visual system
What did Hubel and Wiesel discover?
feature detectors (neurons that respond selectively to lines or edges)
What kind of machine learning techniques are a majority?
SVM
When was deep learning started and why?
in 2006 because CPUs and GPUs have become able to handle more massive amounts of data quickly
How was deep learning different from existing multi-layer perceptrons (when it began)?
added pre-processing through learning, had up to 1000 hidden layers as opposed to 1 to 2
What algorithm solves the overfitting problem?
dropout algorithm
When did Hinton and his students win the Turing award for deep learning achievements?
2018
Since 2009 how has deep learning advanced?
has begun to surpass existing pattern recognition methods
What field is deep learning excelling in?
pattern recognition
What is the difference between classification methods in machine learning and deep learning?
machine learning has separate feature extraction and classification steps while deep learning does those simultaneously and automatically
What kind of data sets does deep learning learn from?
large scale data sets
When is deep learning superior to other learning models?
when there are larger data sets
What kind of operations does a CNN use?
convolutional operations
What is CNN used for?
image analysis and image recognition
How do CNNs learn?
using feature maps
What particular image recognition application are CNNs used for?
car systems
What are 3 major CNNs throughout history?
Neocognitron (1980), LeNet-5 (1989-1998), AlexNet (2012)
How does CNN perform object recognition?
measures the likelihood of an object’s identity in an image
What is a loose definition of a CNN?
a neural network created to mimic the way the eye processes image information
How does CNN compare to FNN?
its more complex but more effective for image processing
How does CNN borrow techniques from cat visual cells?
the neurons that are activated depend on where the “cat” is looking at the screen (only specific areas of neurons are activated at certain times)
How is the CNN designed based off of cat visual cells?
there’s a neural network that corresponds to neurons for each region on the screen
What is convolution?
an operation that multiplies the weights of surrounding pixel values and adds them then uses this as a new pixel
How do we implement a CNN to mimic the flow and processing of visual information in a human brain?
using convolutional filters
What is stride?
distance to apply the kernel (if stride=1 then move kernel 1 pixel at a time, stride=2 skip one pixel each move, etc)
If the original size is mm, kernel size is nn, and stride is s, what does the image size become?
(mn)/(s+1) * (mn)/(s+1) (aka 1/stride)
What are CNN weights?
numbers listed in the filter (correlates to the weight of FNN)
What is perceptron learning?
process of modifying weight (modifying numbers inside the filter) which allows features to be effectively extracted from the image
How do you use CNNs to mimic the human visual cortex?
stack CNNs like perceptrons allowing you to learn simple info in low layers and abstract info in high layers
What is a Hierarchical Feature Map?
the information processed at each layer of multiple stacked CNNs that is organized and expressed graphically
What is another word for a feature map?
activation map
What does a feature map show?
the result of extracting features from the input image
How do CNNs use extracted features?
for object recognition
True or false: As layers become deeper the original image becomes less abstract?
false, it becomes more abstract
How do CNNS build neural networks?
by connecting multiple layers
When and why would CNN use an FNN classifier attached to the back?
for video data, because CNN has excellent ability to compress video data but the output of CNN is only 2D so the FNN helps with that
What reduces the size of input data?
pooling aka subsampling
Why should we use pooling?
calculations are faster when the layer size is smaller and the probability of overfitting decreases with smaller neural network parameters
True or false: The higher the floor, the more information it contains?
true
What kind of features does classic computer vision use?
hand-crafted features
Why is computer able to have a certain level of performance?
because it is designed with human intuition
True or false: Computer vision has high performance on complex data like natural images?
false
What was the paradigm change in feature extraction brought by CNN?
learning feature extraction and classification is simultaneous
Why is CNN end-to-end learning?
since the entire process from input to output is learned at once
Which CNN layers are responsible for feature extraction?
convolution layer and pooling layer
Which CNN layer is responsible for classification?
FC layer
Why is CNN superior to traditional CV or FNN in performance?
end-to-end learning, feature learning, uses tens to hundreds of layers, maintains original structure of data, and partial connectivity and weight sharing
What does partial connectivity and weight sharing do in terms of performance?
dramatically reduces the number of operations
What is a RNN?
a recurrent neural network is a type of ANN that is capable of processing sequential data
What do RNNs have that is different from feedforward neural networks?
a feedback loop
What kind of data is RNN useful for?
sequential data such as time series, speech recognition, and handwriting recognition
