Deep Learning-based Deep Neural Networks and Applications Flashcards
1
Q
What is DL/DNN?
A
a subfield of machine learning that involves training neural networks to learn from data and make predicitons
2
Q
What fields are DNNs used in?
A
computer vision, NLP, speech recognition, etc
3
Q
What are deep learning algorithms designed to do?
A
automatically learn features from raw data by stacking multiple layers of nonlinear transformations
4
Q
What does the term “deep” refer to?
A
The number of layers which can range from a few to hundreds to thousands
5
Q
What can NN based on deep learning be viewed as?
A
a special case of an already existing multi-layer neural network
6
Q
What was the beginning of deep learning?
A
Neocognitron which had 7 layers
7
Q
By who and when was Neocognitron developed?
A
Dr. Fukushina in early 1980s
8
Q
What was Neocognitron used for?
A
handwritten japanese character recognition and pattern recognition
9
Q
What did Neocognitron inspire?
A
CNNs (it just didn’t use backpropagation)
10
Q
What did Hubel and Wiesel win the 1981 Nobel Prize in Physiology or Medicine for?
A
research on the visual system
11
Q
What did Hubel and Wiesel discover?
A
feature detectors (neurons that respond selectively to lines or edges)
12
Q
What kind of machine learning techniques are a majority?
A
SVM
13
Q
When was deep learning started and why?
A
in 2006 because CPUs and GPUs have become able to handle more massive amounts of data quickly
14
Q
How was deep learning different from existing multi-layer perceptrons (when it began)?
A
added pre-processing through learning, had up to 1000 hidden layers as opposed to 1 to 2
15
Q
What algorithm solves the overfitting problem?
A
dropout algorithm
16
Q
When did Hinton and his students win the Turing award for deep learning achievements?
A
2018
17
Q
Since 2009 how has deep learning advanced?
A
has begun to surpass existing pattern recognition methods
18
Q
What field is deep learning excelling in?
A
pattern recognition
19
Q
What is the difference between classification methods in machine learning and deep learning?
A
machine learning has separate feature extraction and classification steps while deep learning does those simultaneously and automatically
20
Q
What kind of data sets does deep learning learn from?
A
large scale data sets
21
Q
When is deep learning superior to other learning models?
A
when there are larger data sets
22
Q
What kind of operations does a CNN use?
A
convolutional operations
23
Q
What is CNN used for?
A
image analysis and image recognition
24
Q
How do CNNs learn?
A
using feature maps
25
What particular image recognition application are CNNs used for?
car systems
26
What are 3 major CNNs throughout history?
Neocognitron (1980), LeNet-5 (1989-1998), AlexNet (2012)
27
How does CNN perform object recognition?
measures the likelihood of an object's identity in an image
28
What is a loose definition of a CNN?
a neural network created to mimic the way the eye processes image information
29
How does CNN compare to FNN?
its more complex but more effective for image processing
30
How does CNN borrow techniques from cat visual cells?
the neurons that are activated depend on where the "cat" is looking at the screen (only specific areas of neurons are activated at certain times)
31
How is the CNN designed based off of cat visual cells?
there's a neural network that corresponds to neurons for each region on the screen
32
What is convolution?
an operation that multiplies the weights of surrounding pixel values and adds them then uses this as a new pixel
33
How do we implement a CNN to mimic the flow and processing of visual information in a human brain?
using convolutional filters
34
What is stride?
distance to apply the kernel (if stride=1 then move kernel 1 pixel at a time, stride=2 skip one pixel each move, etc)
35
If the original size is m*m, kernel size is n*n, and stride is s, what does the image size become?
(m*n)/(s+1) * (m*n)/(s+1) (aka 1/stride)
36
What are CNN weights?
numbers listed in the filter (correlates to the weight of FNN)
37
What is perceptron learning?
process of modifying weight (modifying numbers inside the filter) which allows features to be effectively extracted from the image
38
How do you use CNNs to mimic the human visual cortex?
stack CNNs like perceptrons allowing you to learn simple info in low layers and abstract info in high layers
39
What is a Hierarchical Feature Map?
the information processed at each layer of multiple stacked CNNs that is organized and expressed graphically
40
What is another word for a feature map?
activation map
41
What does a feature map show?
the result of extracting features from the input image
42
How do CNNs use extracted features?
for object recognition
43
True or false: As layers become deeper the original image becomes less abstract?
false, it becomes more abstract
44
How do CNNS build neural networks?
by connecting multiple layers
45
When and why would CNN use an FNN classifier attached to the back?
for video data, because CNN has excellent ability to compress video data but the output of CNN is only 2D so the FNN helps with that
46
What reduces the size of input data?
pooling aka subsampling
47
Why should we use pooling?
calculations are faster when the layer size is smaller and the probability of overfitting decreases with smaller neural network parameters
48
True or false: The higher the floor, the more information it contains?
true
49
What kind of features does classic computer vision use?
hand-crafted features
50
Why is computer able to have a certain level of performance?
because it is designed with human intuition
51
True or false: Computer vision has high performance on complex data like natural images?
false
52
What was the paradigm change in feature extraction brought by CNN?
learning feature extraction and classification is simultaneous
53
Why is CNN end-to-end learning?
since the entire process from input to output is learned at once
54
Which CNN layers are responsible for feature extraction?
convolution layer and pooling layer
55
Which CNN layer is responsible for classification?
FC layer
56
Why is CNN superior to traditional CV or FNN in performance?
end-to-end learning, feature learning, uses tens to hundreds of layers, maintains original structure of data, and partial connectivity and weight sharing
57
What does partial connectivity and weight sharing do in terms of performance?
dramatically reduces the number of operations
58
What is a RNN?
a recurrent neural network is a type of ANN that is capable of processing sequential data
59
What do RNNs have that is different from feedforward neural networks?
a feedback loop
60
What kind of data is RNN useful for?
sequential data such as time series, speech recognition, and handwriting recognition
61
What is sequential data?
ordered data (either temporal or spatial)
62
What are some examples of sequential data?
stock prices, text and audio data
63
What is necessary for processing sequential data and making accurate predictions?
to remember historical data
64
Why is there a need for RNN, aka a neural network to better handle sequential data?
standard neural networks don't remember data from the distant past
65
What are the functions of RNNs?
ability to: handle variable-length inputs, track long-term dependencies, and keep information about the order
66
What is recurrent data?
data used to train recurrent neural networks
67
What is necessary to train and RNN?
cutting data to certain lengths to make multiple training sample
68
What serves as a kind of memory that carries information from the past into the future?
when past learning influences future learning outcomes
69
What are some applications for RNNs?
language modeling and generation, music composition, speech recognition, and time series anomaly detection
70
What is a classification model?
ability to recognize features or objects and recognize emotions based off of facial expressions, mainly focuses on discernment abilities
71
What are some examples of classification models?
multi-layer perceptron, CNN, RNN, etc
72
What is a generative model?
AI that imitates human handwriting, art styles, etc.
73
What are some examples of generative models?
HMM and deep learning based generation model
74
What is the key difference between classification and generative models?
classification models determine which classes inputs belong to, and generative models learn patterns from input data
75
Describe a generative model.
a neural network that automatically discovers patterns in the training data and creates new samples that resemble the probability distribution of the training data
76
What does a generative model do to produce output?
learn the latent space representation of training data
77
What is an auto encoder (AE)?
a neural network that learns input data and performs encoding and decoding to produce an output identical to the input
78
What are AEs used for?
feature learning, dimensionality reduction, representation learning
79
What are some applications of AEs?
anomaly detection, abnormal financial transactions, visualization and restoration of data, meaning extraction, and image search
80
What are the elements of an AE?
encoder, latent space, decoder, loss function
81
What does the encoder in an AE do?
encodes input into the latent space
82
What does the latent space in an AE do?
high-level features that compress and represent the original pattern
83
What does the decoder in an AE do?
unpacks the latent space and restores it to the input
84
What does the loss function in an AE do?
uses MSE of input and output images
85
Does modern deep learning need an AE to achieve high performance?
no
86
Which generation models perform better than AE?
GAN and cycleGAN
87
What is RBM?
Restricted Boltzmann Machine, a model proposed by Hinton and is an unsupervised generative model that captures higher-order correlations in the data
88
How does RBM automatically fin patterns in data?
extracts important features and combines features to form patterns
89
What are applications of RBM?
feature extraction, dimensionality reduction, e.g. spectrum analysis
90
What kind of neural network does an RBM use?
shallow neural network with only 2 layers (visible layer and hidden layer)
91
What is GAN?
Generative Adversarial Network
92
When was GAN introduced and by who?
Ian Goodfellow and others in 2014
93
How is GAN implemented?
2 competing neural networks in a zero-sum game where one aims to fool the other by producing realistic candidates
94
What can GAN do?
create fake images or videos that look real
95
What are the 2 GAN NN called?
the generator (DCNN) and the discriminator (CNN)
96
What does the generator do?
creates new data instances and is trained to produce data that is similar to the training set
97
What does the discriminator do?
evaluates the data instances for authenticity and is trained to distinguish between the generated data and real data
98
What process trains the two NN together?
adversarial training
99
How does adversarial training work?
when is begins, the generator doesn't know what to generate so random noise is supplied as input, and it makes random noise output, then the discriminator has an easy time distinguishing between real and fake, but as training goes on, the replicas and distinguishing gets better
100
Why should we use random noise in generators?
using average pixel value calculation we can produce images but they will be the same every times, so generative models need stochastic random elements to influence the output, essentially the random noise acts as a seed
101
When did Google train a neural network to recognize cats?
2012
102
How did Google train the cat recognition model?
expose it to 10 million randomly chosen youtube video thumbnails over 3 days
103
What kind of learning did the cat recognition model use?
unsupervised learning to identify relevant parts of a photo based on data patterns
104
What kind of accuracy did the cat recognition network have?
70%
105
How many connections and computers did the cat recognition network have?
1 billion connections and 16,000 computers
106
How is deep learning used in computer games?
adjust difficulty, personalize, and improve immersion
107
How do computer games use deep learning to adjust game difficulty?
based on the player's performance it adjusts the difficulty to balance the difficulty and player skill level in order to improve player engagement
108
How do computer games use deep learning to personalize the game?
analyze player data and behavior to create tailored experiences and content recommendations
109
How do computer games use deep learning to improve immersion?
improve text to speech, speech to text, and speech synthesis
110
How does cycleGAN differ from GAN?
GAN uses 2 adversarial neural networks, and cycleGAN uses 4 (2 generators and 2 discriminators)
111
What are some global companies that are using deep learning technology?
google, MS, facebook, twitter, and baidu
112
How has google used deep learning?
voice recognition, translation, cat image recognition, youtube recommendations, automatic tagging, etc.
113
How did google contribute to the deep learning research field?
released TensorFlow source code
114
How has MS used deep learning?
detecting breed of dog from photo, simultaneous interpretation technology for Cortana and Skype
115
What is MS's main deep learning project?
Adam Project
116
How has Facebook used deep learning?
creating DeepFace to recognize human faces, faces can be recognized from various angles or lighting based on user-uploaded pictures, translator tool
117
What kind of accuracy does DeepFace have?
97%
118
How has Twitter used deep learning?
photo analysis
119
How did Baidu break into deep learning?
Prof. Andrew Ng, the leader of the Brain project at Google, moved to Baidu in 2014
120
Which domestic (Korean) companies use deep learning?
Naver, Daum Kakao, Samsung, LG, etc
121
How does Naver use deep learning?
voice recognition, news summaries
122
What is evolving through deep learning?
image recognition
123
How has image recognition improved?
accuracy increased and recognition time decreased
124
What kind of deep learning technology will emerge in the near future?
XAI (Explainable AI) which means in addition to recognition results, the reason is also explained
125
Where are some current limitations of deep learning?
in text, video, and voice recognition
126
What are the current expectations of deep learning?
learning on large scale data, range getting wider, accuracy improving, scope of application is expanding (many high expectations for the 4th industrial revolution)
127
Why are GPUs important for deep learning?
they can performs multiple computations simultaneously
128
Why are GPUs optimized for training deep learning models?
they have thousands of cores
129
How much faster is a GPU than a CPU?
3 times faster when processing parallel tasks
130
What is crucial for speeding up deep learning neural network training and why?
parallelism because it requires millions of calculations
131
What hardware was developed for the parallel processing that is needed by deep learning algorithms?
NPU (neural processing unit)
132
What is NPU?
a processing unit specifically designed to speed up neural networks also called IPUs (intelligent processing unit)
133
How does an NPU differ from a GPU?
NPU is specialized for AI tasks and GPU is versatile
134
What range of tasks is a GPU good for?
graphics rendering, scientific simulations, and computation heavy tasks
135
What market is rapidly increasing?
the AI semiconductor market
136
What hardware is currently being developed for AI?
next generation NPUs
137
Why is TensorFlow used for ML and DL?
it has convenient software kits that returns results when you put information into the library
138
When was TensorFlow released and by who?
2015 by Google
139
What is TensorFlow?
a deep learning framework implemented in C++ with multiple interfaces that can be accessed from multiple languages through Python
140
How does TensorFlow visualize various tasks?
dataflow graphs
141
What language is Keras written in?
Python
142
What is the most popular backend to use with Keras?
TensorFlow
143
What kind of neural networks does Keras use?
feedforward, convolutional, and recurrent neural networks and combinations of them
144
What kind of hardware processing units can Keras run on?
CPU and GPU
145
What model type is used in Keras as the core data structure?
sequential linear stack model
146
How is Google utilizing TensorFlow?
search, ads, google maps, street view, translation, YouTube, etc
147
How is TensorFlow utilized for ML and DL?
recognition or cursive letters or numbers, voice, image, NLP and machine translation
148
Who was PyTorch developed by?
Facebook's AI team
149
What is PyTorch?
a Python-based open source ML library
150
When was PyTorch released?
2016, then stabilized in April 2019
151
what is PyTorch used for?
NLP
152
129-133
