Week 2: Deep Learning Flashcards
How does deep learning take inspiration from neurons?
Just like nuerons, the information, which is the output, will be shared via connections.
What are the layers of a neural network?
input layer, hidden layer, output layer
*a deep nueral network has multiple hidden latent layers
What do hidden layers do?
-Hidden layers tries to learn the right way to represent the data
-Each layer changes the representation of data into a new representation.
Crituque of deep neural network
A lot of the time, we don’t know what’s going on in a neural network
Why was there a lack of breakthroughs in neural networks between 1990 and 2010?
-lack of data
-hardware constraints
Explain how lack of data slowed the progression of neural networks?
Training a neural network requires a lot of data and their was lack of image data during this time
What was the solution to the lack of data problem?
-the advent of the iphone camera in 2007
-With the phone people take many photos, so there’s an explosion of photo data – tons of images are needed to train a neural network
-allowed for creation of ImageNet
What is ImageNet
-database with more than 14 million images that have been hand annotated to indicate what objects are
-used to train AI for image identification
What is AlexNet
a neural netwrok architecture used to identify images
AlexNet significance
-dropped the error rate for recognizing images dramatically
-After this all the other model died and everyone else turned to neural networks/deep learning
-First time a very large network was used and showed that a large data set is better than a small data set
Explain how hardware constraints limited progress?
CPU for sequential processing was not ideal for neural network
What helped resolve the issue of hardware constraints?
The popularity of video game contributed to more money being invested into the software
What do we now use instead of CPUs for deep learning?
GPUs
GPU
graphics processing unit: a specialized processor originally designed to accelerate graphics rendering
How are GPUs better for neural networks?
-GPUs can process and generate images and videos that are very similar to reality
-they are simple, parallel, distributed processing
What is the importance of AlphaGo in 2016?
-The first time the public began to understand the capabilities of AI
How does AlphaGo work?
-It can “see” the pattern of the game board better
-Bypasses strategic reasoning and treats the game board as an image and looks at the probability of winning with each move
-The game board is the input
AI and Faces
-In 2023, the ability of AI to create a human generated face greatly exceeded
-Since it has greatly improved, people are not working on face as much
-Now they are working on creating images beyond just a frontal shot but photos of people with different angles and backgrounds
What is different about deep learning from non-deep learning?
Deep learnings has more neural-networks in the hidden layer
