Deep Neural Networks (PowerPoint)

Question 1

In general, what do deep learning algorithms do?

Answer 1

Learn patterns and make predictions.

Question 2

What type of technique does deep learning algorithms use to adjust their weights?

Answer 2

Backpropogation, a technique used for training feedforward neural networks.

1. Forward pass: The neural network takes in some input data and processes it through its layers to make a prediction
2. Calculate error: After making a prediction, the network compares its prediction to the actual answer to see how wrong it was. This difference is called the error.
3. Backward pass (backpropagation): The network goes backwards through its layers, from the output layer to the input layer, to figure out how much each neuron contributed to the error.
4. Adjusting weights: Once the network knows which neurons were responsible for the error, it can tweak their weights. This helps the network learn from its mistakes and improve its predictions over time.

Question 3

What is deep learning?

Answer 3

Subfield of machine learning that intvolves training artificial neural networks with multiple layers (hence the term “deep”), allowing them to learn hierarchical representations of data.

Question 4

Feature Extraction

Answer 4

Feature extraction is about identifying and selecting the most relevant features from the data. Often, you have a lot of data with many different aspects or characteristics (features), but not all of them are equally important for solving a particular problem or making accurate predictions.

Question 5

What are the pros of deep learning? What is deep learning ideal for?

Answer 5

Highly flexible and can adapt to new data and tasks. This makes them ideal for applications where the data or requirements may change over time.

Question 6

What are Convolutional Neural Networks (CNN)?

Answer 6

A class of deep neural networks primarily used for analyzing visual imagery.

1. Convolutional layers use filters (kernels) to extract features (ex. edges, shapes and textures). The filters slide over the input data, performing a mathematical operation called convolution. By stacking multiple convolutional layers, CNNs can learn more complex and abstract features.
2. Pooling layers used to reduce the size of the feature maps produced by the convolutional layers. This makes the network more computationally efficient. The most common type of pooling is max pooling, which selects the maximum value from a small region of the feature map.
   Helps simplify the information by reducing the size of the feature maps while retaining the most important information.
3. Fully connected layers: The output from the convolution and pooling layers is flattened and fed into the fully connected layers, which perform a classification task.

Question 7

What are Recurrent Neural Network (RNN)?

Answer 7

Designed for data that comes in sequences, like text, speech or time-series data. Instead of treating each piece of data in isolation, they consider the order and context of the entire sequence.

RNNs have loops in their architecture. This enables them to process sequences of inputs.

They maintain an internal state or “memory”. This memory is then used to inform future predictions or decisions.
This allows them to keep track of context and make predictions based on what they’ve seen so far.

Question 8

What can Recurrant Neural Networks (RNN) be used for? What are the limitations?

Answer 8

RNNs can be used for:
- Language modeling
- Speech recognition
- Machine translation
- Image captioning
- Music generation

One limitation is that RNN are computationally expensive to train.

Question 9

What is one of the most common RNN architectures?

Answer 9

Long Short-Term Memory (LSTM).

They are designed to handle long sequences of data and remember important information over extended periods of time.

Question 10

What are Generative Adversarial Networks (GAN)?

Answer 10

A type of neural network architecture that consists of two main parts:
1. A generator network: Takes a random noise vector as input generates new data, ex. images or text
2. A discriminator network: Takes in both real and generated data. Tries to classify which is which.

The two networks are trained together in a game-like fashion where the generator tries to fool the discriminator and the discriminator tries to correctly classify the data.

Question 11

What is the limitation of GANs?

Answer 11

One limitation of GANs is that they can be difficult to train and may suffer from mode collapse, where the generator produces limited variations of the same output instead of diverse outputs.

Question 12

How are deep fake videos created?

Answer 12

Deep fake videos are created by training a deep neural network with a large dataset of real videos and images where it generates new videos that are similar in appearance to the original videos.

The neural network is trained to learn the patterns and characteristics of
- The face and body movements
- Other visual and audio features present in the original videos.

To create a deep fake video, a person first needs to collect a large number of images and videos of the person they want to impersonate. The algorithm then generates a new video that combines the facial and speech features of the person being impersonated with the audio and visual content from another video. This new video is then refined to produce more convincing and realistic results.