lecture 7 Flashcards
What is deep learning?
A subfield of machine learning that focuses on training deep neural networks.
How is a neural network represented?
As a graph where each node represents a scalar value computed from incoming edges.
What is the role of weights in a neural network?
Weights determine how much influence each input has on the output.
What is the feedforward network?
The simplest way of wiring up a neural net where data flows from input to output without loops.
Why is linear algebra important in neural networks?
Most operations in neural networks can be written as matrix multiplications.
What are tensors in deep learning?
Generalizations of vectors and matrices to higher dimensions.
What is the rank of a tensor?
The number of dimensions along which the values change.
What is a scalar in tensor notation?
A rank-0 tensor.
What is a vector in tensor notation?
A rank-1 tensor.
What is a matrix in tensor notation?
A rank-2 tensor.
What is a 3-tensor?
A three-dimensional array of numbers.
What is one common way to represent images in deep learning?
As a 3-tensor with width, height, and color channels.
What is a computation graph?
A graph that details the computations of a model and its loss function.
What is the purpose of a computation graph?
To allow automatic differentiation for backpropagation.
What is automatic differentiation?
A method for computing gradients by tracking computations in a computation graph.
What are the two approaches to automatic differentiation?
Lazy execution and eager execution.
What is lazy execution?
A method where the computation graph is built first, then executed.
What is eager execution?
A method where the computation graph is built on the fly during execution.
What is backpropagation?
An algorithm for computing gradients in neural networks by propagating errors backward.
How does backpropagation use computation graphs?
It walks backward through the graph to compute gradients.
What is the difference between lazy and eager execution?
Lazy execution optimizes computation before running, while eager execution executes immediately but is harder to optimize.
What is an example of a deep learning framework using lazy execution?
TensorFlow 1.
What is an example of a deep learning framework using eager execution?
PyTorch and TensorFlow 2.
What does a function in deep learning consist of?
A forward pass (computing outputs) and a backward pass (computing gradients).
What is gradient descent used for?
To update neural network weights based on computed gradients.
What happens if the learning rate is too high in gradient descent?
The model may fail to converge.
What happens if the learning rate is too low in gradient descent?
The model may take too long to learn.
Why are tensors useful for deep learning computations?
They allow for efficient parallelization and computation on GPUs.
What is the main takeaway from automatic differentiation?
It enables efficient gradient computation, simplifying neural network training.
