Ch 4 Flashcards
What is the primary goal of a simple artificial neuron model?
To reflect some neurophysiological observation, not to reproduce their dynamics.
What are the types of connections in an artificial neuron?
Inhibition and excitation connections.
What is the output of artificial neurons?
Real values.
What is a key limitation of two-layer neural networks?
They cannot solve problems like XOR without additional hidden layers.
How does a hidden layer contribute to neural networks?
It provides non-linear input space transformations.
What is the significance of deep neural networks?
They have more than one hidden layer, making intuition more difficult.
What are hyperparameters in the context of neural networks?
K layers (depth of network) and hidden units per layer (width of network).
What is hyperparameter optimization?
The process of retraining with different hyperparameters.
What is the typical range of layers in deep networks for best results?
50-1000 layers.
In which applications do deep networks yield the best results?
- Computer vision
- Natural language processing
- Graph neural networks
- Generative models
- Reinforcement learning
What theorem do both shallow and deep networks obey?
The universal approximation theorem.
What is depth efficiency in deep networks?
Some functions require a shallow network with exponentially more hidden units than a deep network to achieve an equivalent approximation.
Why are convolutional networks used in deep learning?
They allow weights to operate locally and share across images, integrating information gradually.
What happens to the fitting of deep models beyond about 20 layers?
Fitting becomes harder and various tricks are needed to train deeper networks.
What is a challenge in fitting deep networks?
Fitting of deep models is faster but becomes harder with more layers.
What is the next step after defining flexible networks for mapping multiple inputs to outputs?
To train them.
Fill in the blank: The number of linear regions per parameter increases significantly in deep networks due to their _______.
[depth]
True or False: Deep networks create fewer linear regions per parameter compared to shallow networks.
False.
