13-neural networks Flashcards
What is a neural network?
A neural network is a combination of processing units (perceptrons)
What are the three types of layers in a neural network?
Input layers
Output layers
Hidden layers
What is linear classification?
A classification with a decision boundary of a linear combination of features. It cannot learn feature interactions easily
How does non-linear classification relate to neural networks?
Neural networks with at least 1 hidden layer and non-linear activations are non-linear classifiers. The decision boundary is a non-linear function of the inputs. It captures feature interaction.
What is feature learning and how does it apply to neural networks?
Neural networks take as input raw data and learn features as intermediate representations as part of their task
How are non-numerical features handled in neural networks?
Non-numerical features must be mapped to numerical features
How do activation functions work in neural networks?
Each hidden layer performs a non-linear transformation through an associated activation function.
What are common activation functions?
Sigmoid, hyperbolic tan, rectified linear unit
What does the universal approximation theorem suggest?
A feed-forward neural network with a single hidden layer can approximate any continuous function on R
When is linear classification enough?
If classes are linearly separable
If feature space is very high dimensional
If training set is small
If interpretability is important
What are the pros of neural networks?
Powerful
Universal approximates
Automatic feature learning
Very good performance for many tasks
What are the cons of neural networks?
High chance of overfitting
Requires large training datasets
Requires significant resources
Lacks interpretability
