Neural Networks Flashcards
What is logistic regression used for?
Binary Classification tasks
What is the sigmoid function in logistic regression?
σ(x) = 1 / (1 + e^-x), it converts the relation of the input with the separating hyperplane and outputs a probability.
How are the weights estimated in logistic regression?
Using maximum likelihood estimation
What is a key advantage of neural networks over logistic regression?
Neural networks can learn complex non-linear relationships in data
What is the vanishing gradient problem in deep neural networks?
The gradient becomes very small for earlier layers, making it difficult to train deep networks
What is an embedding in machine learning?
A learned representation of categorical data in a continuous vector space
What is the purpose of an autoencoder?
To learn compact representations (encodings) of data
What is a convolutional neural network (CNN) typically used for?
Back: Image processing and computer vision tasks
What property do convolutions provide to CNNs?
Translation equivariance
What is transfer learning?
Adapting a pre-trained model to a new but related task
What activation function helps mitigate the vanishing gradient problem?
ReLU (Rectified Linear Unit)
What is the purpose of pooling layers in CNNs?
To add some translation invariance and reduce spatial dimensions
What is the softmax function used for in neural networks?
To convert raw scores into probabilities for multi-class classification
What is the difference between batch gradient descent and stochastic gradient descent?
Batch uses all training samples per update, while stochastic uses one sample at a time
What is mini-batch gradient descent?
A compromise between batch and stochastic, using a small subset of samples per update
