Perceptron

Question 1

What are the type of learnign algorithms?

2

Answer 1

1) Supervised learning = you train the model using a labeled dataset, where the input comes with the correct output (target). The goal is to map the inputs to the outputs.
2) Unsupervised learning = you train the model using unlabeled data, so the model tries to find patterns, structures, or groupings in the data without known outputs. Here the goal is to find hidden structures and groupings in the data.

Question 2

Perceptron

Answer 2

An algorithm for supervised learning of binary classifiers.

Question 3

Gradient Descent

Answer 3

Gradient descent is an optimization algorithm used to minimize the loss function of a machine learning model. The loss function measures how bad the model’s predictions are compared to the actual values — the goal is to find the model parameters (like weights) that make this loss as small as possible.

1) Start with some initial guess for the model parameters (e.g. random weights).
2) Calculate the gradient (partial derivatives) of the loss function — this tells us the direction in which the loss increases the most.
3) Move in the opposite direction of the gradient — this is the “descent” part.
4) Update the parameters a little bit using a small step size called the learning rate.
5) Repeat until the loss function reaches a minimum (or gets close enough).

Question 4

Feed Forward Neural Network

Answer 4

Slide 262 and 263

Question 5

What are the different kinds of gradient descent?

3

Answer 5

1) Batch Gradient Descent - all the training data is taken into consideration for each iteration.
2) Stochastic Gradient Descent - process one training sample in every iteration.
3) Mini batch gradient descent - process N number of samples in every iteration.