Overview: Deep Learning

Question 1

What type of data (I.e. format) is used for the input neuron layer in a deep learning model?

Answer 1

Each input node (aka neuron) is going to have 1 independent value

These independent values are going to be from 1 row
I.e. a deep learning net processes 1 observation at a time

E.g. —> = current observation being processed

 Subject.   Height.   Weight.   Gender  —>.     1.          164.          56.          F
       2.          195.          89.          M

Question 2

How does a hidden neuron derive its value?

Answer 2

Each hidden neuron receives multiple signals (input values) from multiple neurons.

When a neuron receives all signals it:

1. Adds up the weighted sum of all the inputs (weight + input)
2. Then applies an activation function

E.g. o = input neuron. O = hidden neuron

o -1-> (pretend -> is tilted so it’s touching O)
o -0.5-> O (1.7 + activation function)
o -0.2->

Note: all input values have been standardised before starting

Question 3

What’s the point of an activation function?

Answer 3

Transforms the weighted sum ‘back’ to a nr between 0-1, so that nee value can be passed on to the next neuron

E.g. o = input neuron. O = hidden neuron

o -1-> (pretend -> is tilted so it’s touching O)
o -0.5-> O (1.7 + activation function = 0.4)
o -0.2->

Question 4

What are the types of activation functions?

Answer 4

Threshold:
Transforms the summed weighted value into 0 or 1 (Yes or No basically) e.g.
Thus, used for binary classification

Sigmoid:
Transforms the summed weighted value into a nr between 0-1
Used for probability… the probability that it’s a cat or dog. Can be used for the probability that it’s yes or no
Often applied to output layer neurons

Rectifier:
Most popular and often applied to hidden neuron layers

Tanh: ?