Week2. Logistic regression as NN

Question 1

Logistic regression for binary classification

Answer 1

Linear regression Y = w^T *x + b Linear regression doesn’t work well as values could be big. Sigmoid function from linear regression gives values between -1 and 1 .

Question 2

Sigmoid function

Answer 2

G(z) = 1 / (1+e^-z)

Question 3

Why Cost function

Answer 3

We need to learn parameters p and W^T given values X and Y of training exemples.

Question 4

Logistic regression loss function

Answer 4

Better NOT to use square error in this case (1/2*(Y-y)²

Good function for logistic regression is:

F(Y,y)= - ( y * log Y + (1-y) * log (1-Y) )

Question 5

Loss function for logistic regreassion

Answer 5

Is a loss function over all exemples

J(w,b) = - 1/m Sum(i=1,m)[y⁽ⁱ⁾ * log Y⁽ⁱ⁾ + (1-y⁽ⁱ⁾ * log(1-Y⁽ⁱ⁾ )]

Question 6

Differencial calculus

Answer 6

Tries to find rate of change at each point on the curve of a function, or slope of the curve at each point.

Derivative - is a slope = height / width of a triangle.

Question 7

Integral calculus

Answer 7

Finds area under the curve of a function untill X axis. If you draw rectangles you can add them up to get area and if you make them infinitely small - it will be an exact area under the curve.

Question 8

Gradient decent

Answer 8

Updates for w:=w-alpha * dJ(w,b) / dw

and for b := b - alpha * dJ(w,b) / db

Derivative measures slope of the function.

In code we will use

dw = dJ(w,b) / dw

Question 9

Gradient decent for logistic regreasion - one example

Answer 9

z=w1x1+w2x2+b

a= Q(Z)

Loss(a,y)

Gradient decent

dz = a-y

dw1 = x1*dz

Question 10

Vectorization

Answer 10

Save a lot of time - SIMD - single instruction multiple data

Z = w^T * x + b

Z=np.dot(w,x)+b

Question 11

Vectorizing Logistic regresion - Forward propogation

Answer 11

For one example z1 = w^Tx1+b and a1=zigmoid(z1)

Vectorized for m exemples Z = np.dot(w.T,X) + b

Z, w b is broadcasted in to a vector

Dimention of vector X is (n_x,m)

A=sigma(Z)

Question 12

Vectorizing Logistic regression Gradient output

Answer 12

Regular dz(1) = a(1) - y(1) … dz(m)

dZ = [dz(1) …dz(m)] / m

Vector version

dz = A - Y

dw = 1/m X dZ^T

db = 1 / m*(np.sum(dz))

Question 13

Logistic regression full vectorized

Answer 13

Z = w^T X + b = np.dot(w.T,X) + b

A = G(Z)

dz = A - Y

dw = 1/m X dz^T

db= 1/m np.sum(dz)

Question 14

Practice: Common steps for pre-processing a new dataset are:

Answer 14

Figure out the dimensions and shapes of the problem (m_train, m_test, num_px, …)

Reshape the datasets such that each example is now a vector of size (num_px * num_px * 3, 1)

“Standardize” the data