Math

Question 1

What are the derivatives of the following?

a. x^5
b. 2x^(-10)
c. log10(a)
d. 7

Answer 1

a = 5x^4
b = -20x^(-11)
c = 1 / (a*ln(10))
d = 0

Question 2

Integrate the following.

a. x^2
b. -sin(x)
c. X^(-1)
d 7x^(1/3)

Answer 2

a = ((x^3) / 3) + C
b = cos(x) + C
c = ln(x) + C
d = (21x^(4/3) / 4) + C

Question 3

What are the derivatives of the following?

a. h(x) = f(x) * g(x)
b. h(x) = f(g(x))
c. h(x) = 6x^3 - 2x^(-2)
d. h(x) = sin(x^2)

Answer 3

a = f’(x)g(x) + f(x)g’(x)
b = f’(g(x))g’(x)
c = 18x^2 + 4x^-3
d = 2x*cos(x^2)

Question 4

What are the derivatives of the following?

a. e^x
b. ln(x)
e. cos(x)
b. 2x^2 * 3x^3

Answer 4

a = e^x
b = 1/x
c = -sin(x)
d = 30x^4

Question 5

When are two vectors orthogonal? When is a single matrix orthogonal?

Answer 5

Vector:
When their dot product is 0.
When the angle between them is 90 degrees.

Matrix:
When A^T*A = Identity matrix

Question 6

When is a matrix invertible?

Answer 6

• When its determinant != 0
• When it is full rank
• If there exists another matrix such that AB = BA = Identity

Question 7

What is an orthogonal matrix?

Answer 7

When all rows and columns are orthogonal vectors and
A*A^t = A^t * A = Identity

Question 8

How to represent a 3D transformation?

Answer 8

You could use a 3x3 matrix (matrix of 3 vectors) to represent the transformation of the unit vectors in the x, y and z direction respectively. This breaks down a complex rotation into three simple rotations.

Question 9

What are eigenvectors and eigenvalues?

Answer 9

Eigenvectors: Vectors that stay on the same span after a transformation is applied. These vectors can be scaled. The factor by which they are scaled are the eigenvalues. Essentially the axis of rotation.

Question 10

What is the determinant of a matrix?

Answer 10

The factor by which the matrix transformation changes any area in space.
It is zero if the dimensionality is reduced after the transformation

Question 11

What is the transpose of a matrix?

Answer 11

When you flip the rows and columns of a matrix

Gives you the reverse of the matrix transformation

Question 12

What is the Jacobian Matrix?

Answer 12

The Jacobian is all the first-order partial derivatives of some multivariable function.

Intuition: Shows how a multivariable function transforms space around a specific point as a linear transformation.

Question 13

When is matrix multiplication possible? What is the size of the matrix after matrix multiplication?

Answer 13

When the # of cols in matrix 1 = the # of rows in matrix 2.

The resulting matrix dimension is the # of rows in matrix 1 x the # of cols in matrix 2.

Question 14

What is rank? What is null Space?

Answer 14

Rank is the the number of dimensions remaining after a transformation is applied
Full rank = All dimensions kept

Null space is all the vectors that are squished into the origin after the matrix transformation
For full rank matrices, the origin is the only vector in null space

Question 15

What is SVD? When can we do eigendecomposition?

Answer 15

SVD (Singular value decomposition):
Any matrix can be composed as 3 matrices A = UεV^T
U = rotation/reflection
ε = scale along axes
V = rotation/reflection

U, V = orthogonal,
ε = Diagonal non-zero elements (called singular values) are the absolute values of original matrix eigenvalues

We can do Eigen decomposition when the matrix is square. This gives us the eigenvalues and the eigenvectors. This is useful for PCA and image compression algorithms.

Question 16

What is the L1 Norm, L2 Norm, LP norm and L-infinity norm?

Answer 16

L1 Norm:
Sum of the magnitude (absolute value) of the scalars in the vector
ex = [2,3,-5] = abs(2)+abs(3)+abs(-5) = 10

L2 Norm:
square root of the sum of the squared values in the vector
ex = [2,3,-5] = sqrt(4+9+25) = 6.16

LP Norm (same as L2 norm but Nth root and power):
ex 3-norm([2,3,-5]) = cuberoot(8+27+125) = 3.36

L-infinite Norm:
Max absolute scalar value in the vector
ex infinity-norm([2,3,-5]) = 5

Norms give you a measure of vector length.

Question 17

Explain positive definite, positive semi-definite, negative definite, negative semi-definite and indefinite matrices. How are these related to ML?

Answer 17

Useful for optimization on the loss function.

PD:
+ only eigenvalues
Hessian of function is strictly increasing everywhere

PSD:
+ or 0 eigenvalues
Hessian of function is increasing everywhere

ND:
- only eigenvalues
Hessian of function is strictly decreasing everywhere

NSD:
0 or - eigenvalues
Hessian of function is decreasing everywhere

Indefinite:
+,-,0 eigenvalues
Increasing, decreasing, or saddle point everywhere

Question 18

What is the integral of ln(x)

Answer 18

Integrate by parts
f(x) = ln(x)
g(x) = x

= f*g - integral(derivative(f) * g)
= xln(x) - x

Question 19

What is the Hessian Matrix?

Answer 19

A NxN matrix containing all the second order partial derivatives of a multivariable function where n is the number of variables. The hessian matrix tells us the definiteness of a matrix based on its non-zero, positive, and negative eigenvalues.

ie. positive definite -> global minimum
ie. negative definite -> global maximum
ie. indefinite -> saddle point

Question 20

Difference between dot product and cosine similarity

Answer 20

Dot product:
The projection of one vector onto another

Cosine Similarity:
The projection of one vector onto another scaled by magnitude

Question 21

What is the Markov Property? What is a finite state machine/markov chain?

Answer 21

Markhov Assumption:
Future actions are not dependant on previous steps.

FSM/Markhov chain
A FSM is a model of computation. This can be use to show the steps that a system can make. A Markov chain is an FSM, where all possible paths are linked with a probability.