Math Flashcards
What are the derivatives of the following?
a. x^5
b. 2x^(-10)
c. log10(a)
d. 7
a = 5x^4
b = -20x^(-11)
c = 1 / (a*ln(10))
d = 0
Integrate the following.
a. x^2
b. -sin(x)
c. X^(-1)
d 7x^(1/3)
a = ((x^3) / 3) + C
b = cos(x) + C
c = ln(x) + C
d = (21x^(4/3) / 4) + C
What are the derivatives of the following?
a. h(x) = f(x) * g(x)
b. h(x) = f(g(x))
c. h(x) = 6x^3 - 2x^(-2)
d. h(x) = sin(x^2)
a = f’(x)g(x) + f(x)g’(x)
b = f’(g(x))g’(x)
c = 18x^2 + 4x^-3
d = 2x*cos(x^2)
What are the derivatives of the following?
a. e^x
b. ln(x)
e. cos(x)
b. 2x^2 * 3x^3
a = e^x
b = 1/x
c = -sin(x)
d = 30x^4
When are two vectors orthogonal? When is a single matrix orthogonal?
Vector:
When their dot product is 0.
When the angle between them is 90 degrees.
Matrix:
When A^T*A = Identity matrix
When is a matrix invertible?
- When its determinant != 0
- When it is full rank
- If there exists another matrix such that AB = BA = Identity
What is an orthogonal matrix?
When all rows and columns are orthogonal vectors and
A*A^t = A^t * A = Identity
How to represent a 3D transformation?
You could use a 3x3 matrix (matrix of 3 vectors) to represent the transformation of the unit vectors in the x, y and z direction respectively. This breaks down a complex rotation into three simple rotations.
What are eigenvectors and eigenvalues?
Eigenvectors: Vectors that stay on the same span after a transformation is applied. These vectors can be scaled. The factor by which they are scaled are the eigenvalues. Essentially the axis of rotation.
What is the determinant of a matrix?
The factor by which the matrix transformation changes any area in space.
It is zero if the dimensionality is reduced after the transformation
What is the transpose of a matrix?
When you flip the rows and columns of a matrix
Gives you the reverse of the matrix transformation
What is the Jacobian Matrix?
The Jacobian is all the first-order partial derivatives of some multivariable function.
Intuition: Shows how a multivariable function transforms space around a specific point as a linear transformation.
When is matrix multiplication possible? What is the size of the matrix after matrix multiplication?
When the # of cols in matrix 1 = the # of rows in matrix 2.
The resulting matrix dimension is the # of rows in matrix 1 x the # of cols in matrix 2.
What is rank? What is null Space?
Rank is the the number of dimensions remaining after a transformation is applied
Full rank = All dimensions kept
Null space is all the vectors that are squished into the origin after the matrix transformation
For full rank matrices, the origin is the only vector in null space
What is SVD? When can we do eigendecomposition?
SVD (Singular value decomposition):
Any matrix can be composed as 3 matrices A = UεV^T
U = rotation/reflection
ε = scale along axes
V = rotation/reflection
U, V = orthogonal,
ε = Diagonal non-zero elements (called singular values) are the absolute values of original matrix eigenvalues
We can do Eigen decomposition when the matrix is square. This gives us the eigenvalues and the eigenvectors. This is useful for PCA and image compression algorithms.
What is the L1 Norm, L2 Norm, LP norm and L-infinity norm?
L1 Norm:
Sum of the magnitude (absolute value) of the scalars in the vector
ex = [2,3,-5] = abs(2)+abs(3)+abs(-5) = 10
L2 Norm:
square root of the sum of the squared values in the vector
ex = [2,3,-5] = sqrt(4+9+25) = 6.16
LP Norm (same as L2 norm but Nth root and power):
ex 3-norm([2,3,-5]) = cuberoot(8+27+125) = 3.36
L-infinite Norm:
Max absolute scalar value in the vector
ex infinity-norm([2,3,-5]) = 5
Norms give you a measure of vector length.
Explain positive definite, positive semi-definite, negative definite, negative semi-definite and indefinite matrices. How are these related to ML?
Useful for optimization on the loss function.
PD:
+ only eigenvalues
Hessian of function is strictly increasing everywhere
PSD:
+ or 0 eigenvalues
Hessian of function is increasing everywhere
ND:
- only eigenvalues
Hessian of function is strictly decreasing everywhere
NSD:
0 or - eigenvalues
Hessian of function is decreasing everywhere
Indefinite:
+,-,0 eigenvalues
Increasing, decreasing, or saddle point everywhere
What is the integral of ln(x)
Integrate by parts
f(x) = ln(x)
g(x) = x
= f*g - integral(derivative(f) * g)
= xln(x) - x
What is the Hessian Matrix?
A NxN matrix containing all the second order partial derivatives of a multivariable function where n is the number of variables. The hessian matrix tells us the definiteness of a matrix based on its non-zero, positive, and negative eigenvalues.
ie. positive definite -> global minimum
ie. negative definite -> global maximum
ie. indefinite -> saddle point
Difference between dot product and cosine similarity
Dot product:
The projection of one vector onto another
Cosine Similarity:
The projection of one vector onto another scaled by magnitude
What is the Markov Property? What is a finite state machine/markov chain?
Markhov Assumption:
Future actions are not dependant on previous steps.
FSM/Markhov chain
A FSM is a model of computation. This can be use to show the steps that a system can make. A Markov chain is an FSM, where all possible paths are linked with a probability.
