Linear Algebra with NumPy Primer Flashcards
What does this mean?
the set of all real-valued matrices with n rows and d columns
What is this called? What does it mean?
1(x)
- binary indicator function,
- returns 1 if x is a true boolean and 0 otherwise
What does this mean?
the set of all real-valued n-dimensional vectors
What does this mean?
the set of real numbers
What is this?
- A pair of input vector and output scalar
- The superscript indicates the index of this pair, not the exponent. In other words, x(5) is the input vector at index 5, not x raised to the power of 5.
What is this?
|C|
- Determinant if C is a matrix
- Cardinality (number of member elements) if C is a set.
What’s a column vector?
Each entry occupies one row.
What’s this?
One use: denotes the i-th entry of a column vector
What does this mean?
Row vector. Each entry covers 1 column
For matrix A, what is
denotes the i-th row of A
For matrix A, what is
denotes the i-th column of A
What can dot product be applied to? What’s the formula?
Two vectors of same dimensions
What’s the inner product?
Same as dot product
- What can outer product be applied to?
- What does it result in?
- What’s the formula?
- Vectors (can have different number of elements)
What’s another way to express a dot product for two column vectors x and y?
What’s this for two vectors x and y?
The outer product
What’s a matrix-vector product?
- What are the properties of matrix multiplication? (2)
- What properties does it not have? (1)
Matrix multiplication is:
- associative, (AB)C=A(BC)
- distributive, A(B+C)=AB+AC
Is not:
- commutative, AB≠BA
What’s a property of transposes?
What’s the Hadamard product?
elementwise matrix multiplication
What does this mean?
A standard basis vector - ith entry is 1 and the other entries are all 0
What does this denote?
Identity matrix of nxn size
What’s the point of a basis vector?
By multiplying a basis vector with a vector x∈Rn, we obtain the single entry xi
Describe the idea of one-hot encoding
- In each row of the resulting one-hot-encoded matrix:
- In each row of the original vector, find the value
- This value will serve as the column index of the “1” entry in the same row in the corresponding matrix.
- The remaining values in that row are 0
What’s a symmetric matrix?
What’s this? What can it be applied to?
- Inverse matrix
- Only applied to square matrices
What’s the fundamental rule of matrix inverses?
What are the properties of matrix inverses?
What’s a diagonal matrix?
a rectangular matrix with non-zero entries only on the main diagonal
What’s this?
- The p-norm (a vector norm).
- It’s indicative of its magnitude or length
What’s the formula of this?
What’s this called? (2)
- L2 norm (aka Euclidean norm):
What’s this?
Gives the number of non-zero entries in the vector
What’s this?
gives the maximum magnitude across the entries in vector x
What’s this? What can it be applied to?
trace(A)
- Sum of the elements across the main diagonal
- Only applied to square matrices
- What’s this?
- What can it be applied to?
- quadratic form
- See screenshot for applied to
What’s vectorization?
a style of programming where operations are applied to whole arrays instead of individual elements (i.e. operate on entire data structure instead of loops)
When should we use vectorization?
Whenever possible
Why are python loops slower than C or Java loops?
- Python performs variable type checks at each loop iteration
- What’s ndarray?
- What’s a characteristic of its data structure?
- Data structure with an interface similar to python lists, but its operations are performed in optimized C code
- A homogeneous data structure
What’s the runtime complexity of matrix multiplication using the original definition of MM? (optional)
O(n^3)
Description:
Here each entry in AB is the result of the dot product between a row in A and a column in B, which are both n-dimensional vectors. Therefore, it takes roughly n computations to yield one entry in AB, and because there are n2 entries, a total of n3 computations are required.
More formally, we say that the runtime complexity of matrix multiplication is O(n3). This notation indicates the growth rate of matrix multiplication – if we increase the size of A and B by a factor of 10, we will need (10n)3=1000n3 computations
What do we know about the matrix multiplication algorithms of data science libraries?
They use algorithms that are more efficient than the textbook definition of matrix multiplication
- What’s a condition for vectorization?
- What do we have to do if it’s not met?
- What are 3 scenarios that pose this problem?
- the elementwise operations are independent of each other
- If not met, need to revert to using loops
- Problem scenarios (examples in screenshot)
- Loop dependency
- Indirect memory access
- Code branching
How do you know if a function is vectorizable?
It doesn’t belong to the 3 problem scenarios of the independence condition
How do you vectorize a function using numpy?
“register” it through the np.vectorize function.
How does the performance of a custom vectorized function compare to alternatives?
Not as fast as functions of same functionality from DS libraries like numpy because they’re not executed in optimized C code
How do you vectorize an interative function? (4)
- Given an iterative formula, organize its expected outputs in a suitable vector or matrix format.
- Break down this vector or matrix into smaller components.
- Vectorize each component by using matrix and vector operations. Honorable mentions include one-hot encoding, quadratic form, XXT along with XTX, and elementwise operators.
- Identify the appropriate NumPy functions to convert the vectorized expression into code.
With two arrays x and y of the same size, how do you make a new array with some elements from x and some from y based on a condition?
numpy.where(condition, x, y)
where x and y are arrays
Returns an array with elements from x where the condition is True, and elements from y elsewhere.
What are honorable mentions changes for when you can’t fully vectorize a component? (4)
- one-hot encoding
- quadratic form
- XXT along with XTX
- elementwise operators
What should you be thinking when you’re trying to vectorized a function and you see this?
This is the same as a one-hot encoding then summing across the rows
What’s a property of the binary indicator function?
𝟙(𝑎 & 𝑏)=𝟙(𝑎)×𝟙(𝑏).
What should you think when you see a vector of dot products?
it very likely comes from a matrix-vector multiplication.
