Lecture 2 - System of linear equations

Question 1

Linear equation

Answer 1

Involves x with exponent n=1

Question 2

Algebraic equation

Answer 2

W(x)=0

Polynimials

Question 3

Homogeneous equation

Answer 3

Polynomial without free coefficient, a0 =0

Question 4

Possible solutions

Answer 4

ONE SOLUTION
System is consistent and independent

INFINITE SOLUTIONS
System is consistent and dependent

NO SOLUTION
System is inconsistent

Question 5

Matrix equation

Answer 5

We can change system of linear equation into matrix equation Ax = b, where
-> A - system matrix, left hand side of equations, coefficients
-> x - vector of unknowns
-> b - vector of right hand side, free coefficients
b=0 system is Homogeneous
else system is in-homogeneous

Question 6

Kronecker-Capelli theorem

Answer 6

A is a matrix, n - number of variables, [A b] - augmented matrix, r() - rank

• > one solution <=> r(A) = r([A b]) = n
• > infinite solutions <=> r(A) = r([A b]) < n
• > no solution <=> r(A) != r([A b])

elementary row operation do not change the rank ( number of the linear independent columns of a matrix)

Question 7

Solution methods

Answer 7

• > DIRECT - allows one to obtain the solution after certain numbers of operations e.g. Cramer, Gauss
• > ITERATIVE - generate the sequence of vector solutions, which converges to the real solution of the system e.g Jacobi, Gauss-Seidel

Question 8

Computational complexity

Answer 8

• > basic arithmetic operations are called floating-point operations
• > complexity of an algorithm is the total number of floating point operations needed, as function of the input dimension
• > this value is often approximated in practice Q()
• > e.g linear complexity (x+y) needs n operation - Q(n)

Question 9

Gauss Elimination (Direct method)

Answer 9

Two steps:

1. Transformation of the system matrix to the triangular form - elementary row operations
2. Forward (backward) substitution
   - > computational complexity Q(n^3)

Question 10

LU Factorization

Answer 10

L - lower matrix
U - upper matrix
variant of Gauss elimination
Digression - symbolic factorization, numerical factorization, solution
A = L*U
Process:
1. Generate L and U, so we have LUx = b
2. Define the auxiliary vector y = Ux
3. Ly = b solve using the forward substitution
4. Solve Ux = y using backward substitution

Question 11

Cholesky ( Banachowicz) Factorization

Answer 11

We can use this if the matrix is symmetric and positive-define.
A = L * Lt
Half of the operation comparing to LU

Question 12

Elementary row operations

Answer 12

• > Interchanging the position of two rows
• > Multiplying a particular row by a nonzero constant
• > Replacing a particular row by that equation plus a non-zero multiple of another row

Question 13

Remarks

Answer 13

• > approximately 75% of the supercomputers computational time is use to solve systems of linear equations
• > Cramer method and explicit matrix inversion is NOT used in practice
• > Algorithms requires that pivot element CANNOT be zero. In such situation interchange of rows in necessary
• > It’s generally desirable to choose a pivot element with LARGE absolute value. It improves the numerical stability