Lecture 3 - Systems of linear equation continue

Question 1

Differences and basics

Answer 1

Direct algorithms -> Q(n^3)
Iterative methods -> Q(n^2) assuming they CONVERGE

The solution by means of Iterative algorithms starts with the guess initial vector x1, which approximates exact solution x

Question 2

Errors

Answer 2

x - solution
x’ - approximate solution
e - solution error

In practical cases it’s NOT possible to compute the real error e, since we DON’T know the true solution vector x.

In order to asses the error introduced by x’, we can use a residual vector r = A*x’ - b

if x’ = x, then r = 0
BUT small value of r DOESN’T ALWAYS guarantee a small real error
Sometimes it’s more comfortable to use scalar error-measure.

Question 3

Vector norms

Answer 3

Norm - function that assigns a real-valued length to each vector

CONDITIONS:

1. Norm of a nonzero vector is positive
2. Scaling a vector scales its norm by the same amount
3. The norm of a vector sum doesn’t exceed the sum of norms of its parts (triangle inequality)

Unit balls

Equation for n norm:
(x and y are random)
((abs(x))^n +(abs(y))^n)^(1/n)

Question 4

FIRST NORM/ norm one

Answer 4

Sum of absolute values of all elements of the vector.

Unit ball graphic look like diamond

Question 5

SECOND NORM/ norm two

Answer 5

WE compute the square of all elements of the vector, and we take square root of their sum.
Unit ball graphic look like circle

Question 6

INFINITY norm

Answer 6

Maximum value from absolute values from the elements of the vector

Unit balls look like square

Question 7

Jacobi

Answer 7

Vector from the previous iteration is used to compute the elements of the vector in subsequent iteration. Xk is used to compute Xk+1
So, in the fist iteration we are using only elements from first (quested one) vector

Converges if:

• > spectral radius of the iteration matrix is smaller than 1
• > A is diagonally dominant

Question 8

Gauss-Seidel

Answer 8

Works the same as Jacobi, but always the most recent information are taken. So, when we are computing element 6 in k+1 iteration, we are using elements 0-5 from k+1 vector, and 7 to n form kth vector

Converges if either:

• > A is symmetric positive-define
• > a is diagonally dominant

Question 9

Division of the matrix A

Answer 9

A = -(U+L)+D
U - upper part
L - lower part 
D - diagonal
L and U are NOT the same as in the LU factorization

Question 10

Jacobi in matrix form

Answer 10

xk+1 = D^(-1)(L+U)xk + D^(-1)*b

Question 11

Gauss-Seidel in matrix form

Answer 11

xk+1 = (D-L)^(-1)Uxk + (D-L)^(-1)*b

Question 12

Remarks

Answer 12

• > The convergence properties of the those methods are dependent on the matrix A
• > Usually the Gauss-Seidel method need SMALLER number of iterations to converge

Question 13

Matrix norm

Answer 13

Must satisfy the same 3 conditions as vector norm

e.g.: Frobenius norm (It’s the same as the vector 2-norm when viewed as a vector composed as the A columns