Week 2

Question 1

What are the sets used in this course?

Answer 1

R- Set of real numbers
R+ - set of positive reals including zero
Z - Set of Integers
Z+ - set of positive integers
R^d - set of d dimensional vectors.

[a,b]^d –> { x belongs to R^d such that xi belongs to [a,b] for all i belongs to 1,2,… ,d

Question 2

What is a metric space?

Answer 2

Simply, it is a set with a structure.. example, point which make up a 2 unit radius from origin.

In this course, we will use R^d space with distance metric D(x,y) = llx-yll = sqrt ((x1-y1)^2 +(x2-y2)^2+…. +(xd-yd)^2)) ; aka euclidean distance.

Question 3

What is an open ball B(x,epsilon) mean?

Answer 3

open ball B(x,Epsilon) = { y belongs to R^d such that D(x.y)<epsilon }

simply, with x as center.. all points whose distance less than epsilon from x.. naturally forms a circle in two dimensions and a Sphere in three dimensions.

Question 4

What is an closed ball B-bar (x, epsilon) mean?

Answer 4

B-bar (x, Epsilon) = { y belongs to R^d such that D(x,y) less than or equal to Epsilon.

Question 5

What are de morgan’s laws in sets?

Answer 5

complementation (Union(A,B) = Complementation(A) Intersection Complementation (B)

Complementation(Intersection(A,B))= Complementation(A) Union Complementation (B)

Question 6

What are the common logic modifiers?

Answer 6

1. For all
2. There exists
3. Implies
4. Equivalent

Question 7

What is a sequence?

Answer 7

Ordered collection of items

Question 8

What is convergent sequence?

Answer 8

Means as limit i-> infinity , xi tends to x* if for all epsilon>0, there exists N such that xn belongs to B(x*, epsilon) for all n greater than or equal to N

Simply put, beyond a particular n, whatever the ball radius is (expect radius being zero), the element of collection will fall under this ball.. or converge into this ball.(and never comes out of the ball after entering it)

Question 9

What is a divergent sequence?

Answer 9

Not a convergent sequence

Question 10

What is norm of a vector?

Answer 10

The norm of a vector provides a way to quantify the magnitude or size of the vector in a mathematical sense. Euclidean norm is sqrt(sum of squares of coordinates)

Question 11

What are vector spaces?

Answer 11

Its also similar to metric space.
Vector space is collection of vectors which satisfies the following properties.
1. uEV , vEV , alpha,Beta belongs to R then Alpha u + Beta v belongs to V.

R^d is a vector space.

Question 12

What is a dot product?

Answer 12

if x and y are two vectors, then
x.y = x^Ty= Summation (i=1 to d) xi.yi

norm(x) =llxll = x.x = x^T.x = Summation (i =1 to d) xi^2

Question 13

How do we say if two vectors x, y are perpendicular / orthogonal?

Answer 13

x. y=0 (i.e., dot product of x and y is zero)

Question 14

What is a d-dimensional function?

Answer 14

f: R^d –> R

Question 15

What are contour plots? contour maps? Heat maps?

Question 16

What is continuity of functions?

Answer 16

lim (i -> infinity) xi = x* implies lim (i-> infinity) f(xi)=f(x)
or compactly
lim (x -> x) f(x) =f(x*)

Continuity should work on any or all sequences.. this will loosely translate that the limit of the function should exist from both the directions. If one sequence obeys and another sequence violates this rule.. the function is still discontinuous. E.g. f(x) = 1 for x>0 , 0 for x=0, -1 for x<1. In this function, x is discontinuous at x=0 since the sequence could converge on 0 but f(x) will only converge to 1 or -1 and NOT 0 hence, the function is not continuous.

A function is continuous if all points in the function are continuous.

Question 17

How to test for continuity of functions?

Answer 17

1. f(a) is defined
2. lim x tends to a f(x) exists and left side limit equals right side limit
3. lim x tends to a f(x) = f(a)

Question 18

What are the types of continuity?

Answer 18

1. continuous
2. Point discontinuity (holes) - removable
3. Jump discontinuity (jumps)- non- removable
4. Infinite discontinuity (asymptotes) - non-removable

Question 19

Continuity of linear functions?

Answer 19

All linear functions of type ax+b are continuous

Question 20

What is a differentiability of Functions?

Answer 20

A function f is differentiable at x* belongs to R if Lim (x->x) (f(x)-f(x)) / (x-x*)

Question 21

What is the link between continuity and differentiability?

Answer 21

If f is not continuous at x* then f is also not differentiable at x* . But the contrary is not true, A function could be continuous but still not differentiable at x. Eg : f(x)=lxl is not differentiable at x=0.

If a function is not differentiable at a point, then it may or may not be continuous at that point.

Question 22

What is a derivative of a function at x?

Answer 22

Derivative is the slope of the function at that point x which closely matches the function and could be used as an Linear approximation for the function ( at very close interval only)

Question 23

What is the linear approximation expression?

Answer 23

Linear approximation, also known as tangent line approximation or linearization, is a method used in calculus to estimate the value of a function near a particular point by using the equation of the tangent line at that point.

f’(x) = f(x)) - f(x) / (x-x*)

f(x) = f(x) + f’(x) (x-x*)

f(x) ~ Lx f

Question 24

What are the steps involved in linear approximation?

Answer 24

1. Find the Equation of the Tangent Line:
2. Approximate f(x) near x=a:

Question 25

How to find equation of a tangent line to a function?

Answer 25

Calculate the slope f ′ (a) of the function f(x) at x=a (the derivative of f(x) at x=a).

Use the point-slope form of the equation of a line:
y−f(a)=f ′ (a)(x−a)

Simplify to obtain the equation of the tangent line:

y=f(a)+f′(a)(x−a)

Question 26

Are higher order approximations more accurate than linear approximations?

Answer 26

Yes
quadratic approximation : f(x)~ f(x)+f’(x)(x-x) + 1/2 f’‘(x) (x-x*)^2.

there are more high approximations available.. third order .. fourth order.. etc.. but for most of this course we use linear approximation.. in special cases, we may need quadratic approximations..

Question 27

What is a tangent?

Answer 27

If one set touched(at most at one place ) another set then it is said to be a tangent.

A tangent line touches a function at atmost one point.

Question 28

What is product rule?

Answer 28

If f(x) = g(x). h(x). What is f’(x)?

f’(x) = g’(x) h(x) + h’(x) g(x)

Question 29

What is a chain rule?

Answer 29

If f(x) - g(h(x)), what is f’(x)?

f’(x) = g’( h(x)) h’(x)

Question 30

What are multivariate calculus?

Answer 30

f: R^d —> R

Question 31

What are some of the points in Geometry of lines?

Answer 31

1. A line in R^d is subset of R^d
2. A line through the point U belongs to R^d along the vector v belongs to R^d = { x belongs to R^d such that u+alpha.v for alpha belongs to R)
3. Line through u, u’ belongs to R^d= { x belongs to R^d such that x=u+alpha (u’ -u) for alpha belongs to R}
   ={ x belongs to R^d: x=(1-apha)u + alpha.u’ for alpha belongs to R^d}

Note: line through u along u’-u is same as line though u’ along u-u’

Question 32

What is a hyper plane?

Answer 32

A (d-1) dimensional hyperplane is subset of R^d. A hyperplane normal to the vector w belongs to R^d with value b belongs to R = {x belongs R^d such that wTx=b}
={x belongs to R^d such that summation (i=1 to d) wi. xi.=b }

Question 33

What are tuples vs Points vs Vectors?

Answer 33

Both points and vectors are represented as tuples in R^d, and it will only be in context we could know which is which

Question 34

What are partial derivatives?

Answer 34

finding derivative of a single variable, keeping all other variables constant. (simply.. finding the derivative or slope of a function in a particular direction)

Question 35

What is a gradient?

Answer 35

Gradients in s vector containing all the partial derivatives)

Gradient = column vector (all partial derivatives)

Question 36

How is gradient found?

Answer 36

Find the Gradient: Compute ∇f(a), which is a vector of partial derivatives of f with respect to each variable evaluated at a.

Question 37

what is the relationship between gradient and linearity?

Answer 37

if gradient vector is constant, then the underlying function is linear.

Question 38

gradients and linear approximations of vectors?

Answer 38

Linear approximation of a multivariate function extends the concept of tangent line approximation from single-variable calculus to functions of multiple variables. It allows us to approximate a complex multivariate function f(x) near a point
a using a linear function.

Given a multivariate function f(x) and a point a in the domain of f:

Tangent Plane: Analogous to the tangent line in single-variable calculus, the tangent plane at a is the best linear approximation of f(x) near a.

Linear Approximation: The linear approximation of f(x) near a is given by:
L(x)=f(a)+∇f(a)⊤(x−a)
where:

∇f(a) is the gradient of f evaluated at a.
(x−a) represents the vector difference between x and a.
∇f(a)⊤ denotes the transpose of the gradient vector.
v belongs to Rd and x belongs to Rd
f(x)~ f(v) + grad(f(v) Transpose . (x-v)

Question 39

What are tangent planes?

Answer 39

The graph of linear approximations is a plane that is tangent to the graph of f at the point (v, f(v))

Question 40

gradient and contour interpretation?

Answer 40

The gradient of a function v is going to be perpendicular to the contour passing through that point.

Question 41

What are directional derivative?

Answer 41

Directional derivate of function f at point v along vector u is

lim (alpha tends to zero) (f(v+alpha.u) - f(v)) / alpha

Question 42

what is Cauchy - Schwarz Inequality?

Answer 42

For any vectors u and v in an inner product space (such as Rn or Cn), the Cauchy-Schwarz inequality states that:
∣⟨u,v⟩∣ ^2 ≤⟨u,u⟩⋅⟨v,v⟩

Where:
⟨u,v⟩ denotes the inner product (or dot product) of u and v.
⟨u,u⟩ and ⟨v,v⟩ denote the inner products of u with itself and v with itself, respectively.

The Cauchy-Schwarz inequality essentially states that the square of the absolute value of the inner product of two vectors is less than or equal to the product of the norms (or lengths) of the two vectors. In other words, the inner product of two vectors is maximized when the vectors are linearly dependent.

Question 43

What are directional derivatives?

Answer 43

The directional derivative of f at p in the direction of v measures the rate of change of f at p when moving in the direction specified by v.

Question 44

How to find directional derivatives?

Answer 44

The directional derivative D1v f(p) is given by:

Dv f(p)=∇f(p)⋅v
where:
∇f(p) is the gradient of f at p.
v is the unit vector specifying the direction.

Question 45

What is the direction of the steepest Ascent?

Answer 45

gradient multiplied by norm of gradient at a point p is the direction of steepest ascent

Question 46

How to see if equation of a line is passing through a point p along the direction of the vector v?

Answer 46

how to check if a equation of the line is passing through [7,8,6] in the direction of vector [1,2,3]?

Equation of line [1,2,3]+α[6,6,3]

proof:
equate equation of line to point , 1+6α=7 , 2+6α=8, 3+3α=6 , if the parameter α is any value other than zero then the equation satisfies the condition that it passes through the point p(7,8,6) in the direction of vector (1,2,3)