Splines

Question 1

y

Answer 1

dependent variable

Question 2

t

Answer 2

explanatory variable (1-dimensional)

Question 3

Underlying Function Assumption

Answer 3

-have yi observations at each ti for i=1,..,n where observation times ti are ordered : t1 < t2 < … < tn
-assume that observations yi are noisy versions of an underlying smooth function:
yi = f(ti) + εi

Question 4

f

Answer 4

• we want to estimate f
• can do this using:
• -parametric models e.g. f(t) = α + βt
• -non-parametric models i.e. f is a smooth function of t

Question 5

Basic Spline Theory

Word Definition

Answer 5

• let t1 < t2 < … < tn be a fixed set of ‘knots’

- a spline of order p≥1 is a piecewise polynomial of order p that is (p-1) times differentiable at the knots

Question 6

Basic Spline Theory

Equation Definition

Answer 6

-let t1 < t2 < … < tn be a fixed set of ‘knots’
f(t) = Σ aij t^j
-for ti ≤ t < ti+1, i=0,…,n and j=0,…,p
-sum from j=0 to j=p
-where to=-∞ and tn+1=+∞

Question 7

Smoothness Constraint

Answer 7

f^(l) (ti-) = f^(l)(ti+)

• for l=0,…,p-1 and i=1,…,n
• i.e. the limit of the lth differential of f(ti) as t approaches ti from below equals the same differential as t approaches ti from above

Question 8

Linear Spline

Answer 8

-consider the sequence t1 < … < tn
-define gi(t) = |t-ti|, i.e. we shift g to be centred at each knot
-then:
f(t) = a0 + a1*t + Σ bi gi(t)
-sum from i=1 to i=n is a linear spline
-at each knot ti, the function gi(t) is continuous

Question 9

Basis for the Vector Space of Linear Splines

Answer 9

B = {1, t, g1(t), …, gn(t)} forms a basis for the vector space of linear splines

Question 10

Natural Spline

Definition

Answer 10

-natural splines are a special case of polynomial splines of odd order
-a spline is said to be natural if beyond the boundary knots t1 & tn its (p+1)/2 higher-order derivatives are zero, i.e.:
f^(j)(t) = 0
for j=(p+1)/2, …, p AND t≤t1 or t≥tn

Question 11

How many constraints does an order-p natural spline have?

Answer 11

-an order-p natural spline has p+1 constraints
f^(j)(t1-) = f^(j)(tn+) = 0
-for (p+1)/2, …, p

Question 12

Natural Linear Splines

Constraints

Answer 12

-a natural Iinear spline has p+1=2 extra constraints:
f’(t1-) = f’(tn+) = 0
-so f(t) is constant in the outer intervals

Question 13

Natural Cubic Splines

Constraints

Answer 13

-a natural cubic spline has p+1=4 extra constraints:
f’‘(t1-) = f’‘(tn+) = 0
f’’‘(t1-) = f’’‘(tn+) = 0
=> f(t) is linear in the outer intervals

Question 14

Total Degrees of Freedom of a Natural Spline

Answer 14

n + p + 1 - (p + 1) = n

-dimensional space of natural splines = n regardless of p

Question 15

Linear Natural Splines

Representation

Answer 15

f(t) = ao + Σ bi |t-ti|

• with Σbi=0
• sum form i=1 to i=n

Question 16

Cubic Natural Splines

Representation

Answer 16

f(t) = a0 + a1*t + Σ bi |t-ti|³

• with Σ bi = Σ bi*t = 0
• sum from i=1 to i=n

Question 17

Roughness

Definition

Answer 17

J^(ν)(f) = ∫ [f^(ν)(t)]² dt , ν≥1

-integrate from -∞ to +∞

Question 18

Roughness of a Natural Linear Spline

Direct Calculation Formula

Answer 18

J^(1)(f) = -2 Σ Σ bi bk |ti-tk|

-sum from i=1 to i=n & k=1 to k=n

Question 19

Roughness of a Natural Linear Spline

Matrix Form

Answer 19

J^(1)(f) = c^(1) b^T K^(1) b

• where c^(1) = -2
• and b is the nx1 vector of spline coefficients b1,…,bn
• and K^(1) is the nxn matrix whose (i,k)th element is |ti-tk|^p
• where p=2ν-1

Question 20

Roughness of a Natural Cubic Spline

Direct Calculation Formula

Answer 20

J^(1)(f) = 12 Σ Σ bi bk |ti-tk|³

-sum from i=1 to i=n & k=1 to k=n

Question 21

Roughness of a Natural Cubic Spline

Matrix Form

Answer 21

J^(2)(f) = c^(2) b^T K^(2) b

• where c^(1) = 12
• and b is the nx1 vector of spline coefficients b1,…,bn
• and K^(1) is the nxn matrix whose (i,k)th element is |ti-tk|^p
• where p=2ν-1

Question 22

The Interpolation Problem and the Optimal Interpolating Function

Answer 22

-for fixed integer ν≥1 representing an order of smoothness, the objective is to find the smoothest function which passes through a set of data points
-i.e. we want to find f to mimimise:
J^(ν)(f) = ∫ [f^(ν)(t)]² dt
-subject to f(ti)=yi for i=1,…,n
-the solution f is the optimal interpolating function