Splines Flashcards
y
dependent variable
t
explanatory variable (1-dimensional)
Underlying Function Assumption
-have yi observations at each ti for i=1,..,n where observation times ti are ordered : t1 < t2 < … < tn
-assume that observations yi are noisy versions of an underlying smooth function:
yi = f(ti) + εi
f
- we want to estimate f
- can do this using:
- -parametric models e.g. f(t) = α + βt
- -non-parametric models i.e. f is a smooth function of t
Basic Spline Theory
Word Definition
- let t1 < t2 < … < tn be a fixed set of ‘knots’
- a spline of order p≥1 is a piecewise polynomial of order p that is (p-1) times differentiable at the knots
Basic Spline Theory
Equation Definition
-let t1 < t2 < … < tn be a fixed set of ‘knots’
f(t) = Σ aij t^j
-for ti ≤ t < ti+1, i=0,…,n and j=0,…,p
-sum from j=0 to j=p
-where to=-∞ and tn+1=+∞
Smoothness Constraint
f^(l) (ti-) = f^(l)(ti+)
- for l=0,…,p-1 and i=1,…,n
- i.e. the limit of the lth differential of f(ti) as t approaches ti from below equals the same differential as t approaches ti from above
Linear Spline
-consider the sequence t1 < … < tn
-define gi(t) = |t-ti|, i.e. we shift g to be centred at each knot
-then:
f(t) = a0 + a1*t + Σ bi gi(t)
-sum from i=1 to i=n is a linear spline
-at each knot ti, the function gi(t) is continuous
Basis for the Vector Space of Linear Splines
B = {1, t, g1(t), …, gn(t)} forms a basis for the vector space of linear splines
Natural Spline
Definition
-natural splines are a special case of polynomial splines of odd order
-a spline is said to be natural if beyond the boundary knots t1 & tn its (p+1)/2 higher-order derivatives are zero, i.e.:
f^(j)(t) = 0
for j=(p+1)/2, …, p AND t≤t1 or t≥tn
How many constraints does an order-p natural spline have?
-an order-p natural spline has p+1 constraints
f^(j)(t1-) = f^(j)(tn+) = 0
-for (p+1)/2, …, p
Natural Linear Splines
Constraints
-a natural Iinear spline has p+1=2 extra constraints:
f’(t1-) = f’(tn+) = 0
-so f(t) is constant in the outer intervals
Natural Cubic Splines
Constraints
-a natural cubic spline has p+1=4 extra constraints:
f’‘(t1-) = f’‘(tn+) = 0
f’’‘(t1-) = f’’‘(tn+) = 0
=> f(t) is linear in the outer intervals
Total Degrees of Freedom of a Natural Spline
n + p + 1 - (p + 1) = n
-dimensional space of natural splines = n regardless of p
Linear Natural Splines
Representation
f(t) = ao + Σ bi |t-ti|
- with Σbi=0
- sum form i=1 to i=n
Cubic Natural Splines
Representation
f(t) = a0 + a1*t + Σ bi |t-ti|³
- with Σ bi = Σ bi*t = 0
- sum from i=1 to i=n
Roughness
Definition
J^(ν)(f) = ∫ [f^(ν)(t)]² dt , ν≥1
-integrate from -∞ to +∞
Roughness of a Natural Linear Spline
Direct Calculation Formula
J^(1)(f) = -2 Σ Σ bi bk |ti-tk|
-sum from i=1 to i=n & k=1 to k=n
Roughness of a Natural Linear Spline
Matrix Form
J^(1)(f) = c^(1) b^T K^(1) b
- where c^(1) = -2
- and b is the nx1 vector of spline coefficients b1,…,bn
- and K^(1) is the nxn matrix whose (i,k)th element is |ti-tk|^p
- where p=2ν-1
Roughness of a Natural Cubic Spline
Direct Calculation Formula
J^(1)(f) = 12 Σ Σ bi bk |ti-tk|³
-sum from i=1 to i=n & k=1 to k=n
Roughness of a Natural Cubic Spline
Matrix Form
J^(2)(f) = c^(2) b^T K^(2) b
- where c^(1) = 12
- and b is the nx1 vector of spline coefficients b1,…,bn
- and K^(1) is the nxn matrix whose (i,k)th element is |ti-tk|^p
- where p=2ν-1
The Interpolation Problem and the Optimal Interpolating Function
-for fixed integer ν≥1 representing an order of smoothness, the objective is to find the smoothest function which passes through a set of data points
-i.e. we want to find f to mimimise:
J^(ν)(f) = ∫ [f^(ν)(t)]² dt
-subject to f(ti)=yi for i=1,…,n
-the solution f is the optimal interpolating function
