Splines

Question 1

Spline function definition

Answer 1

[a,b] -> R with m knots (m-2 internal) of degree l if:
- (l-1) order continuously differentiable;
- Within each m-1 interval, different polynomial of degree l.

Question 2

Number of free parameters in splines

Answer 2

d = l + m - 1

Question 3

Truncated power basis

Answer 3

Bj(z) =
- z^j j=1,…l+1
- (z-k_j-1)^l+ j=l+2,…d

Question 4

B-splines definition

Answer 4

Defined recursively
- Values > 0 only between a pair of knots
- Σ Bj(z) = 1 ∀ z
- ∀ z ∈ [a,b] only (l+1) bases are > 0.

Question 5

MLE for B-splines

Answer 5

β^ = (Z’Z)^-1Z’y

Question 6

Model selection for splines

Answer 6

LRT only if using truncated power basis for add/removal of knots.
All other cases (B-splines, movement of knots), model selection criteria.

Question 7

P-splines definition

Answer 7

1. We start with a high number of m-2 internal knots;
2. We remove some using the penalized log-likelihood pl(β,σ²) = l(β,σ²) - λ/2 J_i(β) = -n/2 ln(σ²) - n/(2σ²) (y-Zβ)’(y-Zβ) - λ/2 β‘K_iβ.
   If λ=0 we have B-splines with no penalization.
   If λ -> + &infty; we tend to have only constant functions (very smooth ones).

Question 8

Order differences

Answer 8

For B-splines, we can use:
J₁(β) = Σ_j=2^d (β_j - β_j-1)² = βK₁β’
J₂(β) = Σ_j=3^d (β_j - 2β_j-1 + β_j-2)² = βK₂β’

K_{i, dXd} matrix made of differential matrices D^(d)_{1, (d-1)Xd} and D^(d-1)_{1, (d-2)Xd}

Question 9

MLE for P-splines

Answer 9

β^ = (Z’Z - ΛK_i)^-1Z’y
- Λ = λσ²
Equivalent to minimizing the penalized least squares: PLS = (y-Zβ)’(y-Zβ) - Λβ‘K_iβ

Question 10

Fitted values P-splines

Answer 10

f^ = y - Zβ^ = y - Z(Z’Z-ΛK)^-1Z’y = [I - Z(Z’Z-ΛK)^-1Z’]y = [I - S]y
S: Smoother matrix

Question 11

Effective number of parameters P-splines

Answer 11

df(S) = trace(S)
{ edf in output }

Question 12

Estimator for σ² P-splines

Answer 12

[Σ(y_i - f^(z_i))²]/[n - df(S)]
If ML (for AIC and BIC): 1/n * Σ(y_i - f^(z_i))²]

Question 13

Choice of best Λ

Answer 13

• CV = 1/n Σ [(y_i - f^(z_i))/(1 - S_ii)]²
• GCV = 1/n Σ [(y_i - f^(z_i))/(1 - df(S)/n)]²
• AIC or BIC, remembering σ²_ML^ and number of parameters = df(S)+1

Question 14

Inference basis for P-splines

Answer 14

Y|Z ∼ N_n (f, σ²I_n) that leads to:
f^|Z ∼ N_n (Sf, σ²SS)
- P-splines are biased estimators;
- Produced p-values for hypothesis testing are anticonservative: they underestimate the true values.

Question 15

B-splines in R

Answer 15

library(splines)
internal = m-2
location = quantile(z, probs=(1:internal)/(internal+1))
bsp = bs(z, knots=location, degree=l, intercept=F)
#Equidistant knots: bsp = bs(z, df=d, degree=l)
fit = lm(y~bsp) #Or -1 if intercept=T

Question 16

P-splines in R

Answer 16

library(mgcv)
smoothers = s(z, bs=”ps”, k=m, m=c((l-1), i), sp=lambda)
fit = gam(y~smoothers)
#edf = df(S) effective number of parameters
#edf+1 total coefficients