Calibration

Question 1

What is the mathematical expression for the model?

Answer 1

y = f(x ; theta)

Question 2

What is the mathematical expression for the data?

Answer 2

{x_i, yhat_i}

Question 3

What is the mathematical expression for the misfit?

Answer 3

S(theta) = sum of squares between model and data

Question 4

Explain ad-hoc calibration.

Answer 4

Manual parameter selection to obtain a good fit. Can be inefficient and we have to retrace our steps. Heavily reliant on the starting point (expert knowledge) Qualitative ‘goodness of fit’ is different for everyone.

Question 5

Explain parameter space

Answer 5

All possible parameters as an N-D space. A misfit can be calculated for each step and creates a surface which we can find the minimum.

Question 6

What is parameter sensitivity

Answer 6

The derivative of the model with respect to the parameters

Question 7

Explain the steps of gradient descent

Answer 7

Find the minimum objective function by travelling ‘downhill’.
1. Initial guess at theta
2. Compute downhill travel direction
3. Travel downhill for new estimate
4. Repeat 1-3 until little change

Question 8

What are the calibration gotchas?

Answer 8

Calibrate model with wrong physics if it has lots of parameters. If a parameter is spatially varying, a model can be well calibrated in some places and poorly in others. Multiple parameter sets can fit data equally well, sometimes required regulation. Sometimes data is only sufficient to calibrate a parameter combination (superparameter).

Question 9

What does the misfit function measure and how is it constructed?

Answer 9

It quantifies the difference between the model and the data. It is the sum of squares between the data and model evaluated at the same space/time.

Question 10

Why does changing a model parameter change the misfit?

Answer 10

Changing a model parameter changes the mode. Because the misfit is a function of the data and the model, the misfit will change.

Question 11

What are the advantages of ad-hoc calibration

Answer 11

You learn something about the model after each iteration. It is also cost-efficient.
It requires expert knowledge to determine a starting point.

Question 12

Gradient descent is a method to find better fitting models, but is it guaranteed to find the best fitting model?

Answer 12

It is possible to find a local minimum instead of global which would not be the best fitting model.

Question 13

Describe how the objective function surface and parameter space are related to each other.

Answer 13

Parameter space is all parameters as an N-D space. The objective function surface is created by calculating the objective function for each set of parameters within the N-D space and laying it over top.

Question 14

How does the concept of diminishing returns apply to model calibration?

Answer 14

With iterative processes such as gradient descent, it is worth considering a threshold of the objective function, otherwise the calibration process can continue forever with little return.

Question 15

Why do models become harder to calibrate as they have more parameters?

Answer 15

It is possible to calibrate it with the wrong physics if there are a lot of parameters. The sensitivity also increases.

Question 16

Why is model runtime a factor in calibration?

Answer 16

Resource allocation, computational complexity, diminishing returns

Question 17

Is it possible for two different models to have the same misfit? How should we decide which is better?

Answer 17

Yes, use regularisation to choose the better one. Add a penalty to the calculation of the misfit that penalises for non-smoothness