What is calibration?
When we build a model we adjust its parameters to ensure it compares well with observed data. This is called calibration.
We calibrate the:
- model data and components
- submodels
- entire assembled model.
Some calibration can be done manually. Some is done automatically using software tools.
We determine the most important calibration measures for a model during model scoping and model planning.
Calibration is complex, and how we calibrate depends on what data or component we are calibrating and why.
When we calibrate, we often transition gradually from calibrating with historical data to calibrating using simulated system fluxes. Calibrating with historical data at the start of helps reduce uncertainty, because we know what the data parameters are.
We might use all available historical data to calibrate, or only some.
The Australian Water Modelling Practice Notes provide more detail about:
Overfitting
When we calibrate with historical data, we assume that data does not contain biases, random errors or trends that the model cannot account for. While we can remove biases, it is more difficult to detect and remove random errors or trends.
Overfitting occurs when calibration focuses too much on matching observed data, including where data may contain random errors resulting in parameters that don’t properly represent the modelled process.
An overfitted model may fail when we run it for time periods other than the period it was calibrated against.
Finalising calibration
When all calibration is complete and has been refined, we compile a calibration report.
We may give this report to a qualified and experienced independent reviewer for comment prior to releasing it, and the model, for stakeholder comment.
Maintaining the model
Our models will be used for many years, so we need to be sure they continue to provide accurate, robust data.
We may need to update our models as modelling technology advances, or when data components change.
Find out more about managing models in the Australian Water Modelling Practice Notes.
Data updates
We ensure that significant model parameters and/or data are kept up to date.
We also consider ways updating data might improve the model, and whether such improvements would be worthwhile in terms of the model’s purpose and use. For example, we might decide to:
- collect and evaluate more data
- update software algorithms or develop new software
- refine a model’s configuration, such as incorporating new submodels.