This is an old revision of the document!
Algorithm Configuration Options
The program_options configuration file currently controls the algorithm selected by the value of AlgoCode set.
On 2009/07/08 20:54 the possible values for AlgoCode are:
| AlgoCode | Description | Status |
|---|---|---|
| 1 | Redistribute accumulated precipitation. | OK |
| 2 | Generate variability parameters. | Development. |
| 3 | General interpolation in both time and space and estimate a modelled value | Development |
| 4 | Temporal interpolation for “one-point” gaps (fill in missing values). Performs the update based on min and max recorded at the end of the hour. | OK |
| 5 | Perform only an interpolation. | For testing and quality control. |
| 6 | SpaceCheck: Set a quality control flag based on a comparison with neighbours. Configure for any implemented interpolation method. | Development |
| 7 | Null call … does nothing. | For testing only. |
| other | Other values are handled gracefully and no algorithm initiated. | OK |
Temporal interpolation for "one-point" gaps
[ALGOCODE:4]
Algorithm location in kvalobs distribution: src/kvQc2/algoithms/ProcessUnitT
Currently configured for hourly temperature values:
- Locates a single missing value
- Checks if data available immediately before and after the interval
- If available generates first estimate from linear interpolation
- Checks for the Max and Min temperature for the missing value from the measurements registered in the hour afterwards
- Performs a second estimate as the average of 0.5*(MaxT + MinT)
TBD … add logic to propose best model value … some example data:
T-Corrected: The current supplied corrected value (typically a model value and/or HQC check)
Linear: Linear interpolation of the proceeding and following value.
T-Tan-Tax: The mean of the Min and Max values for the same hour (recorded in the following hour) is another possible corrected value.
T-neighbour: For reference, the value of the nearest neighbour.
| T0 | T1 | T2 | T-Corrected | Linear | T-Tan-Tax | T-neighbour | TAN | TAX | CFAILED |
|---|---|---|---|---|---|---|---|---|---|
| -0.3 | -32767 | -1.7 | -1.2 | -1 | -1.65 | 0.2322 | -2.1 | -1.2 | QC1-4-211:1,hqc |
| 0.1 | -32767 | -0.1 | -0.1 | 0 | -0.1 | -1.68 | -0.3 | 0.1 | QC1-4-211:1,hqc |
| -8.5 | -32767 | -6.1 | -7.5 | -7.3 | -6.85 | -16.0474 | -7.7 | -6 | QC1-4-211:1,hqc |
| -8.4 | -32767 | -8.6 | -8.5 | -8.5 | -8.55 | -10 | -8.6 | -8.5 | QC1-4-211:1,hqc |
| -6.4 | -32767 | -7.3 | -7 | -6.85 | -7.05 | 6.4144 | -7.3 | -6.8 | QC1-4-211:1,hqc |
| -5.9 | -32767 | -6.4 | -6.2 | -6.15 | -6.1 | 5.4016 | -6.4 | -5.8 | QC1-4-211:1,hqc |
| -16.7 | -32767 | -17.2 | -17 | -16.95 | -17.15 | -1.48 | -17.5 | -16.8 | QC1-4-211:1,hqc |
| -0.7 | -32767 | -2.7 | -1.2 | -1.7 | -1.9 | 0 | -2.7 | -1.1 | QC1-4-211:1,hqc |
| -3 | -32767 | -2.8 | -2.8 | -2.9 | -2.9 | -0.7224 | -3.3 | -2.5 | QC1-4-211:1,hqc |
| 3.4 | -32767 | 3.4 | 3.3 | 3.4 | 3 | 3.26372 | 2.5 | 3.5 | QC1-4-211:1,hqc |
| -4.6 | -32767 | -4.1 | -2.7 | -4.35 | -4.15 | -13.26 | -4.3 | -4 | QC1-4-211:1 |
| 2.7 | -32767 | 1.6 | 2 | 2.15 | 1.35 | 5.194 | 0.9 | 1.8 | QC1-4-211:1,hqc |
| 5.4 | -32767 | 4.2 | 4.3 | 4.8 | 4.3 | 5.7732 | 3.6 | 5 | QC1-4-211:1 |
| 4.9 | -32767 | 5 | 3.4 | 4.95 | 5.1 | 5.47888 | 4.8 | 5.4 | QC1-4-211:1 |
| 3.2 | -32767 | 4.6 | 3.7 | 3.9 | 4.1 | 4.27896 | 3.6 | 4.6 | QC1-4-211:1 |
| -11.6 | -32767 | -9 | -10 | -10.3 | -9.55 | -10 | -10.2 | -8.9 | QC1-4-211:1,hqc,hqc |
| -9.5 | -32767 | -9.5 | -9.5 | -9.5 | -9.65 | -5.74 | -9.8 | -9.5 | QC1-4-211:1,hqc |
| 0.4 | -32767 | 1.4 | 1.2 | 0.9 | 1.8 | 0.50176 | 1.2 | 2.4 | QC1-4-211:1,hqc |
Given that the corrected value must lie between TAN and TAX, the bold values indicate possible actual values (other values are out of range). The model values are sometimes very different, but note (for example):
| -8.4 | -32767 | -8.6 | -8.5 | -8.5 | -8.55 | -10 | -8.6 | -8.5 | QC1-4-211:1,hqc |
Since TAN=-8.6, and TAX=-8.5, the expected value should lie between these values …but -8.5 may still be closer to the value than the mean -8.55!!!! Suggest only to use mean(TAX,TAN) when the corrected value is observed to be outside possible range ??? — Yes this is the current implementation
if ( Tseries[1].corrected() < MinT.begin()→original() || Tseries[1].corrected() > MaxT.begin()→original() ) { / Update if correction is out of TAN TAX range!
But needs testing and debugging !!
Some results comparing the method to the corrections already in place for missing data for March 2008:
Some thoughts on control flag settingsflag settings
===== General interpolation in both time and space and estimate a modelled value =====
[ALGOCODE:3]
This is a generalisation of processUnitT and identifies a gap of arbitrary length in a time series and identifies valid points before and after the gap up until the next gap on either side. This provides a small time series (with missing values) upon which the most appropriate temporal (and spatial … taking into account neighbours) may be applied.
<code>src/kvQc2/algorithms/Process4D.cc</code>