kvalobs:kvoss:system:qc2:user:algorithms

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision 		Previous revision
kvalobs:kvoss:system:qc2:user:algorithms [2009-09-26 23:05:15]
paule 		kvalobs:kvoss:system:qc2:user:algorithms [2022-05-31 09:29:32] (current)
Line 6: Line 6:
  
 ^ AlgoCode ^Description  ^Status  ^ ^ AlgoCode ^Description  ^Status  ^
-| 1 |Redistribute accumulated precipitation. |OK |+| 1 |Redistribute accumulated precipitation. |OK, but not released |
 | 2 |Generate variability parameters.   |Development. | | 2 |Generate variability parameters.   |Development. |
 | 3 |General interpolation in both time and space and estimate a modelled value   |Development | | 3 |General interpolation in both time and space and estimate a modelled value   |Development |
-| 4 |Temporal interpolation for "one-point" gaps (fill in missing values).   |Development |+| 4 |Temporal interpolation for "one-point" gaps (fill in missing values). Performs the update based on min and max recorded at the end of the hour.  |OK, but not released |
 | 5 |Perform only an interpolation.   |For testing and quality control. | | 5 |Perform only an interpolation.   |For testing and quality control. |
-| 6 |Null call ... does nothing.   |For testing only. |+| 6 |SpaceCheck: Set a quality control flag based on a comparison with neighbours. Configure for any implemented interpolation method.   |Development | 
 +| 7 |FlagTester: built in method to support the analysis of flag values.   |OK, but not released | 
 +| 8 |StressTester: method to set Qc2 running forever: the one algorithm repeats, e.g. continuous interpolation roots.   |OK, but not released | 
 +| 9 | SingleMinMaxAverage | **kvqc2_1.0.1** For a given paramid and corresponding Max paramid and Min paramid calculates a correction for an single missing value as average of max and min. If no max or min available or specified then reverts to simple linear. | 
 +| 10 | SingleLinear |**kvqc2_1.0.1** Replaces a single missing value with a linear interpolated value. If specified checks that the correction lies within the available max ... min range, if not sets to nearest of max or min. Run for any paramid and optional max, min specified in the configuration file. |   
 +| 11 |Null call ... does nothing.   |For testing only. |
 | other |Other values are handled gracefully and no algorithm initiated. |OK | | other |Other values are handled gracefully and no algorithm initiated. |OK |
  
Line 31: Line 36:
 **TBD** ... add logic to propose best model value ... some example data: **TBD** ... add logic to propose best model value ... some example data:
  
-^T0 ^T1 ^T2 ^T-Corrected ^Linear  ^T-Tan-Tax  ^T-neighbour  ^TAN ^TAX ^ +**T-Corrected:** The current supplied corrected value (typically a model value and/or HQC check)
-|6.2 |-32767 |6.8 |6.5 |6.5 |**6** |**5.83** |5.1 |6.9 | +
-|-3.5 |-32767 |-4.8 |**-4** |**-4.15** |**-4.35** |-0.7352 |-4.8 |-3.9 | +
-|-0.3 |-32767 |-1.7 |-1.2 |-1 |**-1.65** |0.2322 |-2.1 |-1.2 | +
-|0.1 |-32767 |-0.1 |**-0.1** |**0** |**-0.1** |-1.68 |-0.3 |0.1 | +
-|-8.5 |-32767 |-6.1 |**-7.5** |**-7.3** |**-6.85** |-16.0474 |-7.7 |-6 | +
-|-8.4 |-32767 |-8.6 |-8.5 |-8.5 |-8.55 |-10 |-8.6 |-8.5 | +
-|-6.4 |-32767 |-7.3 |-7 |-6.85 |-7.05 |6.4144 |-7.3 |-6.8 | +
-|-5.9 |-32767 |-6.4 |-6.2 |-6.15 |-6.1 |5.4016 |-6.4 |-5.8 | +
-|-16.7 |-32767 |-17.2 |-17 |-16.95 |-17.15 |-1.48 |-17.5 |-16.8 | +
-|-0.7 |-32767 |-2.7 |-1.2 |-1.7 |-1.9 |0 |-2.7 |-1.1 | +
-|-3 |-32767 |-2.8 |-2.8 |-2.9 |-2.9 |-0.7224 |-3.3 |-2.5 | +
-|3.4 |-32767 |3.4 |3.3 |3.4 |3 |3.26372 |2.5 |3.5 | +
-|-4.6 |-32767 |-4.1 |-2.7 |-4.35 |-4.15 |-13.26 |-4.3 |-4 | +
-|2.7 |-32767 |1.6 |2 |2.15 |1.35 |5.194 |0.9 |1.8 | +
-|5.4 |-32767 |4.2 |4.3 |4.8 |4.3 |5.7732 |3.6 |5 | +
-|4.9 |-32767 |5 |3.4 |4.95 |5.1 |5.47888 |4.8 |5.4 | +
-|3.2 |-32767 |4.6 |3.7 |3.9 |4.1 |4.27896 |3.6 |4.6 | +
-|-11.6 |-32767 |-9 |-10 |-10.3 |-9.55 |-10 |-10.2 |-8.9 | +
-|-9.5 |-32767 |-9.5 |-9.5 |-9.5 |-9.65 |-5.74 |-9.8 |-9.5 | +
-|0.4 |-32767 |1.4 |1.2 |0.9 |1.8 |0.50176 |1.2 |2.4 |+
  
 +**Linear:** Linear interpolation of the proceeding and following value.
 +
 +**T-Tan-Tax:** The mean of the Min and Max values for the same hour (recorded in the following hour) is another possible corrected value.
 +
 +**T-neighbour:** For reference, the value of the nearest neighbour.
 +
 +^T0 ^T1 ^T2 ^T-Corrected ^Linear  ^T-Tan-Tax  ^T-neighbour  ^TAN ^TAX ^CFAILED ^
 +|-0.3 |-32767 |-1.7 |-1.2 |-1 |**-1.65** |0.2322 |-2.1 |-1.2 |QC1-4-211:1,hqc | 
 +|0.1 |-32767 |-0.1 |**-0.1** |**0** |**-0.1** |-1.68 |-0.3 |0.1 |QC1-4-211:1,hqc |
 +|-8.5 |-32767 |-6.1 |**-7.5** |**-7.3** |**-6.85** |-16.0474 |-7.7 |-6 |QC1-4-211:1,hqc |
 +|-8.4 |-32767 |-8.6 |-8.5 |-8.5 |**-8.55** |-10 |-8.6 |-8.5 |QC1-4-211:1,hqc |
 +|-6.4 |-32767 |-7.3 |**-7** |**-6.85** |**-7.05** |6.4144 |-7.3 |-6.8 |QC1-4-211:1,hqc |
 +|-5.9 |-32767 |-6.4 |**-6.2** |**-6.15** |**-6.1** |5.4016 |-6.4 |-5.8 |QC1-4-211:1,hqc |
 +|-16.7 |-32767 |-17.2 |**-17** |**-16.95** |**-17.15** |-1.48 |-17.5 |-16.8 |QC1-4-211:1,hqc |
 +|-0.7 |-32767 |-2.7 |**-1.2** |**-1.7** |**-1.9** |0 |-2.7 |-1.1 |QC1-4-211:1,hqc |
 +|-3 |-32767 |-2.8 |**-2.8** |**-2.9** |**-2.9** |-0.7224 |-3.3 |-2.5 |QC1-4-211:1,hqc | 
 +|3.4 |-32767 |3.4 |**3.3** |**3.4** |**3** |3.26372 |2.5 |3.5 |QC1-4-211:1,hqc |
 +|-4.6 |-32767 |-4.1 |-2.7 |-4.35 |**-4.15** |-13.26 |-4.3 |-4 |QC1-4-211:1 |
 +|2.7 |-32767 |1.6 |2 |2.15 |**1.35** |5.194 |0.9 |1.8 |QC1-4-211:1,hqc |
 +|5.4 |-32767 |4.2 |**4.3** |**4.8** |**4.3** |5.7732 |3.6 |5 |QC1-4-211:1 | 
 +|4.9 |-32767 |5 |3.4 |**4.95** |**5.1** |5.47888 |4.8 |5.4 |QC1-4-211:1 |
 +|3.2 |-32767 |4.6 |**3.7** |**3.9** |**4.1** |4.27896 |3.6 |4.6 |QC1-4-211:1 |
 +|-11.6 |-32767 |-9 |**-10** |-10.3 |**-9.55** |-10 |-10.2 |-8.9 |QC1-4-211:1,hqc,hqc |
 +|-9.5 |-32767 |-9.5 |**-9.5** |**-9.5** |**-9.65** |-5.74 |-9.8 |-9.5 |QC1-4-211:1,hqc |
 +|0.4 |-32767 |1.4 |1.2 |0.9 |**1.8** |0.50176 |1.2 |2.4 |QC1-4-211:1,hqc |
 +
 +Given that the corrected value must lie between TAN and TAX, the bold values indicate possible actual values (other values are out of range). The model values are sometimes very different, but note (for example):
 +
 +|-8.4 |-32767 |-8.6 |-8.5 |-8.5 |**-8.55** |-10 |-8.6 |-8.5 |QC1-4-211:1,hqc |
 +
 +Since TAN=-8.6, and TAX=-8.5, the expected value should lie between these values ...but -8.5 may still be closer to the value than the mean -8.55!!!! Suggest only to use mean(TAX,TAN) when the corrected value is observed to be outside possible range ??? **--- Yes this is the current implementation**
 +
 +//if ( Tseries[1].corrected() <  MinT.begin()->original() || Tseries[1].corrected() >  MaxT.begin()->original() ) {  /// Update if correction is out of TAN TAX range!//FIXME ** But needs testing and debugging !! **
 +
 +Some results comparing the method to the corrections already in place for missing data for March 2008:
 +{{kvalobs:kvoss:system:qc2:user:200803unitt.jpg|}}
 +
 +Some thoughts on control [[.:algorithms:ProcessUnitTFlags|flag settings]]
  
 ===== General interpolation in both time and space and estimate a modelled value ===== ===== General interpolation in both time and space and estimate a modelled value =====
  • kvalobs/kvoss/system/qc2/user/algorithms.1254006315.txt.gz
  • Last modified: 2022-05-31 09:23:19
  • (external edit)