.... in preparation ...

kvqc2d Version: 1.1.2

Source: https://svn.met.no/viewvc/kvoss/kvQc2/branches/kvqc2d-1.1.2/

Algorithms in this release:

AlgoCode	Name	Description
10	SingleLinear_v32	Replaces a single missing value with a linear interpolated value. If a corresponding max or min paramids are also specified then checks that the correction lies within the available max ... min range, if the interpolated values falls outside this range then the algorithm sets the corrected value to the nearest of max or min. Run for any paramid and optional max, min specified in the configuration file. New version updated with respect to a revised specification
1	Redistribution of RR_24	See https://kvalobs.wiki.met.no/doku.php?id=kvoss:system:qc2:requirements:algorithms:rarr_24 and config file below.

Sample data from the new algorithm (scroll right for cfailed format)

NB on dev-vm101 this data could only be generated with I_fd=0 ... normally this is set to I_fd=1 so that only QC1-controlled data is used for interpolation model values for the redistribution.

kvalobs=# select * from data where obstime between '2010-12-07' and 'today' and paramid=110 and cfailed like '%QC2-redist%' order by stationid, obstime;
stationid | obstime | original | paramid | tbtime | typeid | sensor | level | corrected | controlinfo | useinfo | cfailed
-----------+---------------------+----------+---------+---------------------+--------+--------+-------+-----------+------------------+------------------+---------------------------------------------------------------------------
25260 | 2010-12-07 06:00:00 | -32767 | 110 | 2010-12-08 00:30:21 | 302 | 0 | 0 | -1 | 0000001000007000 | 5896900000000000 | QC1-7-110,QC2N_25320_25630_25640_25100_25830_24890,QC2-redist
25260 | 2010-12-08 06:00:00 | -32767 | 110 | 2010-12-09 00:30:27 | 302 | 0 | 0 | -1 | 0000001000007000 | 5896900000000000 | QC1-7-110,QC2N_25320_25630_25640_25100_53950_25830_24890,QC2-redist
25260 | 2010-12-09 06:00:00 | -1 | 110 | 2010-12-09 07:21:52 | 302 | 0 | 0 | -1 | 0110004000007000 | 5336900000000000 | QC1-7-110,QC2N_25320_25630_25640_25100_53950_25830_24890,QC2-redist
45350 | 2010-12-07 06:00:00 | -32767 | 110 | 2010-12-08 00:30:35 | 302 | 0 | 0 | -1 | 0000001000007000 | 5896900000000000 | QC1-7-110,QC2N_42940_45530_45600_45870_44760_42810_42520,QC2-redist
45350 | 2010-12-08 06:00:00 | -32767 | 110 | 2010-12-09 00:30:25 | 302 | 0 | 0 | 5.3 | 0000001000007000 | 5896900000000000 | QC1-7-110,QC2N_42940_45530_45600_45870_44760_42810_42520_44800,QC2-redist
45350 | 2010-12-09 06:00:00 | 9 | 110 | 2010-12-09 11:40:20 | 302 | 0 | 0 | 3.7 | 0110004000007000 | 5336900000000000 | QC1-7-110,QC2N_42940_45530_45600_45870_44760_42810_42520_44800,QC2-redist
84070 | 2010-12-07 06:00:00 | -32767 | 110 | 2010-12-08 00:30:48 | 302 | 0 | 0 | -1 | 0000001000007000 | 5896900000000000 | QC1-7-110,QC2N_84190,QC2-redist
84070 | 2010-12-08 06:00:00 | -32767 | 110 | 2010-12-09 00:30:33 | 302 | 0 | 0 | 0.2 | 0000001000007000 | 5896900000000000 | QC1-7-110,QC2N_84190_85180_83710,QC2-redist
84070 | 2010-12-09 06:00:00 | 1.5 | 110 | 2010-12-09 07:03:10 | 302 | 0 | 0 | 1.3 | 0140004000007000 | 5336900000000001 | QC1-2-72.b12,QC1-7-110,QC2N_84190_83710,QC2-redist
(9 rows)

Example Configuration Files

SingleLinear_v32

*NB* Different/simpler checks applied to the neighbours (CF version 1.0.5)

# AlgCode 10 corresponds to https://svn.met.no/kvoss/kvQc2/branches/kvqc2-1.0.1/src/algorithms/SingleLinear.cc
# Performs simple linear interpolation for a single point replacement.
# If the result is outside an available max min range, the nearest of the
# max or min values are used.
AlgoCode=10
# Time to run the algorithm
RunAtHour=16
RunAtMinute=47
# Check data from the last three days
Last_NDays=40
#[Specific Data Type and Paramters ids etc.]
ParamId=211
# MaxParamId=215 # Optional !!!! The Max and Min options are no longer required (to be modified).
# MinParamId=213 # Optional
# Only write back the result if not previously controlled
W_fhqc=0
#Flag to set if value is corrected
S_ftime=1
change_fmis=3->1
change_fmis=0->4
change_fmis=2->4
#If there is already a 1 in place then we do not want run the algorithm again!
U_2=0

Redistribution

# Algorithm configuration for redistribution of accumulated precipitation
AlgoCode=1
InterpCode=2
RunAtHour=23
RunAtMinute=35
Last_NDays=14
#[Time Range] The time window for when to apply the algorithm
#Start_YYYY=2010
#Start_MM=10
#Start_DD=8
#Start_hh=6
#Start_mm=0
#Start_ss=0
#End_YYYY=2010
#End_MM=10
#End_DD=11
#End_hh=6
#End_mm=0
#End_ss=0
Start_hh=6
End_hh=6 # NB 24 hour values are at 06:00 UTC
#[Time Step]
Step_DD=1
#[Specific Data Type and Paramters ids etc.]
ParamId=110
TypeIds=302
TypeIds=402
#
# Only write data it it has not been analysed by HQC
W_fhqc=0
# Added for a special test run
# CfailedString="any text"
# Condition for running the algorithm
A_fd=2
# Flags to set on successful completion
S_fd=7
change_fmis=3->1
change_fmis=0->4
#Only interpolate the following data
I_fd=1
InterpolationDistance=50.0

Notes

This algorithm will redistribute data flagged with fd=2 (A_fd) and use neighbours with fd=1 for interpolation (I_fd) and set fd=7 (S_fd) for the new corrected data. But fhqc must be 0 for data to be written back.

To re-run the algorithm on the same data again set A_fd=7 and keep W_fhqc=0, i.e. do not re-write any decision by HQC.

The schedule for the algorithm is as follows:

06:35 perform a run looking at the last 4 days - the last 4 days is selected since from the first run on murky (see above) about 30 redistributions were made of which 15 were over 3 days and 10 over 4 days.

01:20 Wednesday Night/Thursday Morning run to check the last 14 days (Note: Could we run every night for the last two weeks to pick up any changes?)