kvqc2d Version: 1.1.0

To Do When deploying into operation the kvalobs control scripts have to be updated to call “kvqc2d” and not “kvqc2”

In consequence, these are suggested steps to follow:

• kvstop -l {and select the value for kvqc2}
• SYSTEM: apt-get remove kvqc2
• SYSTEM: apt-get install kvqc2d
• run kvqc2d on the command line first to check all paths are in order (and also all the log files with new names are created) … Ctrl+C to Quit
• update /etc/kvalobs/kv_ctl.conf … i.e. replace all kvqc2 with kvqc2d
• kvstart

Source: https://svn.met.no/viewvc/kvoss/kvQc2/branches/kvqc2d-1.1.0/

Algorithms in this release:

Sample data from the new algorithm (scroll right for cfailed format)

NB on dev-vm101 this data could only be generated with I_fd=0 … normally this is set to I_fd=1 so that only QC1-controlled data is used for interpolation model values for the redistribution.

kvalobs=# select * from data where obstime between '2010-12-07' and 'today' and paramid=110 and cfailed like '%QC2-redist%' order by stationid, obstime;
 stationid |       obstime       | original | paramid |       tbtime        | typeid | sensor | level | corrected |   controlinfo    |     useinfo      |                                  cfailed                                  
-----------+---------------------+----------+---------+---------------------+--------+--------+-------+-----------+------------------+------------------+---------------------------------------------------------------------------
     25260 | 2010-12-07 06:00:00 |   -32767 |     110 | 2010-12-08 00:30:21 |    302 | 0      |     0 |        -1 | 0000001000007000 | 5896900000000000 | QC1-7-110,QC2N_25320_25630_25640_25100_25830_24890,QC2-redist
     25260 | 2010-12-08 06:00:00 |   -32767 |     110 | 2010-12-09 00:30:27 |    302 | 0      |     0 |        -1 | 0000001000007000 | 5896900000000000 | QC1-7-110,QC2N_25320_25630_25640_25100_53950_25830_24890,QC2-redist
     25260 | 2010-12-09 06:00:00 |       -1 |     110 | 2010-12-09 07:21:52 |    302 | 0      |     0 |        -1 | 0110004000007000 | 5336900000000000 | QC1-7-110,QC2N_25320_25630_25640_25100_53950_25830_24890,QC2-redist

     45350 | 2010-12-07 06:00:00 |   -32767 |     110 | 2010-12-08 00:30:35 |    302 | 0      |     0 |        -1 | 0000001000007000 | 5896900000000000 | QC1-7-110,QC2N_42940_45530_45600_45870_44760_42810_42520,QC2-redist
     45350 | 2010-12-08 06:00:00 |   -32767 |     110 | 2010-12-09 00:30:25 |    302 | 0      |     0 |       5.3 | 0000001000007000 | 5896900000000000 | QC1-7-110,QC2N_42940_45530_45600_45870_44760_42810_42520_44800,QC2-redist
     45350 | 2010-12-09 06:00:00 |        9 |     110 | 2010-12-09 11:40:20 |    302 | 0      |     0 |       3.7 | 0110004000007000 | 5336900000000000 | QC1-7-110,QC2N_42940_45530_45600_45870_44760_42810_42520_44800,QC2-redist

     84070 | 2010-12-07 06:00:00 |   -32767 |     110 | 2010-12-08 00:30:48 |    302 | 0      |     0 |        -1 | 0000001000007000 | 5896900000000000 | QC1-7-110,QC2N_84190,QC2-redist
     84070 | 2010-12-08 06:00:00 |   -32767 |     110 | 2010-12-09 00:30:33 |    302 | 0      |     0 |       0.2 | 0000001000007000 | 5896900000000000 | QC1-7-110,QC2N_84190_85180_83710,QC2-redist
     84070 | 2010-12-09 06:00:00 |      1.5 |     110 | 2010-12-09 07:03:10 |    302 | 0      |     0 |       1.3 | 0140004000007000 | 5336900000000001 | QC1-2-72.b12,QC1-7-110,QC2N_84190_83710,QC2-redist
(9 rows)

Example Configuration Files

*NB* Different/simpler checks applied to the neighbours (CF version 1.0.5)

# AlgCode 10 corresponds to https://svn.met.no/kvoss/kvQc2/branches/kvqc2-1.0.1/src/algorithms/SingleLinear.cc
# Performs simple linear interpolation for a single point replacement.
# If the result is outside an available max min range, the nearest of the
# max or min values are used.
AlgoCode=10
# Time to run the algorithm
RunAtHour=16
RunAtMinute=47
# Check data from the last three days
Last_NDays=40

#[Specific Data Type and Paramters ids etc.]
ParamId=211
# MaxParamId=215   # Optional  !!!! The Max and Min options are no longer required (to be modified).
# MinParamId=213   # Optional
# Only write back the result if not previously controlled
W_fhqc=0

#Flag to set if value is corrected
S_ftime=1
change_fmis=3->1
change_fmis=0->4
change_fmis=2->4

#If there is already a 1 in place then we do not want run the algorithm again!
U_2=0

# Algorithm configuration for redistribution of accumulated precipitation
AlgoCode=1
InterpCode=2
RunAtHour=23
RunAtMinute=35
Last_NDays=14

#[Time Range] The time window for when to apply the algorithm
#Start_YYYY=2010
#Start_MM=10 
#Start_DD=8
#Start_hh=6
#Start_mm=0
#Start_ss=0

#End_YYYY=2010 
#End_MM=10
#End_DD=11
#End_hh=6
#End_mm=0
#End_ss=0


Start_hh=6
End_hh=6   # NB 24 hour values are at 06:00 UTC

#[Time Step]
Step_DD=1

#[Specific Data Type and Paramters ids etc.]
ParamId=110
TypeIds=302
TypeIds=402

#
# Only write data it it has not been analysed by HQC
W_fhqc=0

# Added for a special test run
# CfailedString="any text"


# Condition for running the algorithm
A_fd=2

# Flags to set on successful completion
S_fd=7
change_fmis=3->1
change_fmis=0->4

#Only interpolate the following data
I_fd=1

InterpolationDistance=50.0

Notes

• This algorithm will redistribute data flagged with fd=2 (A_fd) and use neighbours with fd=1 for interpolation (I_fd) and set fd=7 (S_fd) for the new corrected data. But fhqc must be 0 for data to be written back.
• To re-run the algorithm on the same data again set A_fd=7 and keep W_fhqc=0, i.e. do not re-write any decision by HQC.

The schedule for the algorithm is as follows:

• 06:35 perform a run looking at the last 4 days - the last 4 days is selected since from the first run on murky (see above) about 30 redistributions were made of which 15 were over 3 days and 10 over 4 days.
• 01:20 Wednesday Night/Thursday Morning run to check the last 14 days (Note: Could we run every night for the last two weeks to pick up any changes?)