kvqc2d Version: 1.1.0

To Do When deploying into operation the kvalobs control scripts have to be updated to call “kvqc2d” and not “kvqc2”

Source: https://svn.met.no/viewvc/kvoss/kvQc2/branches/kvqc2d-1.1.0/

Algorithms in this release:

AlgoCode	Name	Description
10	SingleLinear_v32	Replaces a single missing value with a linear interpolated value. If a corresponding max or min paramids are also specified then checks that the correction lies within the available max ... min range, if the interpolated values falls outside this range then the algorithm sets the corrected value to the nearest of max or min. Run for any paramid and optional max, min specified in the configuration file. New version updated with respect to a revised specification
1	Redistribution of RR_24	See https://kvalobs.wiki.met.no/doku.php?id=kvoss:system:qc2:requirements:algorithms:rarr_24 and config file below.

Sample data from the new algorithm

NB on dev-vm101 this data could only be generated with I_fd=0 ... normally this is set to I_fd=1 so that only QC1-controlled data is used for interpolation model values for the redistribution.

kvalobs=# select * from data where obstime between '2010-12-07' and 'today' and paramid=110 and cfailed like '%QC2-redist%' order by stationid, obstime;
stationid | obstime | original | paramid | tbtime | typeid | sensor | level | corrected | controlinfo | useinfo | cfailed
-----------+---------------------+----------+---------+---------------------+--------+--------+-------+-----------+------------------+------------------+---------------------------------------------------------------------------
25260 | 2010-12-07 06:00:00 | -32767 | 110 | 2010-12-08 00:30:21 | 302 | 0 | 0 | -1 | 0000001000007000 | 5896900000000000 | QC1-7-110,QC2N_25320_25630_25640_25100_25830_24890,QC2-redist
25260 | 2010-12-08 06:00:00 | -32767 | 110 | 2010-12-09 00:30:27 | 302 | 0 | 0 | -1 | 0000001000007000 | 5896900000000000 | QC1-7-110,QC2N_25320_25630_25640_25100_53950_25830_24890,QC2-redist
25260 | 2010-12-09 06:00:00 | -1 | 110 | 2010-12-09 07:21:52 | 302 | 0 | 0 | -1 | 0110004000007000 | 5336900000000000 | QC1-7-110,QC2N_25320_25630_25640_25100_53950_25830_24890,QC2-redist
45350 | 2010-12-07 06:00:00 | -32767 | 110 | 2010-12-08 00:30:35 | 302 | 0 | 0 | -1 | 0000001000007000 | 5896900000000000 | QC1-7-110,QC2N_42940_45530_45600_45870_44760_42810_42520,QC2-redist
45350 | 2010-12-08 06:00:00 | -32767 | 110 | 2010-12-09 00:30:25 | 302 | 0 | 0 | 5.3 | 0000001000007000 | 5896900000000000 | QC1-7-110,QC2N_42940_45530_45600_45870_44760_42810_42520_44800,QC2-redist
45350 | 2010-12-09 06:00:00 | 9 | 110 | 2010-12-09 11:40:20 | 302 | 0 | 0 | 3.7 | 0110004000007000 | 5336900000000000 | QC1-7-110,QC2N_42940_45530_45600_45870_44760_42810_42520_44800,QC2-redist
84070 | 2010-12-07 06:00:00 | -32767 | 110 | 2010-12-08 00:30:48 | 302 | 0 | 0 | -1 | 0000001000007000 | 5896900000000000 | QC1-7-110,QC2N_84190,QC2-redist
84070 | 2010-12-08 06:00:00 | -32767 | 110 | 2010-12-09 00:30:33 | 302 | 0 | 0 | 0.2 | 0000001000007000 | 5896900000000000 | QC1-7-110,QC2N_84190_85180_83710,QC2-redist
84070 | 2010-12-09 06:00:00 | 1.5 | 110 | 2010-12-09 07:03:10 | 302 | 0 | 0 | 1.3 | 0140004000007000 | 5336900000000001 | QC1-2-72.b12,QC1-7-110,QC2N_84190_83710,QC2-redist
(9 rows)

Example Configuration Files

SingleLinear_v32

*NB* Different/simpler checks applied to the neighbours (CF version 1.0.5)

# AlgCode 10 corresponds to https://svn.met.no/kvoss/kvQc2/branches/kvqc2-1.0.1/src/algorithms/SingleLinear.cc
# Performs simple linear interpolation for a single point replacement.
# If the result is outside an available max min range, the nearest of the
# max or min values are used.
AlgoCode=10
# Time to run the algorithm
RunAtHour=16
RunAtMinute=47
# Check data from the last three days
Last_NDays=40
#[Specific Data Type and Paramters ids etc.]
ParamId=211
# MaxParamId=215 # Optional !!!! The Max and Min options are no longer required (to be modified).
# MinParamId=213 # Optional
# Only write back the result if not previously controlled
W_fhqc=0
#Flag to set if value is corrected
S_ftime=1
change_fmis=3->1
change_fmis=0->4
change_fmis=2->4
#If there is already a 1 in place then we do not want run the algorithm again!
U_2=0

Redistribution_v32

# Algorithm configuration for redistribution of accumulated precipitation
AlgoCode=1
InterpCode=2
RunAtHour=23
RunAtMinute=35
Last_NDays=14
#[Time Range] The time window for when to apply the algorithm
#Start_YYYY=2010
#Start_MM=10
#Start_DD=8
#Start_hh=6
#Start_mm=0
#Start_ss=0
#End_YYYY=2010
#End_MM=10
#End_DD=11
#End_hh=6
#End_mm=0
#End_ss=0
Start_hh=6
End_hh=6 # NB 24 hour values are at 06:00 UTC
#[Time Step]
Step_DD=1
#[Specific Data Type and Paramters ids etc.]
ParamId=110
TypeIds=302
TypeIds=402
#
# Only write data it it has not been analysed by HQC
# W_fhqc=9 # 9 set here because of the test data being used ...
W_fhqc=0
# Added for a special test run
# CfailedString="any text"
# Condition for running the algorithm
A_fd=1
# Flags to set on successful completion
S_fd=7
change_fmis=3->1
change_fmis=0->4
#Only interpolate the following data
I_fd=1
InterpolationDistance=50.0

Bugs and new requirements addressed in this release

General bugs identified in algorithm testing

https://kvoss.bugs.met.no/show_bug.cgi?id=1296

Name Changes

https://kvoss.bugs.met.no/show_bug.cgi?id=1302

renaming of kvqc2 to kvqc2d

renaming of Qc2.log to kvqc2d.lo

Additional logging information

https://kvoss.bugs.met.no/show_bug.cgi?id=1303

The log file now looks like this:

20101213010007: INFO
--------------- Case 1: Redistribute
20101213010007: INFO
--------------- Redistribute Accumulations
20101213010458: INFO
--------------- Redistribution: 45350 2010-12-7 6:0:0 -32767 110 302 0 0 -1
20101213010458: INFO
--------------- Redistribution: 45350 2010-12-8 6:0:0 -32767 110 302 0 0 5.3
20101213010458: INFO
--------------- Redistribution: 45350 2010-12-9 6:0:0 9 110 302 0 0 3.7
20101213010459: INFO
--------------- Redistribution: 25260 2010-12-7 6:0:0 -32767 110 302 0 0 -1
20101213010459: INFO
--------------- Redistribution: 25260 2010-12-8 6:0:0 -32767 110 302 0 0 -1
20101213010459: INFO
--------------- Redistribution: 25260 2010-12-9 6:0:0 -1 110 302 0 0 -1
20101213010459: INFO
--------------- Redistribution: 84070 2010-12-7 6:0:0 -32767 110 302 0 0 -1
20101213010459: INFO
--------------- Redistribution: 84070 2010-12-8 6:0:0 -32767 110 302 0 0 0.2
20101213010459: INFO
--------------- Redistribution: 84070 2010-12-9 6:0:0 1.5 110 302 0 0 1.3
20101213010551: INFO
--------------- Case 1: Completed

By default assume dry (0 -> -1)

https://kvoss.bugs.met.no/show_bug.cgi?id=1304

Revised typeid policy

https://kvoss.bugs.met.no/show_bug.cgi?id=1305

CFAILED

The format of cfailed for the Redistribution algorithm has been changed (please see example data above).