Differences

This shows you the differences between two versions of the page.

--- kvalobs:kvoss:system:qc2:user:version:kvqc2-1.0.5 [2010-09-05 09:49:01]
paule created
+++ kvalobs:kvoss:system:qc2:user:version:kvqc2-1.0.5 [2022-05-31 09:29:32] (current)
@@ Line 1: / Line 1: @@
 ====== kvqc2 Version: 1.0.5 ======
-[[https://svn.met.no/viewvc/kvoss/kvQc2/branches/kvqc2-1.0.5/]]
+[[https://svn.met.no/viewvc/kvoss/kvqc2d/branches/kvqc2-1.0.5/]]
 **Algorithms in this release:**
 ^ AlgoCode ^Name  ^Description  ^
-| 10 | SingleLinear |**kvqc2_1.0.5** Replaces a single missing value with a linear interpolated value. If specified checks that the correction lies within the available max ... min range, if not sets to nearest of max or min. Run for any paramid and optional max, min specified in the configuration file. [[https://svn.met.no/kvoss/kvQc2/branches/kvqc2-1.0.5/src/algorithms/SingleLinear.cc|source]]. Note, there is no change to this algorithm as compared with version 1.0.4 however the initia settings in the configuration file are updated (see below).|
+| 10 | SingleLinear | Replaces a single missing value with a linear interpolated value. If a corresponding max or min paramids are also specified then checks that the correction lies within the available max ... min range, if the interpolated values falls outside this range then the algorithm sets the corrected value to the nearest of max or min. Run for any paramid and optional max, min specified in the configuration file. [[https://svn.met.no/kvoss/kvQc2/branches/kvqc2-1.0.5/src/algorithms/SingleLinear.cc|source]]. Note, there is no change to this algorithm as compared with version 1.0.4 however the initia settings in the configuration file are updated (see below).|
+| 7 | FlagTester |[[https://svn.met.no/kvoss/kvQc2/branches/kvqc2-1.0.5/src/algorithms/FlagTester.cc|source]] Details provided with the example configuration file below.|
+**Example Configuration Files**
+^SingleLinear ^
-**Example Configuration Files**
+<code>
+# AlgCode 10 corresponds to https://svn.met.no/kvoss/kvQc2/branches/kvqc2-1.0.1/src/algorithms/SingleLinear.cc
+# Performs simple linear interpolation for a single point replacement.
+# If the result is outside an available max min range, the nearest of the
+# max or min values are used.
+AlgoCode=10
+# Time to run the algorithm
+RunAtHour=16
+RunAtMinute=47
+# Check data from the last three days
+Last_NDays=40
+#[Specific Data Type and Paramters ids etc.]
+ParamId=211
+MaxParamId=215
+MinParamId=213
+# Only write back the result if not previously controlled
+W_fhqc=0
+#Flag to set if value is corrected
+S_ftime=1
+change_fmis=3->1
+change_fmis=0->4
+change_fmis=2->4
+#If there is already a 1 in place then we do not want run the algorithm again!
+Not_ftime=1   #
+Not_fnum=6    # This is to cut out the data with large values
+U_2=0
+</code>
+^FlagTester ^
+This is a small utility to help check the behaviour of flag settings with the introduction of Qc2. Procedure:
+  - Provide a set of input flags in a text file as "controlinfo useinfo" pairs
+  - Add to the configuration file the desired conntrolinfo changes to apply.
+  - The algorithm changes the controlinfo and then obtains an update to the useinfo from kvUseInfo::setUseFlags based on the values in the controlflag.
+  - The results are written to a text file for later inspection.
+**Example configuration file with further comments inline ...**
+<code>
+#[Flag Analysis Utility]
+AlgoCode=7
+RunAtHour=22
+RunAtMinute=33
+# File containing test flags, the format of the file is as follows
+# controlinfo useinfo
+#
+# If no file is specified the algorithm does not run.
+#
+# e.g.
+# 0000601000000000 9894700000000000
+# 0000002000000000 7039900000000000
+# 0000002000000000 9039900000000000
+# 0000602000000000 7033700000000000
+# 0000003000000000 9899900000000000
+#
+FlagsIn="/metno/kvalobs/FlagsIn.dat"
+#Flags to change in the controlinfo
+S_ftime=1
+change_fmis=3->1
+change_fmis=0->4
+change_fmis=2->4
+# File where the resultant flags are written
+FlagsOut="/metno/kvalobs/FlagsOut.dat"
+#
+# If no file is specified the results are written to "/tmp/Qc2-Flagtest-Output.dat"
+#
+</code>
@@ Line 16: / Line 88: @@
 No algorithms in this release perform any spatial interpolation hence the InterpCode configuration parameter, if set, has no effect.
+====== Test Results ======
+**FlagTester: **
+^Flags before ^^Flags After ^^
+|0000601000000000 | 9894700000000000 | 0000601100000000 | 5894700000000001 |
+|0000002000000000 | 7039900000000000 | 0000004100000000 | 6093900000000000 |
+|0000002000000000 | 9039900000000000 | 0000004100000000 | 6093900000000000 |
+|0000602000000000 | 7033700000000000 | 0000604100000000 | 5033700000000001 |
+|0000003000000000 | 9899900000000000 | 0000001100000000 | 6894900000000000 |
+**SingleLinear ... Standard use case on dev-vm101:**
+<code>
+kvalobs=# select * from data_history where stationid=99737 and paramid=211 and obstime='2010-09-05 20:00:00' order by modificationtime;
+ version  | stationid |       obstime       | original | paramid |       tbtime        | typeid | sensor | level | corrected |   controlinfo    |     useinfo      | cfailed  |      modificationtime
+----------+-----------+---------------------+----------+---------+---------------------+--------+--------+-------+-----------+------------------+------------------+----------+----------------------------
+ 75619821 |     99737 | 2010-09-05 20:00:00 |   -32767 |     211 | 2010-09-05 20:30:19 |    330 | 0      |     0 |    -32767 | 0000003000000000 | 9899900000000000 |          | 2010-09-05 20:30:19.055969
+ 75651169 |     99737 | 2010-09-05 20:00:00 |          |     211 | 2010-09-05 20:30:19 |    330 | 0      |     0 |           |                  |                  |          | 2010-09-05 23:05:23.952628
+ 75651170 |     99737 | 2010-09-05 20:00:00 |   -32767 |     211 | 2010-09-05 20:30:19 |    330 | 0      |     0 |       2.9 | 0000001100000000 | 6894900000000000 |  QC2d-2  | 2010-09-05 23:05:23.962193
+ 75651171 |     99737 | 2010-09-05 20:00:00 |          |     211 | 2010-09-05 20:30:19 |    330 | 0      |     0 |           |                  |                  |          | 2010-09-05 23:05:23.976896
+ 75651172 |     99737 | 2010-09-05 20:00:00 |   -32767 |     211 | 2010-09-05 20:30:19 |    330 | 0      |     0 |       2.9 | 0000001100000000 | 9894900000000000 |  QC2d-2  | 2010-09-05 23:05:23.989779
+(5 rows)
+</code>
+Something odd above needs investigating ... **NOW INVESTIGATED**  ... there were two versions of kvqc2 running at the same time ... now cleaned up.