This is an old revision of the document!
Qc2 Flag End-to-End
| Step | Direction | Expected Result | Pass/Fail Criteria or Comment |
|---|---|---|---|
| 001 | Run kvalobs Qc1 and copy a selection of data that a Qc2 algorithm exists for, e.g. last 5 days. | – | – |
| 002 | Run kvalobs Qc2 on the same interval. | – | – |
| 003 | Copy the same selection of data as for Step 001 to a new file. | – | – |
| 004 | Compare the data before and after to confirm the expected behaviour in the flag changes. | – | – |
Here is a worked example:
Capture the status of these data before Qc2:
kvalobs=# select * from data WHERE obstime>'2010-03-19' and obstime<'2010-03-25' and paramid=211 \g | cat >> ./PRE-ALG004-20100319-24.dat;
Set the Qc2 algorithm running (ProcessUnit for this case).
Capture the status of the same data after Qc2:
kvalobs=# select * from data WHERE obstime>'2010-03-19' and obstime<'2010-03-25' and paramid=211 \g | cat >> ./POST-ALG004-20100319-24.dat;
kvalobs@dev-vm116:~/Qc2Config$ head POST-ALG004-20100319-24.dat
stationid | obstime | original | paramid | tbtime | typeid | sensor | level | corrected | controlinfo | useinfo | cfailed
-----------+---------------------+----------+---------+---------------------+--------+--------+-------+-----------+------------------+------------------+-------------------------------------
12550 | 2010-03-19 01:00:00 | 1.8 | 211 | 2010-03-19 00:04:14 | 342 | 0 | 0 | 1.8 | 1100000000000000 | 7100000300000000 |
80102 | 2010-03-19 01:00:00 | 0.4 | 211 | 2010-03-19 00:41:58 | 330 | 0 | 0 | 0.4 | 1100000000000000 | 7100000300000000 |
48330 | 2010-03-19 01:00:00 | 6.9 | 211 | 2010-03-19 00:54:14 | 330 | 0 | 0 | 6.9 | 1100000000000000 | 7000000000000000 |
64330 | 2010-03-19 01:00:00 | 3.3 | 211 | 2010-03-19 00:56:34 | 311 | 0 | 0 | 3.3 | 1100000000000000 | 7000000000000000 |
36200 | 2010-03-19 01:00:00 | 4.4 | 211 | 2010-03-19 00:50:58 | 330 | 0 | 0 | 4.4 | 1100000000000000 | 7000000000000000 |
46610 | 2010-03-19 01:00:00 | 2.3645 | 211 | 2010-03-19 00:49:52 | 330 | 0 | 0 | 2.3645 | 1100000000000000 | 7100000300000000 |
17150 | 2010-03-19 01:00:00 | 1.9 | 211 | 2010-03-19 00:49:28 | 342 | 0 | 0 | 1.9 | 1100000000000000 | 7100000300000000 |
99760 | 2010-03-19 01:00:00 | -18 | 211 | 2010-03-19 00:51:38 | 311 | 0 | 0 | -18 | 1100000000000000 | 7000000000000000 |
kvalobs@dev-vm116:~/Qc2Config$ head PRE-ALG004-20100319-24.dat
stationid | obstime | original | paramid | tbtime | typeid | sensor | level | corrected | controlinfo | useinfo | cfailed
-----------+---------------------+----------+---------+---------------------+--------+--------+-------+-----------+------------------+------------------+-------------------------------------
12550 | 2010-03-19 01:00:00 | 1.8 | 211 | 2010-03-19 00:04:14 | 342 | 0 | 0 | 1.8 | 1100000000000000 | 7100000300000000 |
80102 | 2010-03-19 01:00:00 | 0.4 | 211 | 2010-03-19 00:41:58 | 330 | 0 | 0 | 0.4 | 1100000000000000 | 7100000300000000 |
48330 | 2010-03-19 01:00:00 | 6.9 | 211 | 2010-03-19 00:54:14 | 330 | 0 | 0 | 6.9 | 1100000000000000 | 7000000000000000 |
64330 | 2010-03-19 01:00:00 | 3.3 | 211 | 2010-03-19 00:56:34 | 311 | 0 | 0 | 3.3 | 1100000000000000 | 7000000000000000 |
36200 | 2010-03-19 01:00:00 | 4.4 | 211 | 2010-03-19 00:50:58 | 330 | 0 | 0 | 4.4 | 1100000000000000 | 7000000000000000 |
46610 | 2010-03-19 01:00:00 | 2.3645 | 211 | 2010-03-19 00:49:52 | 330 | 0 | 0 | 2.3645 | 1100000000000000 | 7100000300000000 |
17150 | 2010-03-19 01:00:00 | 1.9 | 211 | 2010-03-1900:49:28 | 342 | 0 | 0 | 1.9 | 1100000000000000 | 7100000300000000 |
99760 | 2010-03-19 01:00:00 | -18 | 211 | 2010-03-19 00:51:38 | 311 | 0 | 0 | -18 | 1100000000000000 | 7000000000000000 |
Find the rows modified by Qc2 in the “POST” file:
kvalobs@dev-vm116:~/Qc2Config$ grep Qc2 POST-ALG004-20100319-24.dat
99754 | 2010-03-23 15:00:00 | -32767 | 211 | 2010-03-23 15:30:57 | 1 | 0 | 0 | -13.6 | 0000003100000000 | 9899900000000000 | Qc2 UnitT corrected was:-32767
17150 | 2010-03-23 13:00:00 | -32767 | 211 | 2010-03-23 13:30:11 | 342 | 0 | 0 | 4 | 0000003100000000 | 9899900000000000 | Qc2 UnitT corrected was:-32767
61060 | 2010-03-22 23:00:00 | -32767 | 211 | 2010-03-23 02:51:52 | 330 | 0 | 0 | 1.6 | 0000003100000000 | 9899900000000000 | Qc2 UnitT corrected was:-32767
Find the corresponding rows in the “POST” file:
kvalobs@dev-vm116:~/Qc2Config$ grep "99754 | 2010-03-23 15:00:00"
PRE-ALG004-20100319-24.dat
99754 | 2010-03-23 15:00:00 | -32767 | 211 | 2010-03-23 15:30:57 | 1 | 0 | 0 | -32767 | 0000003000000000 | 9899900000000000 |
kvalobs@dev-vm116:~/Qc2Config$ grep "17150 | 2010-03-23 13:00:00 |" PRE-ALG004-20100319-24.dat
17150 | 2010-03-23 13:00:00 | -32767 | 211 | 2010-03-2313:30:11 | 342 | 0 | 0 | -32767 | 0000003000000000 | 9899900000000000 |
kvalobs@dev-vm116:~/Qc2Config$ grep "61060 | 2010-03-22 23:00:00 |" PRE-ALG004-20100319-24.dat
61060 | 2010-03-22 23:00:00 | -32767 | 211 | 2010-03-23 02:51:52 | 330 | 0 | 0 | -32767 | 0000003000000000 | 9899900000000000 |
kvalobs@dev-vm116:~/Qc2Config$