This is an old revision of the document!
Qc2 Flag End-to-End
| Step | Direction | Expected Result | Pass/Fail Criteria or Comment |
|---|---|---|---|
| 001 | Run kvalobs Qc1 and copy a selection of data that a Qc2 algorithm exists for, e.g. last 5 days. | – | – |
| 002 | Run kvalobs Qc2 on the same interval. | – | – |
| 003 | Copy the same selection of data as for Step 001 to a new file. | – | – |
| 004 | Compare the data before and after to confirm the expected behaviour in the flag changes. | – | – |
Here is a worked example:
Capture the status of these data before Qc2:
kvalobs=# select * from data WHERE obstime>'2010-03-19' and obstime<'2010-03-25' and paramid=211 \g | cat >> ./PRE-ALG004-20100319-24.dat;
<emph>Set the Qc2 algorithm running (ProcessUnit for this case).</emph>
Capture the status of the same data after Qc2:
kvalobs=# select * from data WHERE obstime>'2010-03-19' and obstime<'2010-03-25' and paramid=211 \g | cat >> ./POST-ALG004-20100319-24.dat;
Review results
kvalobs@dev-vm116:~/Qc2Config$ head POST-ALG004-20100319-24.dat
stationid | obstime | original | paramid | tbtime | typeid | sensor | level | corrected | controlinfo | useinfo | cfailed
-----------+---------------------+----------+---------+---------------------+--------+--------+-------+-----------+------------------+------------------+-------------------------------------
12550 | 2010-03-19 01:00:00 | 1.8 | 211 | 2010-03-19 00:04:14 | 342 | 0 | 0 | 1.8 | 1100000000000000 | 7100000300000000 |
80102 | 2010-03-19 01:00:00 | 0.4 | 211 | 2010-03-19 00:41:58 | 330 | 0 | 0 | 0.4 | 1100000000000000 | 7100000300000000 |
48330 | 2010-03-19 01:00:00 | 6.9 | 211 | 2010-03-19 00:54:14 | 330 | 0 | 0 | 6.9 | 1100000000000000 | 7000000000000000 |
64330 | 2010-03-19 01:00:00 | 3.3 | 211 | 2010-03-19 00:56:34 | 311 | 0 | 0 | 3.3 | 1100000000000000 | 7000000000000000 |
36200 | 2010-03-19 01:00:00 | 4.4 | 211 | 2010-03-19 00:50:58 | 330 | 0 | 0 | 4.4 | 1100000000000000 | 7000000000000000 |
46610 | 2010-03-19 01:00:00 | 2.3645 | 211 | 2010-03-19 00:49:52 | 330 | 0 | 0 | 2.3645 | 1100000000000000 | 7100000300000000 |
17150 | 2010-03-19 01:00:00 | 1.9 | 211 | 2010-03-19 00:49:28 | 342 | 0 | 0 | 1.9 | 1100000000000000 | 7100000300000000 |
99760 | 2010-03-19 01:00:00 | -18 | 211 | 2010-03-19 00:51:38 | 311 | 0 | 0 | -18 | 1100000000000000 | 7000000000000000 |
Review results
kvalobs@dev-vm116:~/Qc2Config$ head PRE-ALG004-20100319-24.dat
stationid | obstime | original | paramid | tbtime | typeid | sensor | level | corrected | controlinfo | useinfo | cfailed
-----------+---------------------+----------+---------+---------------------+--------+--------+-------+-----------+------------------+------------------+-------------------------------------
12550 | 2010-03-19 01:00:00 | 1.8 | 211 | 2010-03-19 00:04:14 | 342 | 0 | 0 | 1.8 | 1100000000000000 | 7100000300000000 |
80102 | 2010-03-19 01:00:00 | 0.4 | 211 | 2010-03-19 00:41:58 | 330 | 0 | 0 | 0.4 | 1100000000000000 | 7100000300000000 |
48330 | 2010-03-19 01:00:00 | 6.9 | 211 | 2010-03-19 00:54:14 | 330 | 0 | 0 | 6.9 | 1100000000000000 | 7000000000000000 |
64330 | 2010-03-19 01:00:00 | 3.3 | 211 | 2010-03-19 00:56:34 | 311 | 0 | 0 | 3.3 | 1100000000000000 | 7000000000000000 |
36200 | 2010-03-19 01:00:00 | 4.4 | 211 | 2010-03-19 00:50:58 | 330 | 0 | 0 | 4.4 | 1100000000000000 | 7000000000000000 |
46610 | 2010-03-19 01:00:00 | 2.3645 | 211 | 2010-03-19 00:49:52 | 330 | 0 | 0 | 2.3645 | 1100000000000000 | 7100000300000000 |
17150 | 2010-03-19 01:00:00 | 1.9 | 211 | 2010-03-1900:49:28 | 342 | 0 | 0 | 1.9 | 1100000000000000 | 7100000300000000 |
99760 | 2010-03-19 01:00:00 | -18 | 211 | 2010-03-19 00:51:38 | 311 | 0 | 0 | -18 | 1100000000000000 | 7000000000000000 |
Find the rows modified by Qc2 in the “POST” file:
kvalobs@dev-vm116:~/Qc2Config$ grep Qc2 POST-ALG004-20100319-24.dat
99754 | 2010-03-23 15:00:00 | -32767 | 211 | 2010-03-23 15:30:57 | 1 | 0 | 0 | -13.6 | 0000003100000000 | 9899900000000000 | Qc2 UnitT corrected was:-32767
17150 | 2010-03-23 13:00:00 | -32767 | 211 | 2010-03-23 13:30:11 | 342 | 0 | 0 | 4 | 0000003100000000 | 9899900000000000 | Qc2 UnitT corrected was:-32767
61060 | 2010-03-22 23:00:00 | -32767 | 211 | 2010-03-23 02:51:52 | 330 | 0 | 0 | 1.6 | 0000003100000000 | 9899900000000000 | Qc2 UnitT corrected was:-32767
Find the corresponding rows in the “POST” file:
kvalobs@dev-vm116:~/Qc2Config$ grep "99754 | 2010-03-23 15:00:00"
PRE-ALG004-20100319-24.dat
99754 | 2010-03-23 15:00:00 | -32767 | 211 | 2010-03-23 15:30:57 | 1 | 0 | 0 | -32767 | 0000003000000000 | 9899900000000000 |
kvalobs@dev-vm116:~/Qc2Config$ grep "17150 | 2010-03-23 13:00:00 |" PRE-ALG004-20100319-24.dat
17150 | 2010-03-23 13:00:00 | -32767 | 211 | 2010-03-2313:30:11 | 342 | 0 | 0 | -32767 | 0000003000000000 | 9899900000000000 |
kvalobs@dev-vm116:~/Qc2Config$ grep "61060 | 2010-03-22 23:00:00 |" PRE-ALG004-20100319-24.dat
61060 | 2010-03-22 23:00:00 | -32767 | 211 | 2010-03-23 02:51:52 | 330 | 0 | 0 | -32767 | 0000003000000000 | 9899900000000000 |
kvalobs@dev-vm116:~/Qc2Config$
Mmmmmmm … in this case Qc1 had not run on the data yet!
[APRIL 2010] Log of test run
| Date | Comment |
|---|---|
| fre apr 9 12:20:11 UTC 2010 | Qc2 turned off and removed from bin/kvstart. Qc1 set running |
| man apr 12 | Checked status. See below. |
- Disk Full The kvalobs SW had been updated on 2010-04-05 and this included a change in the log directory path leading to the qabase_cron.sh not able to delete the html logs. The paths were updated in the crontab scripts (see QC2-OPE-RUN).
- Not all Qc1 checks running Some of the Qc1 checks were not active … these were turned on. i.e. psql> UPDATE checks SET active='* * * * *';
| Date | Comment |
|---|---|
| man apr 12 19:54:20 UTC 2010 | Qc1 was turned back on |
kvalobs@dev-vm116:~$ bin/kvstart KVBIN=/metno/kvalobs/bin KVPID=/metno/kvalobs/var/run/kvalobs TIMEOUT=70 Starter kvalobs dette kan ta noe tid! Hvis det ikke skjer noe på MER enn 70 sekund bruk CTRL-C for å avbryte! Starter kvQabased ....Ok! Starter kvManagerd ....Ok! Starter kvDataInputd ....Ok! Starter kvServiced ....Ok! kvalobs@dev-vm116:~$ date man apr 12 19:54:20 UTC 2010