====== Test Files ====== * [[.:test:perf|Results of performance monitoring tests]] * [[.:test:algorithms|Specific tests supporting new algorithm development]] * [[.:test:acceptance|Acceptance Test File]] * [[.:test:vm101notes|Installation and test on dev-vm101 notes]] * [[.:test:flagtests|Flag tests]] * [[.:test:SLv32|Single Linear version 3.2 test]] * [[.:test:trpt|Template for official test report]] [[.:test:trpt-en|EN]] * [[.:test:downstream|Tests to propagate data downstream]] ====== Case Studies ====== An interesting example to test in a space-time-qc. A sudden drop (10 degrees) in temp caused by front-passage gives error message in QC1. To be granted in QC2. {{kvalobs:kvoss:system:qc2:1292gartland140709kl14.png?400|1292gartland140709kl14}}|{{kvalobs:kvoss:system:qc2:1292gartland140709kl14lyn.png?400|1292gartland140709kl14lyn}} **Station Gartland, Trøndelag 14 july 2009** Corresponding {{kvalobs:kvoss:system:qc2:20090714event.doc|data}} series (two-days spanning). ====== Test Data ====== **Test data sets for populating test versions of the kvalobs database:** * {{kvoss:system:qc2:2007_paramid_110.doc|March-December, 2007, paramid=110}} **Note** the above is a plain text file. The .doc extension is used as it seems only a limited set of extensions can be uploaded to the wiki. (Now discovered that *.dat.gz is allowed but I have to leave the *docs in place until receive help from a system admin to remove the old files ... and then I can replace them with smaller *.dat.gz). The //text file// comprises data extracted from the operational or historic kvalobs database in the form of lines such as: 17741;2007-04-02 06:00:00;-1;110;2007-10-03 16:39:21;402;0;0;-1;1110000000001000;7000000000000000; and may be loaded into a test version of the kvalobs database with: $ psql kvalobs kvalobs=# COPY data FROM //The quoted ' ' Full-path-of-text-file// USING DELIMITERS ';'; FIXME ... NB some old data files had 'NULL' for cfailed where the filed was empty. Later found that it was cleaner to use just "", i.e. no string, since NULL value did not allow string concatenation. Later files uploaded to the wiki do not use null. **Some other reference data** * {{kvalobs:kvoss:system:qc2:2018-diptest.dat.gz|2018 Simulated DipTest test data.}} ====== Using generator.pl ====== Data Before: kvalobs=# select * from data where obstime>'2032-03-01 07:00:00' and obstime<'2032-03-10 07:00:00' and stationID=75410 and paramid=110 ; stationid | obstime | original | paramid | tbtime | typeid | sensor | level | corrected | controlinfo | useinfo | cfailed -----------+---------------------+----------+---------+---------------------+--------+--------+-------+-----------+------------------+------------------+--------- 75410 | 2032-03-02 06:00:00 | 4 | 110 | 2032-03-02 05:39:14 | -308 | 0 | 0 | 4 | 0000000000000000 | 9999900000000000 | 75410 | 2032-03-03 06:00:00 | 12.5 | 110 | 2032-03-03 05:44:06 | -308 | 0 | 0 | 12.5 | 0000000000000000 | 9999900000000000 | 75410 | 2032-03-04 06:00:00 | 2.4 | 110 | 2032-03-04 05:39:01 | -308 | 0 | 0 | 2.4 | 0000000000000000 | 9999900000000000 | 75410 | 2032-03-05 06:00:00 | 5.4 | 110 | 2032-03-05 06:44:20 | -308 | 0 | 0 | 3.4 | 0000000000000000 | 9999900000000000 | 75410 | 2032-03-06 06:00:00 | 10.7 | 110 | 2032-03-06 05:33:03 | -308 | 0 | 0 | 10.7 | 0000000000000000 | 9999900000000000 | 75410 | 2032-03-07 06:00:00 | 1.8 | 110 | 2032-03-07 05:36:53 | -308 | 0 | 0 | 1.8 | 0000000000000000 | 9999900000000000 | 75410 | 2032-03-08 06:00:00 | 0.6 | 110 | 2032-03-08 05:39:09 | -308 | 0 | 0 | 0.6 | 0000000000000000 | 9999900000000000 | 75410 | 2032-03-09 06:00:00 | 0.7 | 110 | 2032-03-09 05:45:07 | -308 | 0 | 0 | 0.7 | 0000000000000000 | 9999900000000000 | 75410 | 2032-03-10 06:00:00 | 0.3 | 110 | 2032-03-10 05:36:04 | -308 | 0 | 0 | 15.2 | 0000000000000000 | 9999900000000000 | (9 rows) Run generator.pl with the following paramters: kvalobs@pak:~/kvalobs-svn/src/kvQc2/Reference/TestDataGeneration$ ./generator.pl "2032-03-06 06:00:00" 4 75410 -308 110 This generates the folowing SQL statements that are applied to the kvalobs database to generate the test data (see below): SELECT * FROM data WHERE obstime='2032-03-06 06:00:00' AND stationid=75410 AND typeid=-308 AND paramid=110 AND original>-32767 AND (SELECT COUNT(original) FROM data WHERE obstime BETWEEN date '2032-03-06 06:00:00' - interval '3 days' AND '2032-03-06 06:00:00' AND stationid=75410 AND typeid=-308 AND paramid=110 AND original>-32767)=4; SELECT * FROM data WHERE obstime BETWEEN date '2032-03-06 06:00:00' - interval '3 days' AND '2032-03-06 06:00:00' AND stationid=75410 AND typeid=-308 AND paramid=110 AND original>-32767; UPDATE data SET original=((SELECT SUM(original) FROM data WHERE obstime BETWEEN date '2032-03-06 06:00:00' - interval '3 days' AND '2032-03-06 06:00:00' AND stationid=75410 AND typeid=-308 AND paramid=110 AND original>-32767)+(SELECT COUNT(original) FROM data WHERE obstime BETWEEN date '2032-03-06 06:00:00' - interval '3 days' AND '2032-03-06 06:00:00' AND stationid=75410 AND typeid=-308 AND paramid=110 AND original=-1)) WHERE obstime='2032-03-06 06:00:00' AND stationid=75410 AND typeid=-308 AND paramid=110 AND original>-32767 AND (SELECT COUNT(original) FROM data WHERE obstime BETWEEN date '2032-03-06 06:00:00' - interval '3 days' AND '2032-03-06 06:00:00' AND stationid=75410 AND typeid=-308 AND paramid=110 AND original>-32767)=4; UPDATE data SET controlinfo='9999999999992990' WHERE obstime='2032-03-06 06:00:00' AND stationid=75410 AND typeid=-308 AND paramid=110 AND original>-32767 AND (SELECT COUNT(original) FROM data WHERE obstime BETWEEN date '2032-03-06 06:00:00' - interval '3 days' AND '2032-03-06 06:00:00' AND stationid=75410 AND typeid=-308 AND paramid=110 AND original>-32767)=4; UPDATE data SET controlinfo='9999999999992990' WHERE obstime BETWEEN date '2032-03-06 06:00:00' -interval '1 days' AND date '2032-03-06 06:00:00' -interval '0 days' AND stationid=75410 AND typeid=-308 AND paramid=110 AND original>-32767 AND (SELECT COUNT(original) FROM data WHERE obstime BETWEEN date '2032-03-06 06:00:00' - interval '3 days' AND '2032-03-06 06:00:00' AND stationid=75410 AND typeid=-308 AND paramid=110 AND original>-32767)=4; UPDATE data SET original=-11111 WHERE obstime BETWEEN date '2032-03-06 06:00:00' -interval '1 days' AND date '2032-03-06 06:00:00' -interval '0 days' AND stationid=75410 AND typeid=-308 AND paramid=110 AND original>-32767 AND (SELECT COUNT(original) FROM data WHERE obstime BETWEEN date '2032-03-06 06:00:00' - interval '3 days' AND '2032-03-06 06:00:00' AND stationid=75410 AND typeid=-308 AND paramid=110 AND original>-32767)=4; UPDATE data SET controlinfo='9999999999992990' WHERE obstime BETWEEN date '2032-03-06 06:00:00' -interval '2 days' AND date '2032-03-06 06:00:00' -interval '1 days' AND stationid=75410 AND typeid=-308 AND paramid=110 AND original>-32767 AND (SELECT COUNT(original) FROM data WHERE obstime BETWEEN date '2032-03-06 06:00:00' - interval '3 days' AND '2032-03-06 06:00:00' AND stationid=75410 AND typeid=-308 AND paramid=110 AND original>-32767)=4; UPDATE data SET original=-11111 WHERE obstime BETWEEN date '2032-03-06 06:00:00' -interval '2 days' AND date '2032-03-06 06:00:00' -interval '1 days' AND stationid=75410 AND typeid=-308 AND paramid=110 AND original>-32767 AND (SELECT COUNT(original) FROM data WHERE obstime BETWEEN date '2032-03-06 06:00:00' - interval '3 days' AND '2032-03-06 06:00:00' AND stationid=75410 AND typeid=-308 AND paramid=110 AND original>-32767)=4; UPDATE data SET controlinfo='9999999999992990' WHERE obstime BETWEEN date '2032-03-06 06:00:00' -interval '3 days' AND date '2032-03-06 06:00:00' -interval '2 days' AND stationid=75410 AND typeid=-308 AND paramid=110 AND original>-32767 AND (SELECT COUNT(original) FROM data WHERE obstime BETWEEN date '2032-03-06 06:00:00' - interval '3 days' AND '2032-03-06 06:00:00' AND stationid=75410 AND typeid=-308 AND paramid=110 AND original>-32767)=4; UPDATE data SET original=-11111 WHERE obstime BETWEEN date '2032-03-06 06:00:00' -interval '3 days' AND date '2032-03-06 06:00:00' -interval '2 days' AND stationid=75410 AND typeid=-308 AND paramid=110 AND original>-32767 AND (SELECT COUNT(original) FROM data WHERE obstime BETWEEN date '2032-03-06 06:00:00' - interval '3 days' AND '2032-03-06 06:00:00' AND stationid=75410 AND typeid=-308 AND paramid=110 AND original>-32767)=4; SELECT * FROM data WHERE obstime='2032-03-06 06:00:00' AND stationid=75410 AND typeid=-308 AND paramid=110 AND original>-32767 AND (SELECT COUNT(original) FROM data WHERE obstime BETWEEN date '2032-03-06 06:00:00' - interval '3 days' AND '2032-03-06 06:00:00' AND stationid=75410 AND typeid=-308 AND paramid=110 AND original>-32767)=4; SELECT * FROM data WHERE obstime BETWEEN date '2032-03-06 06:00:00' - interval '3 days' AND '2032-03-06 06:00:00' AND stationid=75410 AND typeid=-308 AND paramid=110 AND original>-32767; The following data is created: kvalobs=# select * from data where obstime>'2032-03-01 07:00:00' and obstime<'2032-03-10 07:00:00' and stationID=75410 and paramid=110; stationid | obstime | original | paramid | tbtime | typeid | sensor | level | corrected | controlinfo | useinfo | cfailed -----------+---------------------+----------+---------+---------------------+--------+--------+-------+-----------+------------------+------------------+--------- 75410 | 2032-03-02 06:00:00 | 4 | 110 | 2032-03-02 05:39:14 | -308 | 0 | 0 | 4 | 0000000000000000 | 9999900000000000 | 75410 | 2032-03-03 06:00:00 | -11111 | 110 | 2032-03-03 05:44:06 | -308 | 0 | 0 | 12.5 | 9999999999992990 | 9999900000000000 | 75410 | 2032-03-04 06:00:00 | -11111 | 110 | 2032-03-04 05:39:01 | -308 | 0 | 0 | 2.4 | 9999999999992990 | 9999900000000000 | 75410 | 2032-03-05 06:00:00 | -11111 | 110 | 2032-03-05 06:44:20 | -308 | 0 | 0 | 3.4 | 9999999999992990 | 9999900000000000 | 75410 | 2032-03-06 06:00:00 | 31 | 110 | 2032-03-06 05:33:03 | -308 | 0 | 0 | 10.7 | 9999999999992990 | 9999900000000000 | 75410 | 2032-03-07 06:00:00 | 1.8 | 110 | 2032-03-07 05:36:53 | -308 | 0 | 0 | 1.8 | 0000000000000000 | 9999900000000000 | 75410 | 2032-03-08 06:00:00 | 0.6 | 110 | 2032-03-08 05:39:09 | -308 | 0 | 0 | 0.6 | 0000000000000000 | 9999900000000000 | 75410 | 2032-03-09 06:00:00 | 0.7 | 110 | 2032-03-09 05:45:07 | -308 | 0 | 0 | 0.7 | 0000000000000000 | 9999900000000000 | 75410 | 2032-03-10 06:00:00 | 0.3 | 110 | 2032-03-10 05:36:04 | -308 | 0 | 0 | 15.2 | 0000000000000000 | 9999900000000000 | (9 rows) Finally replace "-11111" with a missing value. Scripts work .. but need to be automated more. Another example after more work on the test data generator.pl script: **Before** 58390 | 2032-03-02 06:00:00 | 0.3 | 110 | 2032-03-02 06:25:19 | 302 | 0 | 0 | 0.3 | 1110000000001001 | 3000000000000070 | 58390 | 2032-03-03 06:00:00 | 4.5 | 110 | 2032-03-03 06:10:32 | 302 | 0 | 0 | 4.5 | 1110000000001001 | 3000000000000070 | 58390 | 2032-03-04 06:00:00 | 2.5 | 110 | 2032-03-04 06:10:54 | 302 | 0 | 0 | 2.5 | 1110000000001000 | 7000000000000000 | 58390 | 2032-03-05 06:00:00 | 8.5 | 110 | 2032-03-05 06:25:24 | 302 | 0 | 0 | 8.5 | 1110000000001000 | 7000000000000000 | 58390 | 2032-03-06 06:00:00 | 7 | 110 | 2032-03-06 06:25:24 | 302 | 0 | 0 | 7 | 1110000000001000 | 7000000000000000 | 58390 | 2032-03-07 06:00:00 | 0.2 | 110 | 2032-03-07 06:25:21 | 302 | 0 | 0 | 0.2 | 1110000000001000 | 7000000000000000 | 58390 | 2032-03-08 06:00:00 | -1 | 110 | 2032-03-08 06:25:17 | 302 | 0 | 0 | -1 | 1110000000001000 | 7000000000000000 | 58390 | 2032-03-09 06:00:00 | 2.4 | 110 | 2032-03-09 06:06:31 | 302 | 0 | 0 | 2.4 | 1110000000001000 | 7000000000000000 | 58390 | 2032-03-10 06:00:00 | 2.2 | 110 | 2032-03-10 06:25:20 | 302 | 0 | 0 | 2.2 | 1110000000001001 | 3000000000000070 | **Command** ./generator.pl "2032-03-09 06:00:00" 7 58390 302 110 **After** 58390 | 2032-03-02 06:00:00 | 0.3 | 110 | 2032-03-02 06:25:19 | 302 | 0 | 0 | 0.3 | 1110000000001001 | 3000000000000070 | 58390 | 2032-03-03 06:00:00 | -32767 | 110 | 2032-03-03 06:10:32 | 302 | 0 | 0 | 4.5 | 9999999999992990 | 3000000000000070 | TEST-DATA-RA_RR24 original=4.5 58390 | 2032-03-04 06:00:00 | -32767 | 110 | 2032-03-04 06:10:54 | 302 | 0 | 0 | 2.5 | 9999999999992990 | 7000000000000000 | TEST-DATA-RA_RR24 original=2.5 58390 | 2032-03-05 06:00:00 | -32767 | 110 | 2032-03-05 06:25:24 | 302 | 0 | 0 | 8.5 | 9999999999992990 | 7000000000000000 | TEST-DATA-RA_RR24 original=8.5 58390 | 2032-03-06 06:00:00 | -32767 | 110 | 2032-03-06 06:25:24 | 302 | 0 | 0 | 7 | 9999999999992990 | 7000000000000000 | TEST-DATA-RA_RR24 original=7 58390 | 2032-03-07 06:00:00 | -32767 | 110 | 2032-03-07 06:25:21 | 302 | 0 | 0 | 0.2 | 9999999999992990 | 7000000000000000 | TEST-DATA-RA_RR24 original=0.2 58390 | 2032-03-08 06:00:00 | -32767 | 110 | 2032-03-08 06:25:17 | 302 | 0 | 0 | -1 | 9999999999992990 | 7000000000000000 | TEST-DATA-RA_RR24 original=-1 58390 | 2032-03-09 06:00:00 | 25.1 | 110 | 2032-03-09 06:06:31 | 302 | 0 | 0 | 2.4 | 9999999999992990 | 7000000000000000 | TEST-DATA-RA_RR24 original=2.4 58390 | 2032-03-10 06:00:00 | 2.2 | 110 | 2032-03-10 06:25:20 | 302 | 0 | 0 | 2.2 | 1110000000001001 | 3000000000000070 | Another tweak of the generator.pl script: BEFORE 59250 | 2032-03-03 06:00:00 | 6 | 110 | 2032-03-03 08:55:29 | 302 | 0 | 0 | 6 | 1110000000001000 | 7000000000000000 | 59250 | 2032-03-04 06:00:00 | 2.2 | 110 | 2032-03-04 14:56:27 | 302 | 0 | 0 | 2.2 | 1110000000001000 | 7000000000000000 | 59250 | 2032-03-05 06:00:00 | 1.4 | 110 | 2032-03-05 07:10:47 | 302 | 0 | 0 | 1.4 | 1110000000001000 | 7000000000000000 | 59250 | 2032-03-06 06:00:00 | 10.1 | 110 | 2032-03-06 07:10:50 | 302 | 0 | 0 | 10.1 | 1110000000001000 | 7000000000000000 | Command ./generator.pl "2032-03-06 06:00:00" 4 59250 302 110 AFTER 59250 | 2032-03-03 06:00:00 | -32767 | 110 | 2032-03-03 08:55:29 | 302 | 0 | 0 | 6 | 1110000000002000 | 7000000000000000 | TEST-DATA-RA_RR24 original=6 59250 | 2032-03-04 06:00:00 | -32767 | 110 | 2032-03-04 14:56:27 | 302 | 0 | 0 | 2.2 | 1110000000002000 | 7000000000000000 | TEST-DATA-RA_RR24 original=2.2 59250 | 2032-03-05 06:00:00 | -32767 | 110 | 2032-03-05 07:10:47 | 302 | 0 | 0 | 1.4 | 1110000000002000 | 7000000000000000 | TEST-DATA-RA_RR24 original=1.4 59250 | 2032-03-06 06:00:00 | 19.7 | 110 | 2032-03-06 07:10:50 | 302 | 0 | 0 | 10.1 | 1140000000002000 | 7000000000000000 | TEST-DATA-RA_RR24 original=10.1 What is this data all about? 18500 | 2032-03-01 14:00:00 | 22.3 | 110 | 2032-03-01 13:50:08 | 342 | 0 | 0 | 22.3 | 1100000000000000 | 7000000000000000 | 18500 | 2032-03-01 15:00:00 | 22.2 | 110 | 2032-03-01 14:50:05 | 342 | 0 | 0 | 22.2 | 1100000000000000 | 7000000000000000 | 18500 | 2032-03-01 16:00:00 | 22.5 | 110 | 2032-03-01 15:50:15 | 342 | 0 | 0 | 22.5 | 1100000000000000 | 7000000000000000 | 18500 | 2032-03-01 17:00:00 | 22.6 | 110 | 2032-03-01 16:50:08 | 342 | 0 | 0 | 22.6 | 1100000000000000 | 7000000000000000 | 18500 | 2032-03-01 18:00:00 | 22.6 | 110 | 2032-03-01 17:50:09 | 342 | 0 | 0 | 22.6 | 1100000000000000 | 7000000000000000 | 18500 | 2032-03-01 19:00:00 | 22.6 | 110 | 2032-03-01 18:50:11 | 342 | 0 | 0 | 22.6 | 1100000000000000 | 7000000000000000 | 18500 | 2032-03-01 20:00:00 | 21.7 | 110 | 2032-03-01 19:50:04 | 342 | 0 | 0 | 21.7 | 1100000000000000 | 7000000000000000 | 18500 | 2032-03-01 21:00:00 | 19.9 | 110 | 2032-03-01 20:50:10 | 342 | 0 | 0 | 19.9 | 1100000000000000 | 7000000000000000 |A ===== Some recent results from dev-vm116 April 2010 ===== kvalobs=# select * from data WHERE obstime>'2010-03-28' and obstime<'2010-04-10' and cfailed like '%Qc2%' and paramid=110; stationid | obstime | original | paramid | tbtime | typeid | sensor | level | corrected | controlinfo | useinfo | cfailed -----------+---------------------+----------+---------+---------------------+--------+--------+-------+-----------+------------------+------------------+-------------------------------------------------------- 22951 | 2010-03-28 06:00:00 | 0.6 | 110 | 2010-03-28 16:30:35 | 302 | 0 | 0 | 0.1 | 1100000000007000 | 7336900000000000 | QC1-7-110:1 Qc2 Redis corrected was:0.6 53950 | 2010-03-28 06:00:00 | -32767 | 110 | 2010-03-29 00:30:36 | 302 | 0 | 0 | -1 | 1000003000007000 | 7899900000000000 | QC1-7-110:1 Qc2 Redis corrected was:-32767 53950 | 2010-03-29 06:00:00 | -1 | 110 | 2010-03-29 11:41:24 | 302 | 0 | 0 | -1 | 1100000000007000 | 7336900000000000 | QC1-7-110:1 Qc2 Redis corrected was:-1 53730 | 2010-03-28 06:00:00 | -32767 | 110 | 2010-03-29 00:30:33 | 302 | 0 | 0 | -1 | 1000003000007000 | 7899900000000000 | QC1-7-110:1 Qc2 Redis corrected was:-32767 53730 | 2010-03-29 06:00:00 | -1 | 110 | 2010-03-29 11:40:23 | 302 | 0 | 0 | -1 | 1100000000007000 | 7336900000000000 | QC1-7-110:1 Qc2 Redis corrected was:-1 58480 | 2010-03-28 06:00:00 | -32767 | 110 | 2010-03-29 00:30:37 | 302 | 0 | 0 | 0.8 | 1000003000007000 | 7899900000000000 | QC1-7-110:1 Qc2 Redis corrected was:-32767,QC1-7-110:1 46700 | 2010-03-28 06:00:00 | -32767 | 110 | 2010-03-29 00:30:24 | 302 | 0 | 0 | 3.1 | 1000003000007000 | 7899900000000000 | QC1-7-110:1 Qc2 Redis corrected was:-32767 46700 | 2010-03-29 06:00:00 | 10 | 110 | 2010-03-29 06:44:24 | 302 | 0 | 0 | 5.7 | 1100000000007000 | 7336900000000000 | QC1-7-110:1 Qc2 Redis corrected was:10 81370 | 2010-03-28 06:00:00 | -32767 | 110 | 2010-03-29 00:30:41 | 302 | 0 | 0 | -1 | 1000003000007000 | 7899900000000000 | QC1-7-110:1 Qc2 Redis corrected was:-32767 81370 | 2010-03-29 06:00:00 | -1 | 110 | 2010-03-29 05:55:21 | 302 | 0 | 0 | -1 | 1100000000007000 | 7336900000000000 | QC1-7-110:1 Qc2 Redis corrected was:-1 55930 | 2010-03-28 06:00:00 | -32767 | 110 | 2010-03-29 00:30:38 | 302 | 0 | 0 | 1.2 | 1000003000007000 | 7899900000000000 | QC1-7-110:1 Qc2 Redis corrected was:-32767 55930 | 2010-03-29 06:00:00 | 5.7 | 110 | 2010-03-29 05:55:21 | 302 | 0 | 0 | 2.6 | 1100000000007000 | 7336900000000000 | QC1-7-110:1 Qc2 Redis corrected was:5.7 48780 | 2010-03-28 06:00:00 | -32767 | 110 | 2010-03-29 00:30:34 | 302 | 0 | 0 | 1.2 | 1000003000007000 | 7899900000000000 | QC1-7-110:1 Qc2 Redis corrected was:-32767 48780 | 2010-03-29 06:00:00 | 3.7 | 110 | 2010-03-29 05:48:46 | 302 | 0 | 0 | 1.8 | 1100000000007000 | 7336900000000000 | QC1-7-110:1 Qc2 Redis corrected was:3.7 47240 | 2010-03-28 06:00:00 | -32767 | 110 | 2010-03-29 00:30:28 | 302 | 0 | 0 | 0.9 | 1000003000007000 | 7899900000000000 | QC1-7-110:1 Qc2 Redis corrected was:-32767 47240 | 2010-03-29 06:00:00 | 4.5 | 110 | 2010-03-29 05:27:24 | 302 | 0 | 0 | 3.2 | 1100000000007000 | 7336900000000000 | QC1-7-110:1 Qc2 Redis corrected was:4.5 92750 | 2010-03-28 06:00:00 | -32767 | 110 | 2010-03-29 00:30:51 | 302 | 0 | 0 | 0 | 1000003000007000 | 7899900000000000 | QC1-7-110:1 Qc2 Redis corrected was:-32767 92750 | 2010-03-29 06:00:00 | 0.1 | 110 | 2010-03-29 06:10:28 | 302 | 0 | 0 | 0.1 | 1100000000007000 | 7336900000000000 | QC1-7-110:1 Qc2 Redis corrected was:0.1 22951 | 2010-03-29 06:00:00 | -32767 | 110 | 2010-03-30 00:30:11 | 302 | 0 | 0 | 0.1 | 1000003000007000 | 7899900000000000 | QC1-7-110:1 Qc2 Redis corrected was:-32767 22951 | 2010-03-30 06:00:00 | -32767 | 110 | 2010-03-31 00:30:08 | 302 | 0 | 0 | 1.5 | 1000003000007000 | 7899900000000000 | QC1-7-110:1 Qc2 Redis corrected was:-32767 22951 | 2010-03-31 06:00:00 | -32767 | 110 | 2010-04-01 00:30:15 | 302 | 0 | 0 | 5.3 | 1000003000007000 | 7899900000000000 | QC1-7-110:1 Qc2 Redis corrected was:-32767 22951 | 2010-04-01 06:00:00 | 22.3 | 110 | 2010-04-01 09:26:50 | 302 | 0 | 0 | 15.4 | 1100000000007000 | 7336900000000000 | QC1-7-110:1 Qc2 Redis corrected was:22.3 (22 rows) ==== ... ordered by station ==== kvalobs=# kvalobs=# kvalobs=# kvalobs=# select * from data WHERE obstime>'2010-03-20' and obstime<'2010-04-10' and stationid=22951 and paramid=110 order by obstime; stationid | obstime | original | paramid | tbtime | typeid | sensor | level | corrected | controlinfo | useinfo | cfailed -----------+---------------------+----------+---------+---------------------+--------+--------+-------+-----------+------------------+------------------+-------------------------------------------------------- 22951 | 2010-03-20 06:00:00 | -32767 | 110 | 2010-03-23 10:06:31 | 302 | 0 | 0 | -32767 | 1000003000002000 | 7899900000000000 | QC1-7-110:1,QC1-7-110:1 22951 | 2010-03-21 06:00:00 | -32767 | 110 | 2010-03-23 05:46:14 | 302 | 0 | 0 | 5.1 | 1000003000007000 | 7899900000000000 | QC1-7-110:1,QC1-7-110:1 Qc2 Redis corrected was:-32767 22951 | 2010-03-22 06:00:00 | -32767 | 110 | 2010-03-23 02:27:12 | 302 | 0 | 0 | 2.9 | 1000003000007000 | 7899900000000000 | QC1-7-110:1,QC1-7-110:1 Qc2 Redis corrected was:-32767 22951 | 2010-03-23 06:00:00 | 11.6 | 110 | 2010-03-23 15:23:15 | 302 | 0 | 0 | 3.6 | 1100000000007000 | 7336900000000000 | QC1-7-110:1 Qc2 Redis corrected was:11.6 22951 | 2010-03-24 06:00:00 | -32767 | 110 | 2010-03-25 00:30:04 | 302 | 0 | 0 | 0.1 | 1000003000007000 | 7899900000000000 | QC1-7-110:1 Qc2 Redis corrected was:-32767 22951 | 2010-03-25 06:00:00 | -32767 | 110 | 2010-03-26 00:30:08 | 302 | 0 | 0 | 0.1 | 1000003000007000 | 7899900000000000 | QC1-7-110:1 Qc2 Redis corrected was:-32767 22951 | 2010-03-26 06:00:00 | -32767 | 110 | 2010-03-27 00:30:20 | 302 | 0 | 0 | 0 | 1000003000007000 | 7899900000000000 | QC1-7-110:1 Qc2 Redis corrected was:-32767 22951 | 2010-03-27 06:00:00 | -32767 | 110 | 2010-03-28 00:30:15 | 302 | 0 | 0 | 0.3 | 1000003000007000 | 7899900000000000 | QC1-7-110:1 Qc2 Redis corrected was:-32767 22951 | 2010-03-28 06:00:00 | 0.6 | 110 | 2010-03-28 16:30:35 | 302 | 0 | 0 | 0.1 | 1100000000007000 | 7336900000000000 | QC1-7-110:1 Qc2 Redis corrected was:0.6 22951 | 2010-03-29 06:00:00 | -32767 | 110 | 2010-03-30 00:30:11 | 302 | 0 | 0 | 0.1 | 1000003000007000 | 7899900000000000 | QC1-7-110:1 Qc2 Redis corrected was:-32767 22951 | 2010-03-30 06:00:00 | -32767 | 110 | 2010-03-31 00:30:08 | 302 | 0 | 0 | 1.5 | 1000003000007000 | 7899900000000000 | QC1-7-110:1 Qc2 Redis corrected was:-32767 22951 | 2010-03-31 06:00:00 | -32767 | 110 | 2010-04-01 00:30:15 | 302 | 0 | 0 | 5.3 | 1000003000007000 | 7899900000000000 | QC1-7-110:1 Qc2 Redis corrected was:-32767 22951 | 2010-04-01 06:00:00 | 22.3 | 110 | 2010-04-01 09:26:50 | 302 | 0 | 0 | 15.4 | 1100000000007000 | 7336900000000000 | QC1-7-110:1 Qc2 Redis corrected was:22.3 22951 | 2010-04-02 06:00:00 | 3.9 | 110 | 2010-04-02 10:38:17 | 302 | 0 | 0 | 3.9 | 1100000000001000 | 7000000000000000 | 22951 | 2010-04-03 06:00:00 | 1.6 | 110 | 2010-04-03 19:17:31 | 302 | 0 | 0 | 1.6 | 0000000000000000 | 9999900000000000 | (15 rows) kvalobs=# select * from data WHERE obstime>'2010-03-20' and obstime<'2010-04-10' and stationid=53950 and paramid=110 order by obstime; stationid | obstime | original | paramid | tbtime | typeid | sensor | level | corrected | controlinfo | useinfo | cfailed -----------+---------------------+----------+---------+---------------------+--------+--------+-------+-----------+------------------+------------------+-------------------------------------------- 53950 | 2010-03-26 06:00:00 | 0.6 | 110 | 2010-03-26 08:11:36 | 302 | 0 | 0 | 0.6 | 1100000000001000 | 7000000000000000 | 53950 | 2010-03-27 06:00:00 | -32767 | 110 | 2010-03-28 00:30:26 | 302 | 0 | 0 | -1 | 1000003000007000 | 7899900000000000 | QC1-7-110:1 Qc2 Redis corrected was:-32767 53950 | 2010-03-28 06:00:00 | -32767 | 110 | 2010-03-29 00:30:36 | 302 | 0 | 0 | -1 | 1000003000007000 | 7899900000000000 | QC1-7-110:1 Qc2 Redis corrected was:-32767 53950 | 2010-03-29 06:00:00 | -1 | 110 | 2010-03-29 11:41:24 | 302 | 0 | 0 | -1 | 1100000000007000 | 7336900000000000 | QC1-7-110:1 Qc2 Redis corrected was:-1 53950 | 2010-03-30 06:00:00 | 0.7 | 110 | 2010-03-30 08:57:13 | 302 | 0 | 0 | 0.7 | 1100000000001000 | 7000000000000000 | kvalobs=# select * from data WHERE obstime>'2010-03-20' and obstime<'2010-04-10' and stationid=53730 and paramid=110 order by obstime; stationid | obstime | original | paramid | tbtime | typeid | sensor | level | corrected | controlinfo | useinfo | cfailed -----------+---------------------+----------+---------+---------------------+--------+--------+-------+-----------+------------------+------------------+-------------------------------------------- 53730 | 2010-03-26 06:00:00 | 1.5 | 110 | 2010-03-26 08:10:30 | 302 | 0 | 0 | 1.5 | 1100000000001000 | 7000000000000000 | 53730 | 2010-03-27 06:00:00 | -32767 | 110 | 2010-03-28 00:30:28 | 302 | 0 | 0 | -1 | 1000003000007000 | 7899900000000000 | QC1-7-110:1 Qc2 Redis corrected was:-32767 53730 | 2010-03-28 06:00:00 | -32767 | 110 | 2010-03-29 00:30:33 | 302 | 0 | 0 | -1 | 1000003000007000 | 7899900000000000 | QC1-7-110:1 Qc2 Redis corrected was:-32767 53730 | 2010-03-29 06:00:00 | -1 | 110 | 2010-03-29 11:40:23 | 302 | 0 | 0 | -1 | 1100000000007000 | 7336900000000000 | QC1-7-110:1 Qc2 Redis corrected was:-1 53730 | 2010-03-30 06:00:00 | 2 | 110 | 2010-03-30 08:57:13 | 302 | 0 | 0 | 2 | 1100000000001000 | 7000000000000000 | kvalobs=# select * from data WHERE obstime>'2010-03-20' and obstime<'2010-04-10' and stationid=58480 and paramid=110 order by obstime; stationid | obstime | original | paramid | tbtime | typeid | sensor | level | corrected | controlinfo | useinfo | cfailed -----------+---------------------+----------+---------+---------------------+--------+--------+-------+-----------+------------------+------------------+-------------------------------------------------------- 58480 | 2010-03-26 06:00:00 | 3.7 | 110 | 2010-03-26 06:15:11 | 302 | 0 | 0 | 3.7 | 1100000000001000 | 7000000000000000 | 58480 | 2010-03-27 06:00:00 | -32767 | 110 | 2010-03-28 00:30:29 | 302 | 0 | 0 | 1.2 | 1000003000007000 | 7899900000000000 | QC1-7-110:1 Qc2 Redis corrected was:-32767,QC1-7-110:1 58480 | 2010-03-28 06:00:00 | -32767 | 110 | 2010-03-29 00:30:37 | 302 | 0 | 0 | 0.8 | 1000003000007000 | 7899900000000000 | QC1-7-110:1 Qc2 Redis corrected was:-32767,QC1-7-110:1 58480 | 2010-03-29 06:00:00 | 4.6 | 110 | 2010-03-30 07:35:12 | 302 | 0 | 0 | 4.6 | 1100000000002000 | 7330900000000000 | QC1-7-110:1 58480 | 2010-03-30 06:00:00 | 4 | 110 | 2010-04-02 02:15:55 | 302 | 0 | 0 | 4 | 1100000000001000 | 7000000000000000 | kvalobs=# select * from data WHERE obstime>'2010-03-20' and obstime<'2010-04-10' and stationid=46700 and paramid=110 order by obstime; stationid | obstime | original | paramid | tbtime | typeid | sensor | level | corrected | controlinfo | useinfo | cfailed -----------+---------------------+----------+---------+---------------------+--------+--------+-------+-----------+------------------+------------------+-------------------------------------------- 46700 | 2010-03-26 06:00:00 | 1 | 110 | 2010-03-26 07:17:55 | 302 | 0 | 0 | 1 | 1100000000001000 | 7000000000000000 | 46700 | 2010-03-27 06:00:00 | -32767 | 110 | 2010-03-28 00:30:17 | 302 | 0 | 0 | 1.2 | 1000003000007000 | 7899900000000000 | QC1-7-110:1 Qc2 Redis corrected was:-32767 46700 | 2010-03-28 06:00:00 | -32767 | 110 | 2010-03-29 00:30:24 | 302 | 0 | 0 | 3.1 | 1000003000007000 | 7899900000000000 | QC1-7-110:1 Qc2 Redis corrected was:-32767 46700 | 2010-03-29 06:00:00 | 10 | 110 | 2010-03-29 06:44:24 | 302 | 0 | 0 | 5.7 | 1100000000007000 | 7336900000000000 | QC1-7-110:1 Qc2 Redis corrected was:10 46700 | 2010-03-30 06:00:00 | 7 | 110 | 2010-03-30 06:50:39 | 302 | 0 | 0 | 7 | 1100000000001000 | 7000000000000000 | kvalobs=# select * from data WHERE obstime>'2010-03-20' and obstime<'2010-04-10' and stationid=81370 and paramid=110 order by obstime; stationid | obstime | original | paramid | tbtime | typeid | sensor | level | corrected | controlinfo | useinfo | cfailed -----------+---------------------+----------+---------+---------------------+--------+--------+-------+-----------+------------------+------------------+-------------------------------------------- 81370 | 2010-03-26 06:00:00 | 0.8 | 110 | 2010-03-26 06:52:41 | 302 | 0 | 0 | 0.8 | 1100000000001000 | 7000000000000000 | 81370 | 2010-03-27 06:00:00 | -32767 | 110 | 2010-03-28 00:30:40 | 302 | 0 | 0 | -1 | 1000003000007000 | 7899900000000000 | QC1-7-110:1 Qc2 Redis corrected was:-32767 81370 | 2010-03-28 06:00:00 | -32767 | 110 | 2010-03-29 00:30:41 | 302 | 0 | 0 | -1 | 1000003000007000 | 7899900000000000 | QC1-7-110:1 Qc2 Redis corrected was:-32767 81370 | 2010-03-29 06:00:00 | -1 | 110 | 2010-03-29 05:55:21 | 302 | 0 | 0 | -1 | 1100000000007000 | 7336900000000000 | QC1-7-110:1 Qc2 Redis corrected was:-1 81370 | 2010-03-30 06:00:00 | -32767 | 110 | 2010-03-31 00:30:26 | 302 | 0 | 0 | -32767 | 1000003000002000 | 7899900000000000 | QC1-7-110:1 kvalobs=# select * from data WHERE obstime>'2010-03-20' and obstime<'2010-04-10' and stationid=55930 and paramid=110 order by obstime; stationid | obstime | original | paramid | tbtime | typeid | sensor | level | corrected | controlinfo | useinfo | cfailed -----------+---------------------+----------+---------+---------------------+--------+--------+-------+-----------+------------------+------------------+-------------------------------------------- 55930 | 2010-03-26 06:00:00 | 4.8 | 110 | 2010-03-26 09:11:16 | 302 | 0 | 0 | 4.8 | 1100000000001000 | 7000000000000000 | 55930 | 2010-03-27 06:00:00 | -32767 | 110 | 2010-03-28 00:30:32 | 302 | 0 | 0 | 2 | 1000003000007000 | 7899900000000000 | QC1-7-110:1 Qc2 Redis corrected was:-32767 55930 | 2010-03-28 06:00:00 | -32767 | 110 | 2010-03-29 00:30:38 | 302 | 0 | 0 | 1.2 | 1000003000007000 | 7899900000000000 | QC1-7-110:1 Qc2 Redis corrected was:-32767 55930 | 2010-03-29 06:00:00 | 5.7 | 110 | 2010-03-29 05:55:21 | 302 | 0 | 0 | 2.6 | 1100000000007000 | 7336900000000000 | QC1-7-110:1 Qc2 Redis corrected was:5.7 55930 | 2010-03-30 06:00:00 | -1 | 110 | 2010-03-30 05:55:17 | 302 | 0 | 0 | -1 | 1100000000001000 | 7000000000000000 | kvalobs=# select * from data WHERE obstime>'2010-03-20' and obstime<'2010-04-10' and stationid=48780 and paramid=110 order by obstime; stationid | obstime | original | paramid | tbtime | typeid | sensor | level | corrected | controlinfo | useinfo | cfailed -----------+---------------------+----------+---------+---------------------+--------+--------+-------+-----------+------------------+------------------+-------------------------------------------- 48780 | 2010-03-26 06:00:00 | -1 | 110 | 2010-03-26 07:45:35 | 302 | 0 | 0 | -1 | 1100000000001000 | 7000000000000000 | 48780 | 2010-03-27 06:00:00 | -32767 | 110 | 2010-03-28 00:30:28 | 302 | 0 | 0 | 0.7 | 1000003000007000 | 7899900000000000 | QC1-7-110:1 Qc2 Redis corrected was:-32767 48780 | 2010-03-28 06:00:00 | -32767 | 110 | 2010-03-29 00:30:34 | 302 | 0 | 0 | 1.2 | 1000003000007000 | 7899900000000000 | QC1-7-110:1 Qc2 Redis corrected was:-32767 48780 | 2010-03-29 06:00:00 | 3.7 | 110 | 2010-03-29 05:48:46 | 302 | 0 | 0 | 1.8 | 1100000000007000 | 7336900000000000 | QC1-7-110:1 Qc2 Redis corrected was:3.7 48780 | 2010-03-30 06:00:00 | 5.3 | 110 | 2010-03-30 06:44:38 | 302 | 0 | 0 | 5.3 | 1100000000001000 | 7000000000000000 | kvalobs=# select * from data WHERE obstime>'2010-03-20' and obstime<'2010-04-10' and stationid=47240 and paramid=110 order by obstime; stationid | obstime | original | paramid | tbtime | typeid | sensor | level | corrected | controlinfo | useinfo | cfailed -----------+---------------------+----------+---------+---------------------+--------+--------+-------+-----------+------------------+------------------+-------------------------------------------- 47240 | 2010-03-26 06:00:00 | 2 | 110 | 2010-03-26 06:43:30 | 302 | 0 | 0 | 2 | 1100000000001000 | 7000000000000000 | 47240 | 2010-03-27 06:00:00 | -32767 | 110 | 2010-03-28 00:30:26 | 302 | 0 | 0 | 0.4 | 1000003000007000 | 7899900000000000 | QC1-7-110:1 Qc2 Redis corrected was:-32767 47240 | 2010-03-28 06:00:00 | -32767 | 110 | 2010-03-29 00:30:28 | 302 | 0 | 0 | 0.9 | 1000003000007000 | 7899900000000000 | QC1-7-110:1 Qc2 Redis corrected was:-32767 47240 | 2010-03-29 06:00:00 | 4.5 | 110 | 2010-03-29 05:27:24 | 302 | 0 | 0 | 3.2 | 1100000000007000 | 7336900000000000 | QC1-7-110:1 Qc2 Redis corrected was:4.5 47240 | 2010-03-30 06:00:00 | 0.7 | 110 | 2010-03-30 05:50:44 | 302 | 0 | 0 | 0.7 | 1100000000001000 | 7000000000000000 | kvalobs=# select * from data WHERE obstime>'2010-03-20' and obstime<'2010-04-10' and stationid=92750 and paramid=110 order by obstime; stationid | obstime | original | paramid | tbtime | typeid | sensor | level | corrected | controlinfo | useinfo | cfailed -----------+---------------------+----------+---------+---------------------+--------+--------+-------+-----------+------------------+------------------+-------------------------------------------- 92750 | 2010-03-26 06:00:00 | 5.1 | 110 | 2010-03-26 07:23:01 | 302 | 0 | 0 | 5.1 | 1100000000001000 | 7000000000000000 | 92750 | 2010-03-27 06:00:00 | -32767 | 110 | 2010-03-28 00:30:47 | 302 | 0 | 0 | 0 | 1000003000007000 | 7899900000000000 | QC1-7-110:1 Qc2 Redis corrected was:-32767 92750 | 2010-03-28 06:00:00 | -32767 | 110 | 2010-03-29 00:30:51 | 302 | 0 | 0 | 0 | 1000003000007000 | 7899900000000000 | QC1-7-110:1 Qc2 Redis corrected was:-32767 92750 | 2010-03-29 06:00:00 | 0.1 | 110 | 2010-03-29 06:10:28 | 302 | 0 | 0 | 0.1 | 1100000000007000 | 7336900000000000 | QC1-7-110:1 Qc2 Redis corrected was:0.1 92750 | 2010-03-30 06:00:00 | 0.1 | 110 | 2010-03-30 06:11:33 | 302 | 0 | 0 | 0.1 | 1100000000001000 | 7000000000000000 | kvalobs=#