This is an old revision of the document!
Single Missing Point Revised
The tests at singlemissingpoint confirmed the basic performance of the interpolation. This page now looks at the revised specification and the flag handling that this implies.
Example Scenarios:
fmis=1
50540 | 2008-03-30 14:00:00 | 6.4 | 211 | 2008-03-30 13:54:58 | 330 | 0 | 0 | 6.4 | 1110100000000010 | 7000000000000000 | 50540 | 2008-03-30 15:00:00 | -32767 | 211 | 2008-03-30 15:30:04 | 3 | 0 | 0 | -32767 | 1000501000000000 | 7890700000000001 | QC1-4-211:1,QC1-4-211:1 50540 | 2008-03-30 15:00:00 | 7.3 | 211 | 2008-03-30 14:54:58 | 330 | 0 | 0 | 7.3 | 1110100000100010 | 7000000000000000 |
fmis=2 : no single point examples found
85891 | 2008-03-30 09:00:00 | 2.3 | 211 | 2008-03-30 08:55:28 | 330 | 0 | 0 | -32766 | 100060200000061A | 3038900000000FF2 | QC1-0-211:1,QC1-4-211:1, hqc
fmis=3
44630 | 2008-03-30 14:00:00 | 6 | 211 | 2008-03-30 14:02:52 | 3 | 0 | 0 | 6 | 1001100000000000 | 7000000000000000 | 44630 | 2008-03-30 14:00:00 | -32767 | 211 | 2008-03-30 14:02:53 | 3 | 0 | 10 | -32767 | 0000003000000000 | 9899900000000000 | 44630 | 2008-03-30 14:00:00 | 6.1 | 211 | 2008-03-30 14:02:52 | 3 | 1 | 10 | 6.1 | 0000000000000000 | 9999900000000000
fmis=4 no examples found for paramid=211, but here is something for 213:
33890 | 2006-12-17 23:00:00 | -48.2 | 213 | 2006-12-18 11:17:15 | 3 | 0 | 0 | -32766 | 15120040000000AA | 3138300400000032 | QC1-1-213:1,QC1-3a-213:1,QC1-9-213:1,hqc
TODO: check kvalobs db on other servers …
The revised algorithm also has to recheck values in case the neighbours have changed. In general terms the logic is as follows:
Test Run
kvalobs=# DELETE from DATA where stationid=76992 and obstime between '2010-07-29' and '2010-07-30' and paramid=211;
DELETE 25
kvalobs=# COPY data FROM '/metno/kvalobs/TESTDATA/kvqc2_1.0.4/DB-Input.dat' USING DELIMITERS ';' ;
COPY 25
kvalobs=# SELECT * FROM data WHERE stationid=76992 AND obstime BETWEEN '2010-07-29 14:00:00' AND '2010-07-29 16:00:00' AND paramid=211;
stationid | obstime | original | paramid | tbtime | typeid | sensor | level | corrected | controlinfo | useinfo | cfailed
-----------+---------------------+----------+---------+---------------------+--------+--------+-------+-----------+------------------+------------------+---------
76992 | 2010-07-29 14:00:00 | 2 | 211 | 2010-07-29 14:06:19 | 11 | 0 | 0 | 2 | 0111000000100000 | 7000000000000000 |
76992 | 2010-07-29 15:00:00 | -32767 | 211 | 2010-07-29 16:30:04 | 11 | 0 | 0 | -32767 | 0000003000000000 | 9899900000000000 |
76992 | 2010-07-29 16:00:00 | 2 | 211 | 2010-07-29 16:06:19 | 11 | 0 | 0 | 2 | 0111000000100000 | 7000000000000000 |
(3 rows)
kvalobs=# SELECT * FROM data WHERE stationid=76992 AND obstime BETWEEN '2010-07-29 14:00:00' AND '2010-07-29 16:00:00' AND paramid=211;
stationid | obstime | original | paramid | tbtime | typeid | sensor | level | corrected | controlinfo | useinfo | cfailed
-----------+---------------------+----------+---------+---------------------+--------+--------+-------+-----------+------------------+------------------+----------
76992 | 2010-07-29 14:00:00 | 2 | 211 | 2010-07-29 14:06:19 | 11 | 0 | 0 | 2 | 0111000000100000 | 7000000000000000 |
76992 | 2010-07-29 15:00:00 | -32767 | 211 | 2010-07-29 16:30:04 | 11 | 0 | 0 | 2 | 0000001100000000 | 6894900000000000 | QC2d-2
76992 | 2010-07-29 16:00:00 | 2 | 211 | 2010-07-29 16:06:19 | 11 | 0 | 0 | 2 | 0111000000100000 | 7000000000000000 |
(3 rows)
kvalobs=# UPDATE data SET corrected=10 WHERE stationid=76992 AND obstime BETWEEN '2010-07-29' AND '2010-07-30' AND paramid=211 AND original=-32767;
UPDATE 1
kvalobs=# SELECT * FROM data WHERE stationid=76992 AND obstime BETWEEN '2010-07-29 14:00:00' AND '2010-07-29 16:00:00' AND paramid=211; stationid | obstime | original | paramid | tbtime | typeid | sensor | level | corrected | controlinfo | useinfo | cfailed
-----------+---------------------+----------+---------+---------------------+--------+--------+-------+-----------+------------------+------------------+----------
76992 | 2010-07-29 14:00:00 | 2 | 211 | 2010-07-29 14:06:19 | 11 | 0 | 0 | 2 | 0111000000100000 | 7000000000000000 |
76992 | 2010-07-29 15:00:00 | -32767 | 211 | 2010-07-29 16:30:04 | 11 | 0 | 0 | 10 | 0000001100000000 | 6894900000000000 | QC2d-2
76992 | 2010-07-29 16:00:00 | 2 | 211 | 2010-07-29 16:06:19 | 11 | 0 | 0 | 2 | 0111000000100000 | 7000000000000000 |
(3 rows)
kvalobs=# SELECT * FROM data WHERE stationid=76992 AND obstime BETWEEN '2010-07-29 14:00:00' AND '2010-07-29 16:00:00' AND paramid=211;
stationid | obstime | original | paramid | tbtime | typeid | sensor | level | corrected | controlinfo | useinfo | cfailed
-----------+---------------------+----------+---------+---------------------+--------+--------+-------+-----------+------------------+------------------+------------------
76992 | 2010-07-29 14:00:00 | 2 | 211 | 2010-07-29 14:06:19 | 11 | 0 | 0 | 2 | 0111000000100000 | 7000000000000000 |
76992 | 2010-07-29 15:00:00 | -32767 | 211 | 2010-07-29 16:30:04 | 11 | 0 | 0 | 2 | 0000001100000000 | 6894900000000000 | QC2d-2 QC2d-2
76992 | 2010-07-29 16:00:00 | 2 | 211 | 2010-07-29 16:06:19 | 11 | 0 | 0 | 2 | 0111000000100000 | 7000000000000000 |
(3 rows)
kvalobs=# UPDATE data SET useinfo='7010000000000000' WHERE stationid=76992 and obstime='2010-07-29 14:00:00' and paramid=211;
UPDATE 1
kvalobs=# SELECT * FROM data WHERE stationid=76992 AND obstime BETWEEN '2010-07-29 14:00:00' AND '2010-07-29 16:00:00' AND paramid=211;
stationid | obstime | original | paramid | tbtime | typeid | sensor | level | corrected | controlinfo | useinfo | cfailed
-----------+---------------------+----------+---------+---------------------+--------+--------+-------+-----------+------------------+------------------+------------------
76992 | 2010-07-29 14:00:00 | 2 | 211 | 2010-07-29 14:06:19 | 11 | 0 | 0 | 2 | 0111000000100000 | 7010000000000000 |
76992 | 2010-07-29 15:00:00 | -32767 | 211 | 2010-07-29 16:30:04 | 11 | 0 | 0 | 2 | 0000001100000000 | 6894900000000000 | QC2d-2 QC2d-2
76992 | 2010-07-29 16:00:00 | 2 | 211 | 2010-07-29 16:06:19 | 11 | 0 | 0 | 2 | 0111000000100000 | 7000000000000000 |
(3 rows)
kvalobs=# SELECT * FROM data WHERE stationid=76992 AND obstime BETWEEN '2010-07-29 14:00:00' AND '2010-07-29 16:00:00' AND paramid=211;
stationid | obstime | original | paramid | tbtime | typeid | sensor | level | corrected | controlinfo | useinfo | cfailed
-----------+---------------------+----------+---------+---------------------+--------+--------+-------+-----------+------------------+------------------+--------------------------
76992 | 2010-07-29 14:00:00 | 2 | 211 | 2010-07-29 14:06:19 | 11 | 0 | 0 | 2 | 0111000000100000 | 7010000000000000 |
76992 | 2010-07-29 15:00:00 | -32767 | 211 | 2010-07-29 16:30:04 | 11 | 0 | 0 | -32767 | 0000003100000000 | 6899900000000000 | QC2d-2 QC2d-2 QC2d-2
76992 | 2010-07-29 16:00:00 | 2 | 211 | 2010-07-29 16:06:19 | 11 | 0 | 0 | 2 | 0111000000100000 | 7000000000000000 |
(3 rows)