Differences
This shows you the differences between two versions of the page.
Both sides previous revision Previous revision Next revision | Previous revision | ||
kvalobs:kvoss:system:qc2:test [2009-12-19 13:41:16] paule |
kvalobs:kvoss:system:qc2:test [2022-05-31 09:29:32] (current) |
||
---|---|---|---|
Line 1: | Line 1: | ||
+ | |||
+ | ====== Test Files ====== | ||
+ | |||
+ | * [[.: | ||
+ | * [[.: | ||
+ | * [[.: | ||
+ | * [[.: | ||
+ | * [[.: | ||
+ | * [[.: | ||
+ | * [[.: | ||
+ | * [[.: | ||
====== Case Studies ====== | ====== Case Studies ====== | ||
An interesting example to test in a space-time-qc. A sudden drop (10 degrees) in temp caused by front-passage gives error message in QC1. To be granted in QC2. | An interesting example to test in a space-time-qc. A sudden drop (10 degrees) in temp caused by front-passage gives error message in QC1. To be granted in QC2. | ||
- | {{: | + | {{kvalobs: |
**Station Gartland, Trøndelag 14 july 2009** | **Station Gartland, Trøndelag 14 july 2009** | ||
- | Corresponding {{: | + | Corresponding {{kvalobs: |
- | + | ||
- | ====== Test Files ====== | + | |
- | * ... | ||
- | * ... | ||
- | * ... | ||
====== Test Data ====== | ====== Test Data ====== | ||
Line 22: | Line 28: | ||
* {{kvoss: | * {{kvoss: | ||
- | **Note** the above is a plain text file. The .doc extension is used since only a limited set of extensions can be uploaded to the wiki FIXME. The //text file// comprises data extracted from the operational or historic kvalobs database in the form of lines such as: | + | **Note** the above is a plain text file. The .doc extension is used as it seems only a limited set of extensions can be uploaded to the wiki. (Now discovered that *.dat.gz is allowed but I have to leave the *docs in place until receive help from a system admin to remove the old files ... and then I can replace them with smaller *.dat.gz). |
+ | |||
+ | The //text file// comprises data extracted from the operational or historic kvalobs database in the form of lines such as: | ||
< | < | ||
- | 17741; | + | 17741; |
- | </ | + | </ |
and may be loaded into a test version of the kvalobs database with: | and may be loaded into a test version of the kvalobs database with: | ||
< | < | ||
$ psql kvalobs | $ psql kvalobs | ||
- | kvalobs=# COPY data FROM //The quoted ' ' Full-path-of-text-file// | + | kvalobs=# COPY data FROM //The quoted ' ' Full-path-of-text-file// |
</ | </ | ||
+ | |||
+ | FIXME ... NB some old data files had ' | ||
+ | |||
+ | **Some other reference data** | ||
+ | |||
+ | * {{kvalobs: | ||
====== Using generator.pl ====== | ====== Using generator.pl ====== | ||
Line 116: | Line 130: | ||
< | < | ||
- | + | 58390 | 2032-03-02 06:00:00 | 0.3 | 110 | 2032-03-02 06:25:19 | 302 | 0 | 0 | 0.3 | 1110000000001001 | 3000000000000070 | | |
+ | 58390 | 2032-03-03 06:00:00 | 4.5 | 110 | 2032-03-03 06:10:32 | 302 | 0 | 0 | 4.5 | 1110000000001001 | 3000000000000070 | | ||
+ | 58390 | 2032-03-04 06:00:00 | 2.5 | 110 | 2032-03-04 06:10:54 | 302 | 0 | 0 | 2.5 | 1110000000001000 | 7000000000000000 | | ||
+ | 58390 | 2032-03-05 06:00:00 | 8.5 | 110 | 2032-03-05 06:25:24 | 302 | 0 | 0 | 8.5 | 1110000000001000 | 7000000000000000 | | ||
+ | 58390 | 2032-03-06 06:00:00 | 7 | 110 | 2032-03-06 06:25:24 | 302 | 0 | 0 | 7 | 1110000000001000 | 7000000000000000 | | ||
+ | 58390 | 2032-03-07 06:00:00 | 0.2 | 110 | 2032-03-07 06:25:21 | 302 | 0 | 0 | 0.2 | 1110000000001000 | 7000000000000000 | | ||
+ | 58390 | 2032-03-08 06:00:00 | -1 | 110 | 2032-03-08 06:25:17 | 302 | 0 | 0 | -1 | 1110000000001000 | 7000000000000000 | | ||
+ | 58390 | 2032-03-09 06:00:00 | 2.4 | 110 | 2032-03-09 06:06:31 | 302 | 0 | 0 | 2.4 | 1110000000001000 | 7000000000000000 | | ||
+ | 58390 | 2032-03-10 06:00:00 | 2.2 | 110 | 2032-03-10 06:25:20 | 302 | 0 | 0 | 2.2 | 1110000000001001 | 3000000000000070 | | ||
</ | </ | ||
**Command** | **Command** | ||
< | < | ||
+ | ./ | ||
</ | </ | ||
Line 127: | Line 150: | ||
**After** | **After** | ||
< | < | ||
+ | 58390 | 2032-03-02 06:00:00 | 0.3 | 110 | 2032-03-02 06:25:19 | 302 | 0 | 0 | 0.3 | 1110000000001001 | 3000000000000070 | | ||
+ | 58390 | 2032-03-03 06:00:00 | | ||
+ | 58390 | 2032-03-04 06:00:00 | | ||
+ | 58390 | 2032-03-05 06:00:00 | | ||
+ | 58390 | 2032-03-06 06:00:00 | | ||
+ | 58390 | 2032-03-07 06:00:00 | | ||
+ | 58390 | 2032-03-08 06:00:00 | | ||
+ | 58390 | 2032-03-09 06:00:00 | 25.1 | 110 | 2032-03-09 06:06:31 | 302 | 0 | 0 | 2.4 | 9999999999992990 | 7000000000000000 | TEST-DATA-RA_RR24 original=2.4 | ||
+ | 58390 | 2032-03-10 06:00:00 | 2.2 | 110 | 2032-03-10 06:25:20 | 302 | 0 | 0 | 2.2 | 1110000000001001 | 3000000000000070 | | ||
+ | |||
+ | </ | ||
+ | |||
+ | Another tweak of the generator.pl script: | ||
+ | |||
+ | BEFORE | ||
+ | < | ||
+ | 59250 | 2032-03-03 06:00:00 | 6 | 110 | 2032-03-03 08:55:29 | 302 | 0 | 0 | 6 | 1110000000001000 | 7000000000000000 | | ||
+ | 59250 | 2032-03-04 06:00:00 | 2.2 | 110 | 2032-03-04 14:56:27 | 302 | 0 | 0 | 2.2 | 1110000000001000 | 7000000000000000 | | ||
+ | 59250 | 2032-03-05 06:00:00 | 1.4 | 110 | 2032-03-05 07:10:47 | 302 | 0 | 0 | 1.4 | 1110000000001000 | 7000000000000000 | | ||
+ | 59250 | 2032-03-06 06:00:00 | 10.1 | 110 | 2032-03-06 07:10:50 | 302 | 0 | 0 | 10.1 | 1110000000001000 | 7000000000000000 | | ||
+ | </ | ||
+ | |||
+ | Command | ||
+ | < | ||
+ | ./ | ||
+ | </ | ||
+ | |||
+ | AFTER | ||
+ | < | ||
+ | 59250 | 2032-03-03 06:00:00 | | ||
+ | 59250 | 2032-03-04 06:00:00 | | ||
+ | 59250 | 2032-03-05 06:00:00 | | ||
+ | 59250 | 2032-03-06 06:00:00 | 19.7 | 110 | 2032-03-06 07:10:50 | 302 | 0 | 0 | 10.1 | 1140000000002000 | 7000000000000000 | TEST-DATA-RA_RR24 original=10.1 | ||
</ | </ | ||
Line 143: | Line 199: | ||
18500 | 2032-03-01 20:00:00 | 21.7 | 110 | 2032-03-01 19:50:04 | 342 | 0 | 0 | 21.7 | 1100000000000000 | 7000000000000000 | | 18500 | 2032-03-01 20:00:00 | 21.7 | 110 | 2032-03-01 19:50:04 | 342 | 0 | 0 | 21.7 | 1100000000000000 | 7000000000000000 | | ||
18500 | 2032-03-01 21:00:00 | 19.9 | 110 | 2032-03-01 20:50:10 | 342 | 0 | 0 | 19.9 | 1100000000000000 | 7000000000000000 |A | 18500 | 2032-03-01 21:00:00 | 19.9 | 110 | 2032-03-01 20:50:10 | 342 | 0 | 0 | 19.9 | 1100000000000000 | 7000000000000000 |A | ||
+ | </ | ||
+ | |||
+ | ===== Some recent results from dev-vm116 April 2010 ===== | ||
+ | |||
+ | < | ||
+ | kvalobs=# select * from data WHERE obstime>' | ||
+ | | ||
+ | -----------+---------------------+----------+---------+---------------------+--------+--------+-------+-----------+------------------+------------------+-------------------------------------------------------- | ||
+ | 22951 | 2010-03-28 06:00:00 | 0.6 | 110 | 2010-03-28 16:30:35 | 302 | 0 | 0 | 0.1 | 1100000000007000 | 7336900000000000 | QC1-7-110:1 Qc2 Redis corrected was:0.6 | ||
+ | 53950 | 2010-03-28 06:00:00 | | ||
+ | 53950 | 2010-03-29 06:00:00 | -1 | 110 | 2010-03-29 11:41:24 | 302 | 0 | 0 | -1 | 1100000000007000 | 7336900000000000 | QC1-7-110:1 Qc2 Redis corrected was:-1 | ||
+ | 53730 | 2010-03-28 06:00:00 | | ||
+ | 53730 | 2010-03-29 06:00:00 | -1 | 110 | 2010-03-29 11:40:23 | 302 | 0 | 0 | -1 | 1100000000007000 | 7336900000000000 | QC1-7-110:1 Qc2 Redis corrected was:-1 | ||
+ | 58480 | 2010-03-28 06:00:00 | | ||
+ | 46700 | 2010-03-28 06:00:00 | | ||
+ | 46700 | 2010-03-29 06:00:00 | 10 | 110 | 2010-03-29 06:44:24 | 302 | 0 | 0 | 5.7 | 1100000000007000 | 7336900000000000 | QC1-7-110:1 Qc2 Redis corrected was:10 | ||
+ | 81370 | 2010-03-28 06:00:00 | | ||
+ | 81370 | 2010-03-29 06:00:00 | -1 | 110 | 2010-03-29 05:55:21 | 302 | 0 | 0 | -1 | 1100000000007000 | 7336900000000000 | QC1-7-110:1 Qc2 Redis corrected was:-1 | ||
+ | 55930 | 2010-03-28 06:00:00 | | ||
+ | 55930 | 2010-03-29 06:00:00 | 5.7 | 110 | 2010-03-29 05:55:21 | 302 | 0 | 0 | 2.6 | 1100000000007000 | 7336900000000000 | QC1-7-110:1 Qc2 Redis corrected was:5.7 | ||
+ | 48780 | 2010-03-28 06:00:00 | | ||
+ | 48780 | 2010-03-29 06:00:00 | 3.7 | 110 | 2010-03-29 05:48:46 | 302 | 0 | 0 | 1.8 | 1100000000007000 | 7336900000000000 | QC1-7-110:1 Qc2 Redis corrected was:3.7 | ||
+ | 47240 | 2010-03-28 06:00:00 | | ||
+ | 47240 | 2010-03-29 06:00:00 | 4.5 | 110 | 2010-03-29 05:27:24 | 302 | 0 | 0 | 3.2 | 1100000000007000 | 7336900000000000 | QC1-7-110:1 Qc2 Redis corrected was:4.5 | ||
+ | 92750 | 2010-03-28 06:00:00 | | ||
+ | 92750 | 2010-03-29 06:00:00 | 0.1 | 110 | 2010-03-29 06:10:28 | 302 | 0 | 0 | 0.1 | 1100000000007000 | 7336900000000000 | QC1-7-110:1 Qc2 Redis corrected was:0.1 | ||
+ | 22951 | 2010-03-29 06:00:00 | | ||
+ | 22951 | 2010-03-30 06:00:00 | | ||
+ | 22951 | 2010-03-31 06:00:00 | | ||
+ | 22951 | 2010-04-01 06:00:00 | 22.3 | 110 | 2010-04-01 09:26:50 | 302 | 0 | 0 | 15.4 | 1100000000007000 | 7336900000000000 | QC1-7-110:1 Qc2 Redis corrected was:22.3 | ||
+ | (22 rows) | ||
+ | |||
+ | </ | ||
+ | |||
+ | |||
+ | ==== ... ordered by station ==== | ||
+ | |||
+ | < | ||
+ | kvalobs=# | ||
+ | kvalobs=# | ||
+ | kvalobs=# | ||
+ | kvalobs=# select * from data WHERE obstime>' | ||
+ | | ||
+ | -----------+---------------------+----------+---------+---------------------+--------+--------+-------+-----------+------------------+------------------+-------------------------------------------------------- | ||
+ | 22951 | 2010-03-20 06:00:00 | | ||
+ | 22951 | 2010-03-21 06:00:00 | | ||
+ | 22951 | 2010-03-22 06:00:00 | | ||
+ | 22951 | 2010-03-23 06:00:00 | 11.6 | 110 | 2010-03-23 15:23:15 | 302 | 0 | 0 | 3.6 | 1100000000007000 | 7336900000000000 | QC1-7-110:1 Qc2 Redis corrected was:11.6 | ||
+ | 22951 | 2010-03-24 06:00:00 | | ||
+ | 22951 | 2010-03-25 06:00:00 | | ||
+ | 22951 | 2010-03-26 06:00:00 | | ||
+ | 22951 | 2010-03-27 06:00:00 | | ||
+ | 22951 | 2010-03-28 06:00:00 | 0.6 | 110 | 2010-03-28 16:30:35 | 302 | 0 | 0 | 0.1 | 1100000000007000 | 7336900000000000 | QC1-7-110:1 Qc2 Redis corrected was:0.6 | ||
+ | 22951 | 2010-03-29 06:00:00 | | ||
+ | 22951 | 2010-03-30 06:00:00 | | ||
+ | 22951 | 2010-03-31 06:00:00 | | ||
+ | 22951 | 2010-04-01 06:00:00 | 22.3 | 110 | 2010-04-01 09:26:50 | 302 | 0 | 0 | 15.4 | 1100000000007000 | 7336900000000000 | QC1-7-110:1 Qc2 Redis corrected was:22.3 | ||
+ | 22951 | 2010-04-02 06:00:00 | 3.9 | 110 | 2010-04-02 10:38:17 | 302 | 0 | 0 | 3.9 | 1100000000001000 | 7000000000000000 | | ||
+ | 22951 | 2010-04-03 06:00:00 | 1.6 | 110 | 2010-04-03 19:17:31 | 302 | 0 | 0 | 1.6 | 0000000000000000 | 9999900000000000 | | ||
+ | (15 rows) | ||
+ | |||
+ | kvalobs=# select * from data WHERE obstime>' | ||
+ | | ||
+ | -----------+---------------------+----------+---------+---------------------+--------+--------+-------+-----------+------------------+------------------+-------------------------------------------- | ||
+ | 53950 | 2010-03-26 06:00:00 | 0.6 | 110 | 2010-03-26 08:11:36 | 302 | 0 | 0 | 0.6 | 1100000000001000 | 7000000000000000 | | ||
+ | 53950 | 2010-03-27 06:00:00 | | ||
+ | 53950 | 2010-03-28 06:00:00 | | ||
+ | 53950 | 2010-03-29 06:00:00 | -1 | 110 | 2010-03-29 11:41:24 | 302 | 0 | 0 | -1 | 1100000000007000 | 7336900000000000 | QC1-7-110:1 Qc2 Redis corrected was:-1 | ||
+ | 53950 | 2010-03-30 06:00:00 | 0.7 | 110 | 2010-03-30 08:57:13 | 302 | 0 | 0 | 0.7 | 1100000000001000 | 7000000000000000 | | ||
+ | |||
+ | |||
+ | kvalobs=# select * from data WHERE obstime>' | ||
+ | | ||
+ | -----------+---------------------+----------+---------+---------------------+--------+--------+-------+-----------+------------------+------------------+-------------------------------------------- | ||
+ | 53730 | 2010-03-26 06:00:00 | 1.5 | 110 | 2010-03-26 08:10:30 | 302 | 0 | 0 | 1.5 | 1100000000001000 | 7000000000000000 | | ||
+ | 53730 | 2010-03-27 06:00:00 | | ||
+ | 53730 | 2010-03-28 06:00:00 | | ||
+ | 53730 | 2010-03-29 06:00:00 | -1 | 110 | 2010-03-29 11:40:23 | 302 | 0 | 0 | -1 | 1100000000007000 | 7336900000000000 | QC1-7-110:1 Qc2 Redis corrected was:-1 | ||
+ | 53730 | 2010-03-30 06:00:00 | 2 | 110 | 2010-03-30 08:57:13 | 302 | 0 | 0 | 2 | 1100000000001000 | 7000000000000000 | | ||
+ | |||
+ | kvalobs=# select * from data WHERE obstime>' | ||
+ | | ||
+ | -----------+---------------------+----------+---------+---------------------+--------+--------+-------+-----------+------------------+------------------+-------------------------------------------------------- | ||
+ | 58480 | 2010-03-26 06:00:00 | 3.7 | 110 | 2010-03-26 06:15:11 | 302 | 0 | 0 | 3.7 | 1100000000001000 | 7000000000000000 | | ||
+ | 58480 | 2010-03-27 06:00:00 | | ||
+ | 58480 | 2010-03-28 06:00:00 | | ||
+ | 58480 | 2010-03-29 06:00:00 | 4.6 | 110 | 2010-03-30 07:35:12 | 302 | 0 | 0 | 4.6 | 1100000000002000 | 7330900000000000 | QC1-7-110:1 | ||
+ | 58480 | 2010-03-30 06:00:00 | 4 | 110 | 2010-04-02 02:15:55 | 302 | 0 | 0 | 4 | 1100000000001000 | 7000000000000000 | | ||
+ | |||
+ | kvalobs=# select * from data WHERE obstime>' | ||
+ | | ||
+ | -----------+---------------------+----------+---------+---------------------+--------+--------+-------+-----------+------------------+------------------+-------------------------------------------- | ||
+ | 46700 | 2010-03-26 06:00:00 | 1 | 110 | 2010-03-26 07:17:55 | 302 | 0 | 0 | 1 | 1100000000001000 | 7000000000000000 | | ||
+ | 46700 | 2010-03-27 06:00:00 | | ||
+ | 46700 | 2010-03-28 06:00:00 | | ||
+ | 46700 | 2010-03-29 06:00:00 | 10 | 110 | 2010-03-29 06:44:24 | 302 | 0 | 0 | 5.7 | 1100000000007000 | 7336900000000000 | QC1-7-110:1 Qc2 Redis corrected was:10 | ||
+ | 46700 | 2010-03-30 06:00:00 | 7 | 110 | 2010-03-30 06:50:39 | 302 | 0 | 0 | 7 | 1100000000001000 | 7000000000000000 | | ||
+ | |||
+ | kvalobs=# select * from data WHERE obstime>' | ||
+ | | ||
+ | -----------+---------------------+----------+---------+---------------------+--------+--------+-------+-----------+------------------+------------------+-------------------------------------------- | ||
+ | 81370 | 2010-03-26 06:00:00 | 0.8 | 110 | 2010-03-26 06:52:41 | 302 | 0 | 0 | 0.8 | 1100000000001000 | 7000000000000000 | | ||
+ | 81370 | 2010-03-27 06:00:00 | | ||
+ | 81370 | 2010-03-28 06:00:00 | | ||
+ | 81370 | 2010-03-29 06:00:00 | -1 | 110 | 2010-03-29 05:55:21 | 302 | 0 | 0 | -1 | 1100000000007000 | 7336900000000000 | QC1-7-110:1 Qc2 Redis corrected was:-1 | ||
+ | 81370 | 2010-03-30 06:00:00 | | ||
+ | |||
+ | |||
+ | kvalobs=# select * from data WHERE obstime>' | ||
+ | | ||
+ | -----------+---------------------+----------+---------+---------------------+--------+--------+-------+-----------+------------------+------------------+-------------------------------------------- | ||
+ | 55930 | 2010-03-26 06:00:00 | 4.8 | 110 | 2010-03-26 09:11:16 | 302 | 0 | 0 | 4.8 | 1100000000001000 | 7000000000000000 | | ||
+ | 55930 | 2010-03-27 06:00:00 | | ||
+ | 55930 | 2010-03-28 06:00:00 | | ||
+ | 55930 | 2010-03-29 06:00:00 | 5.7 | 110 | 2010-03-29 05:55:21 | 302 | 0 | 0 | 2.6 | 1100000000007000 | 7336900000000000 | QC1-7-110:1 Qc2 Redis corrected was:5.7 | ||
+ | 55930 | 2010-03-30 06:00:00 | -1 | 110 | 2010-03-30 05:55:17 | 302 | 0 | 0 | -1 | 1100000000001000 | 7000000000000000 | | ||
+ | |||
+ | kvalobs=# select * from data WHERE obstime>' | ||
+ | | ||
+ | -----------+---------------------+----------+---------+---------------------+--------+--------+-------+-----------+------------------+------------------+-------------------------------------------- | ||
+ | 48780 | 2010-03-26 06:00:00 | -1 | 110 | 2010-03-26 07:45:35 | 302 | 0 | 0 | -1 | 1100000000001000 | 7000000000000000 | | ||
+ | 48780 | 2010-03-27 06:00:00 | | ||
+ | 48780 | 2010-03-28 06:00:00 | | ||
+ | 48780 | 2010-03-29 06:00:00 | 3.7 | 110 | 2010-03-29 05:48:46 | 302 | 0 | 0 | 1.8 | 1100000000007000 | 7336900000000000 | QC1-7-110:1 Qc2 Redis corrected was:3.7 | ||
+ | 48780 | 2010-03-30 06:00:00 | 5.3 | 110 | 2010-03-30 06:44:38 | 302 | 0 | 0 | 5.3 | 1100000000001000 | 7000000000000000 | | ||
+ | |||
+ | kvalobs=# select * from data WHERE obstime>' | ||
+ | | ||
+ | -----------+---------------------+----------+---------+---------------------+--------+--------+-------+-----------+------------------+------------------+-------------------------------------------- | ||
+ | 47240 | 2010-03-26 06:00:00 | 2 | 110 | 2010-03-26 06:43:30 | 302 | 0 | 0 | 2 | 1100000000001000 | 7000000000000000 | | ||
+ | 47240 | 2010-03-27 06:00:00 | | ||
+ | 47240 | 2010-03-28 06:00:00 | | ||
+ | 47240 | 2010-03-29 06:00:00 | 4.5 | 110 | 2010-03-29 05:27:24 | 302 | 0 | 0 | 3.2 | 1100000000007000 | 7336900000000000 | QC1-7-110:1 Qc2 Redis corrected was:4.5 | ||
+ | 47240 | 2010-03-30 06:00:00 | 0.7 | 110 | 2010-03-30 05:50:44 | 302 | 0 | 0 | 0.7 | 1100000000001000 | 7000000000000000 | | ||
+ | |||
+ | kvalobs=# select * from data WHERE obstime>' | ||
+ | | ||
+ | -----------+---------------------+----------+---------+---------------------+--------+--------+-------+-----------+------------------+------------------+-------------------------------------------- | ||
+ | 92750 | 2010-03-26 06:00:00 | 5.1 | 110 | 2010-03-26 07:23:01 | 302 | 0 | 0 | 5.1 | 1100000000001000 | 7000000000000000 | | ||
+ | 92750 | 2010-03-27 06:00:00 | | ||
+ | 92750 | 2010-03-28 06:00:00 | | ||
+ | 92750 | 2010-03-29 06:00:00 | 0.1 | 110 | 2010-03-29 06:10:28 | 302 | 0 | 0 | 0.1 | 1100000000007000 | 7336900000000000 | QC1-7-110:1 Qc2 Redis corrected was:0.1 | ||
+ | 92750 | 2010-03-30 06:00:00 | 0.1 | 110 | 2010-03-30 06:11:33 | 302 | 0 | 0 | 0.1 | 1100000000001000 | 7000000000000000 | | ||
+ | |||
+ | kvalobs=# | ||
</ | </ |