kvalobs:kvoss:system:qc2:test:acceptance:alg001

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision 		Previous revision
kvalobs:kvoss:system:qc2:test:acceptance:alg001 [2010-03-21 20:27:18]
paule 		kvalobs:kvoss:system:qc2:test:acceptance:alg001 [2022-05-31 09:29:32] (current)
Line 1: Line 1:
 ====== ALG001 ====== ====== ALG001 ======
  
-Working ... FIXME 
  
 ===== Acceptance Test for Redistribution of 24 hour accumulated precipitation ===== ===== Acceptance Test for Redistribution of 24 hour accumulated precipitation =====
Line 9: Line 8:
 | |$psql kvalobs | -- | -- | | |$psql kvalobs | -- | -- |
 | |psql> DELETE FROM data WHERE obstime>'2027-01-01' and obstime<'2028-01-01';| -- | -- | | |psql> DELETE FROM data WHERE obstime>'2027-01-01' and obstime<'2028-01-01';| -- | -- |
-|002 |Load the test data: | -- | -- |+|002 |Load the test data, e.g.: | -- | -- | 
 + 
 +{{kvalobs:kvoss:system:qc2:test:acceptance:2027processedtestdata.dat.gz|test data}} 
 | |psql> \COPY data FROM '/metno/kvalobs/TESTDATA/2027processedtestdata.dat' WITH DELIMITER AS ';' | -- | -- | | |psql> \COPY data FROM '/metno/kvalobs/TESTDATA/2027processedtestdata.dat' WITH DELIMITER AS ';' | -- | -- |
 | |Check the copy is successful: | -- | -- | | |Check the copy is successful: | -- | -- |
 | | select count(*) from data WHERE obstime>'2027-01-01' and obstime<'2028-01-01'; | count=124515 | -- | | | select count(*) from data WHERE obstime>'2027-01-01' and obstime<'2028-01-01'; | count=124515 | -- |
-|003 | -- | -- | -- | +|003 | Download the {{kvalobs:kvoss:system:qc2:test:acceptance:alg001-qcd2027.cfg.gz|Configuration File}} and edit the run time as required to run the algorithm. | -- | -- | 
-|004 | -- | -- | -- | + 
-|005 | -- | -- | -- | + 
-|006 | -- | -- | -- | +Also edit the config file so that it contains the following text:   
-|007 | -- | -- | -- |+<code> 
 +... 
 +# Flags to set on successful completion 
 +S_fd=7 
 +change_fmis=3->1 
 +change_fmis=0->4 
 + 
 +... 
 +</code> 
 +(NB this is a recent update) 
 + 
 +|004 |Look for results prior to running the control: | -- | -- | 
 +| |select count(*) from data where obstime>'2027-01-01' and obstime<'2028-01-01' and cfailed like '%TEST%Qc2%'; | count=0 | -- | 
 +|005 | Start the kvQc2 daemon and monitor until the algorithm run is complete. | -- | -- | 
 +| |select count(*) from data where obstime>'2027-01-01' and obstime<'2028-01-01' and cfailed like '%TEST%Qc2%'; | count=172 | -- | 
 +<code>select * from data where obstime>'2027-01-01' and obstime<'2028-01-01' and cfailed like '%TEST%Qc2%' \g | cat >> ALG001.dat</code> 
 +|006 | Download the following processing and plotting scripts {{kvalobs:kvoss:system:qc2:test:acceptance:alg001_plot.bash.gz|alg001_plot.bash}}, {{kvalobs:kvoss:system:qc2:test:acceptance:binplot.r.gz|binplot.R}} and {{kvalobs:kvoss:system:qc2:test:acceptance:plot_xy_analysis.r.gz|plot_xy_analysis.r}}| -- | -- | 
 +|007 | ./alg001_plot.bash ALG001.dat | {{kvalobs:kvoss:system:qc2:test:acceptance:alg001.dat.xy.dat.jpg?100x100|}} | -- | 
 +| | If files are downloaded from dokit: Please note dokit changes uploaded files to lower case ... an R script, example.R, becomes example.r Also need to "chmod +x" for scripts downloaded from dokit. | -- | -- | 
 +| | On dev-vm116, R is installed but "Rscript" is not. Best to copy data to a machine where R is installed fully. | -- | -- |
 |008 | -- | -- | -- | |008 | -- | -- | -- |
 |009 | -- | -- | -- | |009 | -- | -- | -- |
 |010 | -- | -- | -- | |010 | -- | -- | -- |
 +
 +Alternative configuration file for performing triangulation interpolation: {{kvalobs:kvoss:system:qc2:test:acceptance:alg001-qcd2027_triangle.cfg.gz|}}
 +
 +
 +===== Notes =====
  
 **Prerequisites** **Prerequisites**
Line 68: Line 94:
 </code> </code>
  
-{{:kvoss:system:qc2:test:acceptance:2027.doc|}}+{{kvalobs:kvoss:system:qc2:test:acceptance:2027.doc|}}
  
-{{:kvoss:system:qc2:test:acceptance:2027processedtestdata.doc|}}+{{kvalobs:kvoss:system:qc2:test:acceptance:2027processedtestdata.dat.gz|}}
  
 +NB 
 +<code>
 +$ cksum 2027processedtestdata.dat.gz
 +1877035137 1486134 2027processedtestdata.dat.gz
 +</code>
  
-{{:kvoss:system:qc2:test:acceptance:ra24-results-idw.jpg|}}+{{kvalobs:kvoss:system:qc2:test:acceptance:ra24-results-idw.jpg|}}
  
-{{:kvoss:system:qc2:test:acceptance:ra24-results-triangle.jpg|}}+{{kvalobs:kvoss:system:qc2:test:acceptance:ra24-results-triangle.jpg|}}
  
 generator.pl generator.pl
Line 160: Line 191:
 </code> </code>
  
-{{:kvoss:system:qc2:test:acceptance:gencom2027.doc|List of generator.pl calls to generate the test data.}}+{{kvalobs:kvoss:system:qc2:test:acceptance:gencom2027.doc|List of generator.pl calls to generate the test data.}}
  
  
-====== EDIT ======+====== Further Investigations ======
  
-Add single use case examples for using generator.pl+It is possible to use the script generator.pl modify some real data and substitute missing values and an accummulated value, while also storing the original values. (NB copy and paste generator.pl from the listing above). 
 +^Step ^Direction ^Expected Result ^Pass/Fail Criteria or Comment ^ 
 +|I |Select candidate data. One needs a start time and a number of days interval. | -- | -- | 
 +|II |Run "generator.pl" on the interval: | -- | -- | 
 +| |     ./generator.pl \"YYYY-MM-DD 06:00:00\" NUMBER_MISSING_POINTS StationID TypeID ParamID| -- | -- | 
 +| |Repeat as many times as required to build test data. | -- | -- | 
 +|III |Run the algorithm and analyse the results. | -- | -- |
  
 +
 +
 +===== For example: =====
 +
 +
 +**Before**
 +
 +<code>
 +     58390 | 2032-03-02 06:00:00 |      0.3 |     110 | 2032-03-02 06:25:19 |    302 | 0      |     0 |       0.3 | 1110000000001001 | 3000000000000070 | 
 +     58390 | 2032-03-03 06:00:00 |      4.5 |     110 | 2032-03-03 06:10:32 |    302 | 0      |     0 |       4.5 | 1110000000001001 | 3000000000000070 | 
 +     58390 | 2032-03-04 06:00:00 |      2.5 |     110 | 2032-03-04 06:10:54 |    302 | 0      |     0 |       2.5 | 1110000000001000 | 7000000000000000 | 
 +     58390 | 2032-03-05 06:00:00 |      8.5 |     110 | 2032-03-05 06:25:24 |    302 | 0      |     0 |       8.5 | 1110000000001000 | 7000000000000000 | 
 +     58390 | 2032-03-06 06:00:00 |        7 |     110 | 2032-03-06 06:25:24 |    302 | 0      |     0 |         7 | 1110000000001000 | 7000000000000000 | 
 +     58390 | 2032-03-07 06:00:00 |      0.2 |     110 | 2032-03-07 06:25:21 |    302 | 0      |     0 |       0.2 | 1110000000001000 | 7000000000000000 | 
 +     58390 | 2032-03-08 06:00:00 |       -1 |     110 | 2032-03-08 06:25:17 |    302 | 0      |     0 |        -1 | 1110000000001000 | 7000000000000000 | 
 +     58390 | 2032-03-09 06:00:00 |      2.4 |     110 | 2032-03-09 06:06:31 |    302 | 0      |     0 |       2.4 | 1110000000001000 | 7000000000000000 | 
 +     58390 | 2032-03-10 06:00:00 |      2.2 |     110 | 2032-03-10 06:25:20 |    302 | 0      |     0 |       2.2 | 1110000000001001 | 3000000000000070 | 
 +
 +</code>
 +
 +**Command**
 +<code>
 +./generator.pl "2032-03-09 06:00:00" 7 58390 302 110
 +</code>
 +
 +
 +**After**
 +<code>
 +     58390 | 2032-03-02 06:00:00 |      0.3 |     110 | 2032-03-02 06:25:19 |    302 | 0      |     0 |       0.3 | 1110000000001001 | 3000000000000070 | 
 +     58390 | 2032-03-03 06:00:00 |   -32767 |     110 | 2032-03-03 06:10:32 |    302 | 0      |     0 |       4.5 | 9999999999992990 | 3000000000000070 | TEST-DATA-RA_RR24 original=4.5
 +     58390 | 2032-03-04 06:00:00 |   -32767 |     110 | 2032-03-04 06:10:54 |    302 | 0      |     0 |       2.5 | 9999999999992990 | 7000000000000000 | TEST-DATA-RA_RR24 original=2.5
 +     58390 | 2032-03-05 06:00:00 |   -32767 |     110 | 2032-03-05 06:25:24 |    302 | 0      |     0 |       8.5 | 9999999999992990 | 7000000000000000 | TEST-DATA-RA_RR24 original=8.5
 +     58390 | 2032-03-06 06:00:00 |   -32767 |     110 | 2032-03-06 06:25:24 |    302 | 0      |     0 |         7 | 9999999999992990 | 7000000000000000 | TEST-DATA-RA_RR24 original=7
 +     58390 | 2032-03-07 06:00:00 |   -32767 |     110 | 2032-03-07 06:25:21 |    302 | 0      |     0 |       0.2 | 9999999999992990 | 7000000000000000 | TEST-DATA-RA_RR24 original=0.2
 +     58390 | 2032-03-08 06:00:00 |   -32767 |     110 | 2032-03-08 06:25:17 |    302 | 0      |     0 |        -1 | 9999999999992990 | 7000000000000000 | TEST-DATA-RA_RR24 original=-1
 +     58390 | 2032-03-09 06:00:00 |     25.1 |     110 | 2032-03-09 06:06:31 |    302 | 0      |     0 |       2.4 | 9999999999992990 | 7000000000000000 | TEST-DATA-RA_RR24 original=2.4
 +     58390 | 2032-03-10 06:00:00 |      2.2 |     110 | 2032-03-10 06:25:20 |    302 | 0      |     0 |       2.2 | 1110000000001001 | 3000000000000070 | 
 +
 +</code>
  • kvalobs/kvoss/system/qc2/test/acceptance/alg001.1269203238.txt.gz
  • Last modified: 2022-05-31 09:23:18
  • (external edit)