ALG004

Acceptance Test for single point missing temperature values with TAN, TAX method

Step	Direction	Expected Result	Pass/Fail Criteria or Comment
001	Prepare test data in the test database.	--	--
Download Test Data and gunzip.	--	--
$ psql kvalobs	--	--
DELETE FROM data WHERE obstime BETWEEN '2024-02-28' AND '2024-04-01';	--	--
COPY data FROM '/home/kvalobs/alg004-kvalobsdb.dat' WITH DELIMITER AS '|';	--	--
NB specify the actual path to the test data above as required.	--	--
--	--
Check that the time interval contains no Qc2 processed data:	--	--
kvalobs=# select count(*) from data WHERE obstime BETWEEN '2024-02-28' AND '2024-04-01' and cfailed like '%Qc2%';	count=0	--
002	Download and gunzip alg004.cfg.gz to the /etc/kvalobs/Qc2Config directory and edit RunAtHour and RunAtMinute fields to the time that the test shall be run (e.g. a few minutes UTC in the future).	--	--
The configuration by default has W_fhqc=0 and Wbool=true. The algorithm will only work on data that ha not been controlled by HQC	--	--

The configuration file also needs to be edited to include further flag controls ( -> ) and include this:

...
#Flag to set if value is corrected
S_ftime=1
change_fmis=3->1
change_fmis=0->4
...

003	Check log files to verify that the algorithm runs as scheduled, e.g. $ tail -f /var/log/Kvalobs/Qc2.log	--	--
004	Check that Qc2 processed data now resides in the database:	--	--
kvalobs=# select count(*) from data WHERE obstime BETWEEN '2024-02-28' AND '2024-04-01' and cfailed like '%Qc2%';	count=55	--
005	Delete any existing data output (rm ALG004-Step5.dat) and then EXTRACT THE DATA:	DATA-S5	--

psql> select * from data WHERE obstime BETWEEN '2024-02-28' AND '2024-04-01' and cfailed like '%Qc2%' \g | cat >> ./ALG004-Step5.dat;

006	Plot the data. Download and use the two scripts ./unit_t_plot.bash and unitt.r.gz	--	--
Make the bash file executable:	--	--
chmod u+x unit_t_plot.bash	--	--
Run the script	--	--
./unit_t_plot.bash ALG004-Step5.dat	--
and inspect the produced image ALG004-Step5.dat.jpg file.	--	--
007	Edit the configuration file and set Wbool=false. Rerun the algorithm. The algorithm will reprocess the data flagged as controlled by HQC already.	--	--
kvalobs=# select count(*) from data WHERE obstime>'2024-02-28' and obstime<'2024-04-01' and cfailed like '%Qc2%';	count=121	--
Capture the results. (rm ALG004-Step7.dat any previous test results)	DATA-S7	--

psql> select * from data WHERE obstime BETWEEN '2024-02-28' AND '2024-04-01' and cfailed like '%Qc2%' \g | cat >> ./ALG004-Step7.dat;

008	Plot results as in step 006 above.	--

Prerequisites

Set up the test data (these are prepatory notes for now for set up on pak)

Test Data comprising original data from 2008-03 with all the years set to 2024, the file only contains the parameters for the algorithm.

Configuration File

Setting-up or refreshing the database. (some database management may be required to stop page errors and the like ????, vacuum cleaning etc ???)

$ psql kvalobs

psql=> DELETE FROM data WHERE obstime BETWEEN '2024-02-28' AND '2024-04-01';

psql=> COPY data FROM ./alg004-kvalobsdb.dat WITH DELIMITER AS '|';

Script for plotting the test results. See the Step # above: unit_t_plot.bash.gz

... and not to forget the corresponding R-script called by the shell script: unitt.r.gz