Differences

This shows you the differences between two versions of the page.

--- kvalobs:kvoss:system:qc2:requirements:algorithms:rarr_24 [2008-10-10 13:02:49]
paule created
+++ kvalobs:kvoss:system:qc2:requirements:algorithms:rarr_24 [2022-05-31 09:29:32] (current)
@@ Line 1: / Line 1: @@
-==== DESCRIPTION OF METHOD ====
-	RR_24 is the precipitation collected over the last 24 hours. The Qc1 check precipcollected_flag identifies if an RR_24 value is valid (fd=1), erroneous (fd=3) or is the accumulation (fd=2) of a number (N) of consecutive 24 hour periods, where N>1. For the case fd=2, the accumulated value shall be distributed over the N 24 hours periods. For each day the calculated
+====== Redistribution of 24 hour precipitation ======
-	RR_24= model_value*observed_accumulation/sumof_model_values
-where the model values for the N missing values of RR_24 are calculated by an interpolation from the other stations making measurements at the same time and for which fd=1. The initial interpolation method is an inverse 2-D distance weighting. (The method of interpolation applied shall eventually be pluggable and include 3-D options.)
-The calculated RR_24 updates the corrected value in the kvalobs database.
+RR_24 is the precipitation collected over the last 24 hours. The Qc1 check precipcollected_flag identifies if an RR_24 value is valid (fd=1), erroneous (fd=3) or is the accumulation (fd=2) of a number (N) of consecutive 24 hour periods, where N>1. For the case fd=2, the accumulated value shall be distributed over the N 24 hours periods.
-==== QUALITY CONTROL LEVEL ====
+For each day the calculated
-	Non-real time (Qc2).
+	RR_24= model_value*observed_accumulation/sum_of_model_values
+where the model values for the N missing values of RR_24 are calculated by an interpolation from the other stations making measurements at the same time and for which fd=1.
+At present, the interpolation uses an inverse 2-D squared distance weighting.
+Other weighting and interpolation techniques might be added later.
+The calculated RR_24 updates the corrected value in the kvalobs database.
-==== RUN PARAMETERS ====
+A warning is given if fmis=0 and original!=-1.
-	It shall be possible to run the algorithm for
+==== Configuration and Parameters ====
--	a fixed time range or
--	an arbitrary interval from a past date up to the current time
-	The typeid to which the algorithm is applied
-==== CHARACTERISTICS OF DATA TO WHICH METHOD IS APPLIED: ====
+The standard configuration options detailed in the [[kvoss:system:qc2:user|user guide]] also apply to ''Redistribution''.
-== CONTROLINFO AND/OR USEINFO: ==
+Further configuration options are:
-	Interpolation utilises data for which fd=1 [c(12)=1] and the corrections are applied to fd=2 and fhqc=0 	[i.e. c(15)=0, indicated that no HQC is applied to the observation.]
+^ Option ^ Default ^ Description ^
+| endpoint_cflags | fmis=[04]&fd=2&fhqc=0    | specify the flags for accumulation endpoints. Endpoints not matching these flags will not be detected |
+| endpoint_uflags | no constraints | ::: |
+| missingpoint_cflags | fmis=3&fd=2 | specify the flags for the missing points before the accumulation |
+| missingpoint_uflags | no constraints | ::: |
+| neighbor_cflags | fd=1 | specify requirements for the neighbor stations used for the interpolations. Neighbor stations that do not match these flags will be ignored. |
+| neighbor_uflags | U2=0 | ::: |
+| warn_and_stop_cflags | fhqc=)0( | define for which flags for the accumulation point or the missing points before the accumulation the only action should be to print a warning and stop |
+| warn_and_stop_uflags | no constraints | ::: |
+| update_flagchange | %%fd=7;fmis=3->fmis=1%% | flags to change for the accumulation endpoint and the missing points |
+| ParamId | --- | selects the parameter ids to use. The only value tested here is 110 (RR_24). |
+| tids | --- | accepted typeids |
+| measurement_hour | 6 | specifies the hour at which the data are expected. Series containing data with other times  will be rejected with a warning. |
+| min_neighbors | 1 | specifies the minimum number of neighbor stations with good data |
+| max_neighbors | 5 | specifies the maximum number of neighbor stations to use |
+| InterpolationDistance | 100 | radius in km inside which to search neighbor stations |
+| warning_distance_closest_neighbor | 50 | minimum neighbor distance to print an INFO message about not so good neighbors |
+| days_before_no_neighbor_warning | 14 | specifies the number of days after which a WARN is written to the logfile if there are not enough neighbor data |
+| days_before_redistributing_zeroes_warning | 14 | specifies the number of days after which a WARN is written to the logfile if a non-dry accumulation has only dry neighbors |
-== TYPEID, PARAMIDS, OTHER METADATA: ==
+==== Parameters ====
-	Paramid=110
+This algorithm has only been tested for paramid 110.
-==== SPECIFICATION OF CONTROLFLAGS TO BE SET FOR EXPECTED RESULTS OF THE ALGORITHM, I.E. CONSIDER BOTH POSITIVE AND NEGATIVE RESULTS. ====
+==== Log Messages ====
-	controlinfo fd=8, fstat=3
+There might be many different WARN messages in the log file ''/var/log/kvalobs/kvqc2d.log'' from this algorithm. These messages are (as of version 1.3.14pre3):
-==== EXPECTATIONS OF THE USEINFO FLAGS CORRESPONDING TO CONTROLINFO FLAG SETTINGS. ====
+  * ''original = missing/rejected for endpoint'' -- an endpoint (i.e. a row matching ''endpoint_?flags'') has a missing original value; this might also indicate a row which somehow obtained the same flags as an endpoint.
+  * ''fmis=0 and original!=-1 for endpoint'' -- fmis should only be 0 for dry accumulations [[kvalobs:kvalobs-flagg#manglende_observasjon_missing|flagg-spec]]
+  * ''fd=2 and original!=corrected for endpoint'' -- redistribution has apparently been performed without setting fd=7
+  * ''endpoint matches warn_and_stop_flags'' or ''missing point ... matches warn_and_stop_flags for accumulation'' -- a row matches ''warn_and_stop_?flags'', which is ''never'' by default
+  * ''missing point ... has different fd flag than endpoint'' -- a redistribution seems to have been done without setting fd flags consistenly
+  * ''missing point ... has fmis!=3 while fd=2 for endpoint'' -- normally, all missing rows (if they exist) should have fmis=3 before redistribution
+  * ''missing point ... has fmis!=1 while fd=7 for endpoint'' -- normally, all missing rows should have fmis=1 after redistribution
+  * ''found ... but expected ... rows with corrected values for accumulation ...'' -- normally, only one row should have a corrected value if the endpoint has fd=2, and all rows if fd=6/7
+  * ''redistributed sum ... starting ... differs from original in endpoint ...'' -- the sum of corrected values in the accumulation period does not match the original value in the endpoint row; unless fhqc!=0, an attempt will be made to fix this automatically
+  * ''fhqc mess for accumulation from .. to endpoint ...'' -- some rows in the accumulation period have fhqc=0, while others have fhqc!=0
+  * ''suspicious row ... before endpoint ..; giving up'' -- a row with missing original value has been found before the assumed accumulation period; to avoid a wrong redistribution in case the correct start date for the accumulation has not been detected, the algorithm stops
+  * ''no really good neighbors at obstime ... for accumulation ending ...'' -- the nearest neighbor is quite far away (more than ''warning_distance_closest_neighbor'')
+  * ''accumulation ... > 0 would be redistributed to zeros for endpoint ...'' -- there is non-zero accumulated precipitation, but all accessible neighbor stations are dry so that the redistribution cannot be performed
+  * ''could not avoid difference of ... between distributed sum and accumulated value at endpoint ...'' -- some precipitation has been lost due to numerical (rounding) problems
-	unknown
-==== DEPENDENCIES ====
-	Precipcollected_flag.pl
-	Additional Qc2 space controls
--	to determine the level of  variability in the data volume
--	selection of data in a wet or dry volume
-	These methods will have an impact on the selection of points used in the interpolation to generate 	model values and can lead to additional quality control flag settings.