This is an old revision of the document!
GAP INTERPOLATION
The Akima Spline interpolates based on the gradients between points of the nearest neighbours in the time series. At least five neighbours are required (this may be satisfied by having two to three points on either side of a gap). The values of more distant neighbours appear to have little effect on the finally interpolated data.
Advantage is taken of this behaviour in the proposed variable length gap algorithm, as described here:
- For a given station an arbitrary length time series of data is selected.
- Points in the time series that satisfy a given quality condition are used to generate the Akima Spline. (in the first version of the algorithm only data where useinfo(2)=0 , any other criteria may also be set).
- The time series is then parsed for missing values. If a value is missing but it is sufficiently close to enough good neigbours (exact details of this to be decided - please see discussion below) then it can be replaced with the value from the Akima spline.
Using Akima For Variable Length Gaps
An example illustrating the performance of the Akima interpolation.
Original Data
Simulated Gaps
For this example we generate our own gaps so that the results may also be compared to the original values.
Akima Fit
Akima Fit also indicating the interpolated data points (triangles) and original values (red circles)
Comparison of data
| Hour | … | 21 | 22 | 23 | 24 | 25 | … | 38 | 39 | 40 | … | 48 | 49 | 50 | 51 | … | 97 | 98 | 99 | 100 | 101 | 102 | 103 |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Original Value | … | -0.2 | 0.1 | -0.4 | 1 | 0.7 | … | 3.8 | 4.5 | 4 | … | 2.7 | 2.5 | 2.4 | 2.5 | … | 1.8 | 1.7 | 1.5 | 1.6 | 1.4 | 1.4 | 1.8 |
| Akima | … | -0.6 | -0.6 | -0.4 | 0.0 | 0.3 | … | 4.1 | 3.8 | 3.7 | … | 2.7 | 2.6 | 2.6 | 2.6 | … | 1.7 | 1.5 | 1.4 | 1.4 | 1.4 | 1.6 | 1.8 |
Some Thoughts about when to fit and when not to fit
The criteria to decide if to input an interpolation may be one or a combination of the following:
- Length of the gap
- Distance of a missing point from the nearest neighbour
- Distance of a missing point from the second nearest neighbour
- etc. …
- Other criteria
The following sketch illustrates these points:
Think Tank
Legend:
| G | Good Point |
|---|---|
| X | Missing Point |
| I | Interpolated Point |
| Time Series | Gap ⇐ 4 hours | Two Neighbours in the same direction ⇐ 2 hours |
|---|---|---|
| GGGXXXXXXGGG | GGGXXXXXXGGG | GGGIIXXIIGG |
| GGGXXXXGXXXXGGG | GGGIIIIGIIIIGGG | GGGIIXXGXXIIGGG |
etc. …
| Which cases do we want to apply long interpolations to? |
|---|
FLAGS and CFAILED values to SET
In the current implementation the same flag settings are used as is the case for the Single Linear Interpolation (ftime=1) and CFAILED is set to QC2d-2-A, the extra string (“LONG GAPS” or similar) is temporary and just helps my testing.
67560 | 2008-04-18 06:00:00 | 2.1 | 215 | 2008-04-18 05:48:32 | 330 | 0 | 0 | 2.1 | 1111000000000010 | 7100000300000000 |
67560 | 2008-04-18 07:00:00 | 2.8 | 215 | 2008-04-18 06:49:28 | 330 | 0 | 0 | 2.8 | 1111000000000010 | 7100000300000000 |
67560 | 2008-04-18 08:00:00 | -32767 | 215 | 2008-04-18 08:30:18 | 330 | 0 | 0 | 3.8 | 0000001100000005 | 2892900000000041 | hqc,QC2d-2-A,XLONG GAPS
67560 | 2008-04-18 09:00:00 | -32767 | 215 | 2008-04-18 09:30:38 | 330 | 0 | 0 | 4.8 | 0000001100000005 | 2892900000000041 | hqc,QC2d-2-A,XLONG GAPS
67560 | 2008-04-18 10:00:00 | 5.6 | 215 | 2008-04-18 10:11:32 | 330 | 0 | 0 | 5.6 | 1110000000000010 | 7000000000000000 |
67560 | 2008-04-18 11:00:00 | 5.9 | 215 | 2008-04-18 10:48:32 | 330 | 0 | 0 | 5.9 | 1111000000000010 | 7100000300000000 |
Note the same algorithm will also do the single points as well
12320 | 2008-03-23 02:00:00 | -13.9 | 211 | 2008-03-23 02:07:03 | 330 | 0 | 0 | -13.9 | 1111100000000010 | 7000000000000000 |
12320 | 2008-03-23 03:00:00 | -32767 | 211 | 2008-03-23 03:31:13 | 330 | 0 | 0 | -14.6 | 1000601100000007 | 1892900000000052 | QC1-4-211:1,hqc,QC2d-2-A,LONG GAPS
12320 | 2008-03-23 04:00:00 | -15.1 | 211 | 2008-03-23 04:07:04 | 330 | 0 | 0 | -15.1 | 1110100000000010 | 7000000000000000 |
Here is the data history for the above point: