Roadmap to Metamod 2.4, "Osisaf"

Minutes of meetings between Heiko and Egil, 2009-12-03 and 2009-12-04.

The Osisaf-project wishes to use Metamod for search and serve of data as http or opendap, but the most important users of Osisaf want to have data in well-defined places on a ftp-server. To avoid data-duplication, it is therefore important that Metamod leaves full control about data-placement to the data-production step. Metamod should not move any Osisaf-data, or set requirements to data-names.

A independent upload task will be written. This task will be triggered based upon an url which is:

 http://metamod.../...?dataset=...&dirkey=...&filename=...&filename=...

Calling this url will extract the metadata from the files named by filename and add the metadata to the dataset. The dirkey is used as a password and needs to be set up when setting up the dataset. The data-files need be uploaded to the final data-store (ftp-server) before triggering that task. This task will only harvest metadata, but not move any data.

Comments by — Egil Støren 2009/12/08 14:34: This solution (with no restrictions on file names belonging to a given dataset) gives us some problems regarding THREDDS catalogs. Up to now, we have used the datasetScan element in the THREDDS catalog to create a dataset entry within THREDDS corresponding to all files in a directory. We can continue to use this method if all files in a dataset belongs to the same directory. Then the consequences will not be wery difficult to cope with: We can contunue to use the same method to create THREDDS catalogs, and few changes need to be done in the web interface for creating new datasets. On the other hand, if several datasets are to share the same directory, we have to do major changes both in THREDDS catalog generation and in the web interface for creating new datasets. If files in the same dataset are allowed to be situated in different directories, the situation will be even worse. So my queston is: Can we assume a one to one correspondence between dataset and lowest level directory in the data repository?

Tentative date: 2010-01-15

There are some thought about switching the internal map-search to a standard PostGIS based map-search. This would solve bug 45, and improve the change of projection in other implementations of Metamod.