Minutes 2008-10-23

Heiko, Egil

We have currently to places to store metadata:

1. XML-files: These are the files as received from the (meta-)data provider in the original metadata-standard. Currently, providers are: digest_nc.pl (reading from nc-cf-1.0 files), quest, oai-pmh harvest
2. SQL-database: The SQL database keeps a normalized and indexed view of the XML files. The SQL-database has a known set of supported metadata-names, e.g. institution, variable, datacollection_period, abstract. Those can be found in the table MetadataType. The normalized metadata in the SQL-database can be searched through the search-module, and can be exported to other formats in the oai-pmh module (currently, conversion to DIF).

Here is the state as planned for Metamod 2.1, not everything exists yet!

• search: read from database
• base: read XML-files, normalize, write to SQL-database (import_dataset.pl)
• quest: write to XML-files, read old parameters from SQL-database this will change the metadata-format to our internal format
• upload: write to XML-files (digest_nc.pl), eventually edit metadata via quest
• pmh: read from SQL-database
• harvest: write to XML-files

• writing to SQL-database (base: import_dataset.pl) is asynchronus (once per hour) - this is required due to possible ftp-uploads
• we don't keep track of changes to XML-files (history required?) (connected to previous)
• pmh might translate metadata twice (once during harvest, once during output) - possible loss of information

• store all XML files in a blob in the database, including a history
• upload of XML-files to SQL-database including normalization should be automatically triggered by web-interface (base:import_dataset.pl). Asynchronous reading only required by (upload:digest_nc.pl).
• pmh: output original XML-files if requested metadata-standard = original metadata-standard