Differences
This shows you the differences between two versions of the page.
Next revision | Previous revision | ||
metamod:roadmap [2008-10-21 11:02:12] heikok created |
metamod:roadmap [2022-05-31 09:29:32] (current) |
||
---|---|---|---|
Line 1: | Line 1: | ||
- | ====== Roadmap ====== | + | ====== Roadmap |
+ | |||
+ | **due Jan/Feb 2011** | ||
+ | |||
+ | The roadmap is currently work in progress. | ||
+ | |||
+ | The requirements specification for the release can found here https:// | ||
+ | |||
+ | Current open questions can be found here [[open_questions28]] | ||
+ | |||
+ | ===== Planned changes ===== | ||
+ | |||
+ | * Rewrite the web frontend from PHP to the [[http:// | ||
+ | * Implement collection basket. | ||
+ | |||
+ | Of the planned changes the conversion to Catalyst is expected use most of the time. | ||
+ | |||
+ | ===== Conversion to Catalyst ===== | ||
+ | |||
+ | ==== Functionality ==== | ||
+ | |||
+ | The functionality is expected to be the same for most parts of the application except that the use of sessions will be removed from the search page. | ||
+ | |||
+ | ==== Technology ==== | ||
+ | |||
+ | The conversion will be based on the following Perl modules: | ||
+ | |||
+ | * Moose | ||
+ | * Template:: | ||
+ | * DBIx:: | ||
+ | |||
+ | ==== Installation and deployment ==== | ||
+ | |||
+ | Installation of dependencies will be based on .deb packages. As we cannot be restricted by the packages that are provided as part of Debian we need to create our own packages for all CPAN packages that are used. | ||
+ | |||
+ | When deploying an application update_target.pl will still be used. update_target.pl will still be able to replace any file from the source directory to the target directory. Over time it is expected that this functionality is not needed and that it will be replaced by a different system. What this will be is not known at this time. | ||
+ | |||
+ | To override the CSS styles for the application, | ||
+ | |||
+ | |||
+ | ====== Roadmap to 2.9 ====== | ||
+ | |||
+ | This version due in august 2011 | ||
+ | |||
+ | ===== Priority tasks ===== | ||
+ | |||
+ | ==== Simplify Installation (ØT) ==== | ||
+ | |||
+ | * Goals: | ||
+ | * All scripts (Perl and bash) run from source catalogue. | ||
+ | * Source can be packaged as .deb and installed on server, e.g. in / | ||
+ | * Several instances share the same installed METAMOD source. | ||
+ | * Why: | ||
+ | * Simplified development. | ||
+ | * Simplified deployment. | ||
+ | * Required changes: | ||
+ | * Bash scripts must read master_config.txt at runtime. No extra generation step can be used. | ||
+ | * We have a prototype for this, but we must also check all scripts that the variables are used correctly. | ||
+ | * All scripts (Perl and bash) must take the path to master_config.txt as a commandline parameter and/or an environment variable. | ||
+ | * I am unsure how this shall be done for scripts installed as services in / | ||
+ | * We must have separate scripts per site as today (due to limitations in System V init scripts); ergo these still must be generated, e.g. by install_jobs. Putting the config in / | ||
+ | * Path to custom catalogue not hardcoded, but moved to master_config.txt. | ||
+ | * Path to staticdata/ not hardcoded but moved to master_config.txt. | ||
+ | * Disagree to both. Instead of pointing to the master_config file, we should instead point to the applic directory. This should have a fixed structure so that master_config, | ||
+ | |||
+ | * Limitations: | ||
+ | * Only one version of METAMOD installed per server when installed via Debian package. | ||
+ | * Presumably this means one //copy// of the code per server. We can still allow different versions by including the version number in the path, e.g. / | ||
+ | |||
+ | < | ||
+ | |||
+ | After the changes METAMOD will be started like this: | ||
+ | |||
+ | From source: | ||
+ | |||
+ | catalyst/ | ||
+ | |||
+ | On deployment server (for testing): | ||
+ | |||
+ | metamod_server.pl --metamod-config <path to application dir> | ||
+ | |||
+ | On deployment server (inside init script): | ||
+ | |||
+ | start-stop-daemon --pidfile $PIDFILE ... \ | ||
+ | --startas $BINDIR/ | ||
+ | --metamod-config $APPLICATION_DIR ... | ||
+ | |||
+ | where PIDFILE, BINDIR, PORT and APPLICATION_DIR are all specified in / | ||
+ | |||
+ | ==== Near Real-time data ==== | ||
+ | |||
+ | Changing XML-files gives often wrong feedback to the user because files are changed in the file-database, | ||
+ | |||
+ | This will also improve testability since mmtime will no longer be required. | ||
+ | |||
+ | * **import_dataset** should **not** poll datestamp changes on XML-files, but instead run from a queue/ | ||
+ | * everything changing the XML-files should trigger a import_dataset directly in the database. Import_dataset should not be called as a system command - but as a library function. | ||
+ | * everything changing the XML-files runs currently through Metamod:: | ||
+ | * direct connection to database, Metamod:: | ||
+ | * **TASKS** | ||
+ | * < | ||
+ | * Add better validation to questionnaire data as it is now inserted directly into the database. | ||
+ | * < | ||
+ | * Add proper error handling to Metamod:: | ||
+ | * < | ||
+ | * Refactor Metamod:: | ||
+ | * Remove the user of shared metadata to ensure proper cleanup of database on re-import. | ||
+ | * Find out where to add call to subscription service as it should not be the responsibility of Metamod:: | ||
+ | * Fix rights for database web user. | ||
+ | * < | ||
+ | * This will remove upload-time from 1h10m to some seconds | ||
+ | * Syntheses run multiple times, problem? | ||
+ | * evaluate use of {{http:// | ||
+ | * < | ||
+ | |||
+ | ==== ncdigest independent of datasource ==== | ||
+ | |||
+ | * ncdigest should only know about the actual file to digest and older xml-files -- it should not try to read all files in the repository | ||
+ | * the ' | ||
+ | * old data-repository files may have moved to archive | ||
+ | |||
+ | ==== Improve Metadata editor ==== | ||
+ | |||
+ | **This feature is on hold until more information is available.** | ||
+ | |||
+ | * more info, please.. Geir | ||
+ | * NETMAR requirements? | ||
+ | * special editor for xmd-files (based upon ForeignDataset, | ||
+ | * Improving admin/ | ||
+ | * Additional admin/ | ||
+ | * Improving current admin/ | ||
+ | |||
+ | |||
+ | |||
+ | ==== Smaller changes / Reviews ==== | ||
+ | |||
+ | * RSS-feeds with changable content (same content as in ' | ||
+ | * Memory analysis of perl/ | ||
+ | * SRU2JDBC- full-text column should be without xml-tags | ||
+ | * upload-monitor: | ||
+ | * review: remove all intrinsic / no needed config-variables | ||
+ | * review logger categories | ||
+ | * more tests | ||
+ | * catalyst-test runs full tests through http? FIXME (clarify, please .. robot?) | ||
+ | * remove all ' | ||
+ | |||
+ | |||
+ | ===== Low priority ===== | ||
+ | |||
+ | ==== Improved Full Text search ==== | ||
+ | |||
+ | * allow ' | ||
+ | * allow phrase searches (“norwegian meteorological institute”) | ||
+ | * test cases for above, test against lucid database (pg 8.4) if possible | ||
+ | * search in all metadata-elements per dataset at once! | ||
+ | * currently ' | ||
+ | |||
+ | |||
+ | ====== Future changes ====== | ||
+ | |||
+ | * change of configuration format: https:// | ||
+ | * change import_searchdata | ||
+ | * use libxml instead of XML:: | ||
+ | * make searchdata.xml using xml-structures, | ||
+ | * use searchdata.xml only for search-interface (e.g. using xslt) -- don't restrict imported data by searchdata. | ||
+ | * change code structure | ||
+ | * join databases: move metabase to a schema of userbase and only drop schema | ||
+ | * reduce number of database-handles | ||
+ | * no need to reimport functions, i.e. tsearch/ | ||
+ | * no need to drop complete database (which requires stopping all db-connections) | ||
+ | * evaluate oai-pmh in perl | ||
+ | * digest_nc | ||
+ | * don't try to express CF-1.X in digest_nc.xml | ||
+ | * simplify digest_nc.xml only include general translations of metadata | ||
+ | * allow ncml per dataset to make specializations | ||
+ | * allow digest_nc to read from Opendap (PDL:: | ||
- | Next Release: possible December including | ||
- | * OAI-PMH harvest, distribution | ||
- | * Editing metadata through quest | ||
- | Future Development: | ||
- | * security enhancements | ||
- | * |