Deploying a METAMOD application

Back to Index

Before you can deploy a METAMOD application you must set up a configuration directory, see METAMOD Configuration.

After the initial configuration has been completed, a number of scripts must be run before the application is up and running.

Installing services

After installation of the software and one or more configurations, you should now install the application services where the operating system can find it. Currently only Debian-based Linux systems are supported.

This script will create an Apache configuration file, as well as init.d scripts for automatic starting and stopping of Catalyst and the various METAMOD services for the specific application. These files are normally owned by root and will be placed in in <configuration dir>/etc, and then symlinked into /etc/apache2 and /etc/init.d.

  $ cd /opt/metno-metamod-2.10 # or elsewhere if installed from source
  $ ./install_jobs.sh <config>

Users with restricted sudo privileges

Some users may not have full sudo rights, and/or having PERL5LIB stripped out when running under sudo (see http://www.sudo.ws/sudo/alerts/perl_env.html). You will then typically get the following error message:

  sudo: sorry, you are not allowed to set the following environment variables: PERL5LIB

If so you may run install_jobs with the -u option (for unprivileged), in which case all files will be owned by yourself instead of root:

  $ ./install_jobs.sh -u <config>

Note that this is not recommended in production environments where several login users are administering the system.

Setting the environment

Before running any other scripts you must tell METAMOD where to find its libraries:

  $ export PERL5LIB="/opt/metno-perl-webdev-ver1/lib/perl5"

If desired you may also predefine the application config path. Then you won't have to specify the <config> parameter for each script.

  $ export METAMOD_MASTER_CONFIG="/path/to/applic/config"

All commands also run without these variables, but you must then specify the correct executable paths, configuration directory and PERL5LIB for all commands.

Using activate_env (optional)

For simplicity you may instead use the bash script activate_env that sets the relevant environment variables.

From anywhere execute the following command:

  $ source /opt/metno-metamod-<version>/activate_env <PATH TO CONFIGURATION DIR>

This will enable the correct environment. To disable the environment again execute

  $ source /opt/metno-metamod-<version>/activate_env deactivate

Creating the webrun directory

The application also needs several directories to store files during operation. Run the script

  prepare_runtime_env.sh <config>

which will initialize a runtime directory used by the application for logging, temporary files etc. owned by the APPLICATION_USER user. If already existing nothing will be deleted, although file ownership may be updated to the current configuration. (Note that if your current user does not have write privileges to WEBRUN_DIRECTORY this script will fail.)

Initializing the BASE module

At this point, the deployment of the application is complete for all instances not using the BASE module. For an application using the BASE module, a few steps remains:

As explained in the Data base paragraph, several METAMOD applications may co-operate in a cluster, and share a common data base. One of these applications must use the BASE module, and no other. The last steps (described below) required for the base application must be done after the initial steps (described above), for all applications in the cluster, are completed.

Authentication (FIXME)

Change local ident to trust in pg_hba.conf:

    $ $EDITOR /etc/postgresql/8.4/main/pg_hba.conf
    change local ident to trust

    # Database administrative login by UNIX sockets
    local   all         postgres                          ident sameuser

    # TYPE  DATABASE    USER        CIDR-ADDRESS          METHOD

    # "local" is for Unix domain socket connections only
    #local   all         all                              ident sameuser
    local   all         all                               trust
    # IPv4 local connections:
    host    all         all         127.0.0.1/32          trust
    host    all         all         157.249.0.1/16        trust
    # IPv6 local connections:
    host    all         all         ::1/128               md5

Then restart PostgreSQL:

    /etc/init.d/postgresql stop
    /etc/init.d/postgresql start

Alternatively, use .pgpass (FIXME)

Do not use PG_CONNECTSTRING_SHELL for shell connections to localhost (use sockets instead).

Creating databases

The two PostgreSQL databases used by the cluster must be initialized. Note that the two databases are operated through two database users. The name of these users are found in the master_config.txt file as the value of PG_ADMIN_USER and PG_WEB_USER configuration variables. These users must be defined in the PostgreSQL database environment before the database initialization scripts can be run:

    base/init/createusers.sh

It is only necessary to run this script once for a PostgreSQL database environment.

The first SQL database initialization script will create the Meta database.

        create_and_load_all.sh

This script will create the database, and load the static content of the database (taken from staticdata/searchdata.xml). You may check the output of this script in the create_and_load_all.out file in the directory where you are running the command.

The next SQL database initialization script will create the User database.

        run_createuserdb.sh

Note that this script must only be used during setup of a completely new METAMOD cluster. If a User database already exists, all data in the database will be lost. If this should happen, you must recreate the database from a backup copy you hopefully have. But METAMOD will not by itself ensure that such a backup exists.

This warning is only adequate for the run_createuserdb.sh script. The other scripts in this paragraph (prepare_runtime_env.sh, create_and_load_all.sh) may all be used on an existing installation, without harming existing data. The create_and_load_all.sh script will take some time to complete, though. On an existing installation, all metadata in the database will be loaded from the XML archive.

Email configuration (FIXME)

configure exim

    $ sudo dpkg-reconfigure exim4-config

Starting METAMOD

Catalyst daemon

Start Catalyst by running

        sudo /etc/init.d/catalyst-APPLICATION_ID start

Then check if METAMOD is responding (usually on port 3000, use server name if not running locally):

        http://localhost:3000/

If all set you should see the METAMOD search interface.

Apache

If this has not already been done you need to enable the Apache proxy modules. Look in /etc/apache2/mods-enabled to see if they have already been enabled.

   sudo a2enmod proxy
   sudo a2enmod proxy_http

Restart Apache:

        sudo apache2ctl restart

or if the site is busy:

        sudo apache2ctl graceful

which will wait until all connections are terminated before restarting.

The site should now hopefully be available on

        http://localhost/APPLICATION_ID

METAMOD services

Start METAMOD:

        sudo /etc/init.d/metamodServices-APPLICATION_ID start

This should start the required services daemons.

----------------

Copy the example config:

  $ cp -r /opt/metno-metamod-2.10/app/example .

Then edit example/master_config.txt as needed. For local testing, set APPLICATION_USER to your own login.

  $ /opt/metno-metamod-2.10/common/prepare_runtime_env.sh example

  $ export PERL5LIB=/opt/metno-metamod-2.10/common/lib:/opt/metno-metamod-2.10/catalyst/lib:/opt/metno-perl-webdev-ver1/lib/perl5

Try starting Catalyst:

  $ /opt/metno-metamod-2.10/catalyst/script/metamodweb_server.pl -d example

MetamodWeb should now be available on http://localhost:3000/. Stop with ctrl-c.

To install all services:

  $ /opt/metno-metamod-2.10/install_jobs.sh example

To start all services (substitute EXAMPLE with your APPLICATION_ID):

  $ sudo /etc/init.d/catalyst-EXAMPLE start
  $ sudo /etc/init.d/metamodServices-EXAMPLE start

--------------------

TESTING (OBSOLETE - FIXME)

2. To copy the software to the target directory and make the substitutions prescribed by the master_config.txt file, do the following: cd to the source directory (i.e. this directory) run: ./update_target.pl app/example

3. Assuming the PostgreSQL software is already installed, the PostgreSQL users (admin and webuser) that are to access the database must be created. This step is only needed one time. PostgreSQL users are for the whole PostgreSQL installation, and not connected to specific databases. cd to the target directory run: scripts/createusers.sh

4. Initialize the runtime environment: cd to the target directory run: ./prepare_runtime_env.sh

5. Initialize the database, and load static data: cd to the target directory cd init run: ./create_and_load_all.sh You may check the output of this script in the create_and_load_all.out file in the same init directory.

6. Ensure that you have a working Apache 2.x installation with mod_proxy. The httpd daemon should run as the same user as the METAMOD 2.x perl scripts. Otherwise problems with access rights to files in the webrun directory will arise. These problems could be solved by using 'umask' and similar tools, but the METAMOD 2.x software is not prepared for this. The Apache installation must allow .htaccess files and softlinks.

7. Make the METAMOD 2.x URLs accessible through the Apache server. This can either be done by making the METAMOD 2.x htdocs directory the Apache 2.x DocumentRoot directory. Otherwise, this can be done by providing a symbolic link in the Apache 2.x DocumentRoot directory to the METAMOD 2.x htdocs directory. This symbolic link must agree with the LOCAL_URL set up in the master_config.txt file.

8. At this time all the web applications should work. Confirm this by visiting the URLs for the two main web pages in the example application (METAMODSEARCH and METAMODUPLOAD) from a browser (the URLs are found in the master_config.txt file). Check also that the administration web page is working.

9. Enter the METAMODUPLOAD web page from a browser and create a user account. The operator will recieve an E-mail with the user details and an URL. Activating this URL will admit the user into the system. After activating this URL, the user recieves an E-mail with a password. In the test environment, this E-mail will be sent to the operator instead.

10. Start the perl scripts responsible for loading data into the database: cd to the target directory run: ./metamodInit.sh start

11. Enter the METAMODUPLOAD web page from a browser and log into the user account you have created. Enter the "Administration" page and create the directory 'test1' (this will also be the name of a new dataset that later will be loaded into the database). Go back to the "Upload files" page and upload the two files test1_arctic20.200611.cdl and test1_synop_99710.cdl found in the testdata directory.

12. After a few minutes, check that metadata from the files have been loaded into the database. This can be done by looking at the system log (path defined by the LOG4ALL_SYSTEM_LOG variable in master_config.txt), and by entering the METAMODSEARCH web page and search for the data.

13. Do a similar exercise with the files test2_arctic20.200611.cdl and test2_synop_99710.cdl. A new directory/dataset has to be created for these files in the administration page: 'test2'. These files do not satisfy the requirements specified in the etc/conf_digest_nc.xml file. The perl script digesting these files will generate an error report that will be accessible from the METAMODUPLOAD web page. Also, the data provider will be sent an E-mail with a reference to this error report. In the test environment, this E-mail will be sent to the operator instead.

14. Stop the perl scripts responsible for loading data into the database: cd to the target directory run: ./metamodInit.sh stop