Batch Diana on Headless Systems

Normally, Diana is configured to run on systems with an X server, using OpenGL for accelerated rendering of maps and data. When running Diana in batch mode with the bdiana tool, nothing is displayed on the screen via the X server, but the user is often running bdiana on a system with an X server present. When building Diana normally, the bdiana tool that is produced is linked to the X libraries on the system and can be run by users on workstations without problems.

On servers, where it is not desirable to run an X server, we cannot use the bdiana tool produced by the normal Diana build without some kind of X server. Traditionally, Xvfb has been used to provide the support that this version of bdiana needs to run. However, this was not always a reliable solution, so we have used an alternative configuration of Diana to produce a version of bdiana that runs without an X server. This does two things to avoid dependencies on X:

1. It uses a custom build of the Qt libraries that do not depend on X libraries.
2. It provides a wrapper that redirects OpenGL calls to Qt, rendering products entirely in software. Only the subset of OpenGL functions that Diana uses are wrapped.

We build an alternative set of Qt libraries using the standard Qt 4.8 sources, applying some patches and using an “embedded” configuration to remove the dependency on X.

Packages were built with Qt4 versions

• 4.8.1
• 4.8.3
• 4.8.6

The patches we use are part of the Debian packaging we use to build packages for Ubuntu 12.04 and 14.04. These can be obtained from github qt4-headless:

git clone https://github.com/metno/qt4-headless qt4-headless
# the branch 4.8-qpa contains the most recent patches (as of 2016-02-26)
git checkout 4.8-qpa

To build debian packages, we re-package the Qt4 source code to obtain the directory structure expected by the dpkg build tools, then extract the re-packaged source archive, copy in the debian directory from git git repository, and build packages with debuild or similar.

Packages for some ubuntu releases are available from the MET Norway PPA.

If you are building on another distribution, you will need to patch the files by hand. Enter the Qt 4.8 source directory and run a command of the following form on each of the patches in ascending numerical order; for example:

patch -p1 < ../qt4-headless/debian/patches/metno_00_remove_qpicture_checksum.diff
patch -p1 < ../qt4-headless/debian/patches/metno_01_identitymodel_feature_fix.diff
patch -p1 < ../qt4-headless/debian/patches/metno_02_wms_configuration.diff
patch -p1 < ../qt4-headless/debian/patches/metno_03_dummy_screen.diff
patch -p1 < ../qt4-headless/debian/patches/metno_04_qpa.diff

You can then run the configure script using the command given in the debian/rules file:

./configure -confirm-license -depths 16,32 -qpa dummy -qconfig wms \
            -exceptions -fast -glib -iconv -largefile -nis \
            -no-accessibility -no-audio-backend -no-cups -no-dbus \
            -no-declarative -no-declarative-debug -no-gfx-linuxfb \
            -no-gfx-multiscreen -no-javascript-jit -nomake demos \
            -nomake docs -nomake examples -nomake tests -nomake tools \
            -no-multimedia -no-opengl -no-phonon -no-phonon-backend \
            -no-qt3support -no-script -no-scripttools \
            -no-separate-debug-info -no-sql-ibase -no-sql-sqlite2 \
            -no-webkit -xmlpatterns -opensource -openssl \
            -optimized-qmake -pch -prefix-install -prefix /opt/qt4-headless \
            -release -rpath -shared -stl -svg -system-freetype \
            -no-libjpeg -no-libmng -system-libpng -no-libtiff -system-zlib

If this fails then you will need to provide the appropriate dependencies for your system.

Once configured, build and install in the usual way:

make
sudo make install

Diana is typically only built for batch use on headless systems since the GUI is only useful when there is an attached display. This means that only the bdiana tool is built. The debian_bdiana directory supplied with the Diana sources contains information about the dependencies required for this configuration (in the control file) and the commands required to configure and build bdiana (in the rules file).

Packages compiled for headless systems are available from the MET Norway PPA for some ubuntu releases.

Only a subset of the libraries mentioned on the Building Metlibs and Diana with Autotools page are required. Follow the instructions there to build and install Fimex and the miLogger, puCtools, puTools, miRaster and diField libraries.

Enter the Diana source directory, and configure, build and install bdiana with the following commands. Change the prefix option if you do not want to install the application under /usr/local:

./autogen.sh
./configure --disable-silent-rules \
            --prefix=/usr/local \
            --enable-obs-bufr \
            --enable-video-export \
            --enable-geotiff \
            --with-qt4=/opt/qt4-headless \
            --with-qt4-includedir=/opt/qt4-headless/include \
            --with-qt4-libdir=/opt/qt4-headless/lib \
            --with-qmake=/opt/qt4-headless/bin/qmake \
            --with-moc=/opt/qt4-headless/bin/moc \
            --with-rcc=/opt/qt4-headless/bin/rcc \
            --with-uic=/opt/qt4-headless/bin/uic \
            --with-lupdate=/opt/qt4-headless/bin/lupdate \
            --with-lrelease=/opt/qt4-headless/bin/lrelease \
            --enable-embedded \
            --enable-perl \
            --enable-batch-only
make
sudo make install

See the Installing Data Files section of the Getting Started guide for instructions on how to install the data files that bdiana needs.

Optional: See the Python Bindings for Diana on Headless Systems for instructions on building software that can be used to create services based on Python and Diana.