Installation

Pre-build Packages

If you are lucky to run one of the following operating systems, you can install fimex from a package repository:

Ubuntu 10.04 Lucid

Ubuntu Lucid packages are available since 0.28 at: https://launchpad.net/~heiko-klein/+archive/fimex

Ubuntu 8.04 Hardy

Ubuntu Lucid packages are available up to 0.25 at: https://launchpad.net/~heiko-klein/+archive/fimex

Dependencies

Fimex requires at least the following libraries to be installed for compilation:

c99/c++ compiler

libxml2 >= 2.5.0

boost library >= 1.32

proj-4 >= 4.4.9

udunits >= 2.1.x or 1.12.x (deprecated)

To configure the different file formats or features it requires:

NetCDF (netcdf-3 > 3.6)

grib_api (> 1.4)

postgres-client libraries for wdb

Log4cpp: advanced logging library

OpenMP-3.0 enable compiler: Parallelization

Fortran-2003 compiler (tested with gfortran-4.6 and ifortran-12)

R (>= 2.14)

Build

The installation follows the usual autoconf rules, see 'INSTALL' for extensive information. Since Fimex makes some floating-point calculations in large loops, it is advisable to switch on SIMD/SSE operations in your compiler. On a Xeon machine with a x386 machine and gcc, I use the following flags (those are default for x86-64) (see microbenchmark of sse for more information)

Since fimex 0.21

CFLAGS='-O2 -mfpmath=sse -msse2 -ftree-vectorize -fno-math-errno' CXXFLAGS='-O2 -mfpmath=sse -msse2 -ftree-vectorize -fno-math-errno' ./configure --with-netcdf=/usr --with-udunits=/usr
make
make check
make install

Static netcdf4

Use nc-config or package-config scripts to detect library dependencies, e.g.

./configure --with-netcdf=/paht/to/nc-config

Additional Libraries

Some libraries need to be added before running ./configure. These libraries should be added into the LDFLAGS variable, f.e. for libfoo in /opt/foo/lib put

export LDFLAGS='-L/opt/foo/lib -lfoo'
./configure

If you need to add several libraries, append them with space to the LDFLAGS line, e.g. for an additional library libbar:

export LDFLAGS='-L/opt/foo/lib -lfoo -lbar'
./configure

grib_api

Most grib_api versions do not come with an automatic library detection program like package-config. When linking against grib_api, you need to know the required library flags, e.g. -L/path/to/jasper -ljasper -lpng -ljpeg or -L/path/to/openjpeg -lopenjpeg -lpng -ljpeg. Put this string into the LDFLAGS variable before running configure, e.g.

export LDFLAGS='-L/path/to/openjpeg -lopenjpeg -lpng -ljpeg'
./configure --with-grib_api=/path/to/grib_api

udunits2

udunits2 requires the expat library. Depending on the library version, this is compiled into the library, or put into the same path as udunits2, or it uses the systems expat library. So, if you run into problems with ./configure udunits2 detection, try to add -lexpat to the LDFLAGS variable:

export LDFLAGS='-lexpat'
./configure --with-udunits2

Optimizations with gcc

The following optimizations have been tried with gcc 4.4 and kd-tree interpolation.

-O2 general optimizations like loop-unrolling/inline

-mfpmath=sse -msse2 on i386 computers, use math from sse-unit. The build-in FPU is not IEEE conform and performs very slowly (> factor 2) when calculating sqrt(nan). This is already the default on i386-64.

-ftree-vectorize -fno-math-errno gives tiny performance gain through auto-vecotrization. -fno-math-errno is required to enable vectorization of intrinsic functions like sqrt.

-fno-trapping-math -fno-signaling-nans fimex does not make use of trapping-math or signaling-nans, so this option can safely be switched on. Unfortunately, this does not give any performance gain.

-O3 gives no visible performance gain. This should be tested again with newer compiler-versions.

-fprofile-generate/-fprofile-use gives a performance gain of ~4%. Since it complicates the build, it is not used by default.

Parallelization (OpenMP)

Fimex can be build with OpenMP parallelization support with the configure option --enable-openmp. The following operations are parallelized:

interpolation: fill2d (good parallelization on z-axis)

interpolation with coord_nearestneighbor (good parallelization in startup)

CAVEAT (fimex < 0.38.1)

Due to a bug in libtool, the options required for linking against a parallelized libfimex are not set automatically and need to be entered manually by the programmer using libfimex: https://lists.gnu.org/archive/html/bug-libtool/2011-03/msg00022.html . Libtool has been upgraded to a newer version in fimex 0.38.1 and this issue is no longer relevant.

Test

Download the file with testdata: flth00.dat.gz

Put this file into the test-catalogue

gunzip the file

run 'make check' (make check will work even without the testdata and skip a lot of tests)

Known Problems

'make -j 2 check' fails

The targets in make check need to be run in a specific order and some will write to the same file if run in parallel. Avoid any parallelism (-j X) for make check

udunits1 and FMRC

udunits1 does not differ between a date and a timespan, e.g.

$ ./udunits
You have: hours
You want: hours since 2010-01-01 00:00:00
<hours since 2010-01-01 00:00:00> = <hours>*1 - 78888
<hours since 2010-01-01 00:00:00> = <hours>/1 - 78888

Therefore, data with two time-axis, e.g. in ForecastModelRuntimeConvention cannot get read proberly. This results in a test-failure:

unknown location:0: fatal error in "test_cs_slicebuilder_reftime": unknown type
testCoordinateSystemSliceBuilder.cc:69: last checkpoint
*** 1 failure detected in test suite "Master Test Suite"
FAIL: testCoordinateSystemSliceBuilder

If you don't intend to use FMRC, you can ignore this error, otherwise, you should upgrade to udunits2.

udunits1 and udunits2

If both udunits1 and udunits2 are available on the machine, fimex might prefer udunits1 unless explicitly disabled. To explicitly disable udunits1, use:

./configure --without-udunits --with-udunits2 ...

MacOS X and gcc

Under MacOS X strings may come in different flavour for gcc, either static or fully dynamic. Libraries compiled with one type of strings don't work well with programs/libraries compiled with the other type. This might result in an segmentation fault when running fimex with the message:

pointer being freed not allocated

boost::program_options is affected by this and must be compiled with the same compiler as fimex is compiled, e.g.

download boost

add using darwin : : g++-mp-4.6 ; in user-config.jam

run

./bootstrap.sh --prefix=/usr/local
./b2 install

For more information, see http://lists.boost.org/boost-users/2010/06/59548.php

AIX

The boost libraries (1.44) are not compatible with the AIX XL/C++ compiler V10 and we had to compile boost and fimex with gcc to get it working under AIX. This might have been improved with newer versions of the AIX compiler, but hasn't been tested: http://www-01.ibm.com/support/docview.wss?uid=swg27018656