fimex:faq

This is an old revision of the document!


Frequently Asked Questions

I have problems with a felt-file with a variable of type 'short'. fimex abborts with

change in scaling-factor for parameter: 31,2,1000 consider using float or double datatpye

What should I do?

  1. While the netcdf-format allows for only 1 scaling-factor per variable, the felt-format can have a scaling-factor per 2d-slice per variable (that are level*time scaling-factors). You should change the type of the variable and its _FillValue attribute from short to float in the file felt2nc_variables.xml.
  2. If you want to save disk-space, and you know that all data of the variable can be subscribed sufficiently with a packed short data-array, use the ncml-configuration as follows:
    <?xml version="1.0" encoding="UTF-8"?>
    <netcdf xmlns="http://www.unidata.ucar.edu/namespaces/netcdf/ncml-2.2"
               xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
               xsi:schemaLocation="http://www.unidata.ucar.edu/namespaces/netcdf/ncml-2.2 ncml-2.2.xsd ">
    <!-- change the type of a variable, repack -->
    <variable name="precipitation_amount" type="short">
       <attribute name="_FillValue" value="-32767" type="short" />
       <attribute name="scale_factor" value="0.1" type="float" />
       <attribute name="add_offset" value="0" type="double" />
    </variable>
    </netcdf>

In this example, the precipitation will be described with an accuracy of 0.1mm and up to 3276.7mm. The ncml-configfile will be applied right before writing with the –ncml.config=….ncml fimex-option.

The config-file for felt-files must at least define 1 existing parameter in the corresponding felt-file. Update your config-file (e.g. felt2nc_variables.xml) with the output from the felt-index, which can be read by felt-tools like rfinh.

since fimex 0.40.1, it is possible to give multiple optional file-sources:

fimex --input.file=/opdata/lameps/mle/eceps_201211260000_ens00.mars \
      --input.type=grb --input.config=/home/heikok/Programme/MetSis/Fimex/share/etc/cdmGribReaderConfig.xml \
      --input.optional=/opdata/lameps/mle/eceps_201211260000_ens01.mars \
      --input.optional=/opdata/lameps/mle/eceps_201211260000_ens02.mars \
      --output.file=eceps.nc4

Prerequisites

  • netcdf4 (4.1.1) from unidata
  • curl library: sudo apt-get install libcurl4-gnutls-dev (on ubuntu hardy)
  • make sure to remove the libnetcdf4-dev package (on ubuntu hardy): apt-get remove libnetcdf4-dev

Build

Configuring, without HDF-file support (can be added if desired). Important: include dap-support via curl; build shared libraries (libtool autodetect dependencies)

./configure --prefix=/usr/local/netcdf4.1.1 --enable-dap --with-curl-config=/usr/bin/curl-config --enable-shared --disable-static
make -k # there might be errors building the man-pages depending on tex-installation
make check
make -k install

Build

./configure --with-netcdf=/usr/local/netcdf4.1.1

Running an extraction from opendap

fimex --input.file=http://thredds.met.no/thredds/dodsC/topaz/dataset-topaz3-arc-myocean-be \
      --input.type=netcdf \
      --output.type=netcdf \
      --output.file=test.nc \
\
      --extract.selectVariables=time \
      --extract.selectVariables=x \
      --extract.selectVariables=y \
      --extract.selectVariables=depth \
      --extract.selectVariables=longitude \
      --extract.selectVariables=latitude \
      --extract.selectVariables=temperature \
\
      --extract.reduceTime.start=2010-10-05 \
      --extract.reduceTime.end=2010-10-07 \
\
      --extract.reduceToBoundingBox.south 55.0 \
      --extract.reduceToBoundingBox.north 80.0 \
      --extract.reduceToBoundingBox.east 20.0 \
      --extract.reduceToBoundingBox.west -5.0

This happens with older versions of gcc, seen with 3.3 on debian sarge.

Use -gstabs or -ggdb instead of -g. For debugging, this is better suited to c++ code and works with the GNU utils and most other debugger, too. (But unfortunately not with /valgrind/.

This website uses cookies. By using the website, you agree with storing cookies on your computer. Also you acknowledge that you have read and understand our Privacy Policy. If you do not agree leave the website.More information about cookies
  • fimex/faq.1354017816.txt.gz
  • Last modified: 2022-05-31 09:23:15
  • (external edit)