Differences
This shows you the differences between two versions of the page.
Both sides previous revision Previous revision Next revision | Previous revision | ||
fimex:faq [2009-01-23 13:18:04] heikok |
fimex:faq [2024-07-03 07:11:55] (current) alexanderb opendap auth on jammy |
||
---|---|---|---|
Line 1: | Line 1: | ||
====== Frequently Asked Questions ====== | ====== Frequently Asked Questions ====== | ||
- | ==== Linking with '' | + | ===== Does fimex use file-locks for reading/ |
- | Use '' | + | |
- | ==== Problems | + | It is very common, that NWP (numerical weather prediction) files are updated while the model is running. It is often desirable to be able to read the file already when the model is still running, e.g. to see the first few ours of weather prediction. |
+ | |||
+ | * Fimex does not use file-locks for reading: Graphical interfaces like [[http:// | ||
+ | * Fimex does not implement file-locking for writers generally, though in particularly with the [[http:// | ||
+ | |||
+ | |||
+ | ===== How can I change the units of a variable? ===== | ||
+ | |||
+ | If the original variable e.g. '' | ||
+ | |||
+ | If the input-file has the wrong units, e.g. Celsius while the data in fact is Kelvin, you should use the input.config=... | ||
+ | |||
+ | |||
+ | ===== How to solve problems | ||
I have problems with a felt-file with a variable of type ' | I have problems with a felt-file with a variable of type ' | ||
- While the netcdf-format allows for only 1 scaling-factor per variable, the felt-format can have a scaling-factor per 2d-slice per variable (that are level*time scaling-factors). You should change the //type// of the variable and its // | - While the netcdf-format allows for only 1 scaling-factor per variable, the felt-format can have a scaling-factor per 2d-slice per variable (that are level*time scaling-factors). You should change the //type// of the variable and its // | ||
- | - If you want to save disk-space, and you know that all data of the variable can be subscribed | + | - If you want to save disk-space, and you know that all data of the variable can be subscribed |
- | <variable name="31,2,1000" | + | <?xml version="1.0" encoding=" |
- | < | + | <netcdf xmlns=" |
- | < | + | |
- | < | + | |
+ | <!-- change the type of a variable, repack --> | ||
+ | < | ||
+ | | ||
+ | | ||
+ | | ||
</ | </ | ||
- | </ | + | </ |
+ | </ | ||
+ | In this example, the precipitation | ||
+ | |||
+ | ===== Vector-rotation ===== | ||
+ | |||
+ | ==== Which vectors are auto-detected for rotation ==== | ||
+ | |||
+ | Vectors are autorotated if one of the following applies: | ||
+ | |||
+ | * the ''< | ||
+ | * the vector has a known CF-standard-name, | ||
+ | * x_wind, y_wind | ||
+ | * *_x_velocity, | ||
+ | * *_x_transport, | ||
+ | * *_x_displacement, | ||
+ | * *_eastward_*, | ||
+ | * the complete list can be found in the sources after '' | ||
+ | |||
+ | ==== Vector-rotation gives error-message (>= 0.48) and wrong values due to different units ==== | ||
+ | |||
+ | When rotating vectors for spatial interpolations, | ||
+ | |||
+ | A solution is to configure the vector to have the same scale/ | ||
+ | |||
+ | < | ||
+ | < | ||
+ | < | ||
+ | < | ||
+ | <remove name=" | ||
+ | < | ||
+ | </ | ||
+ | |||
+ | < | ||
+ | < | ||
+ | < | ||
+ | <remove name=" | ||
+ | < | ||
+ | </ | ||
+ | </ | ||
+ | |||
+ | |||
+ | ===== What does this error-message mean? ===== | ||
+ | |||
+ | ==== CDMException: | ||
+ | |||
+ | The config-file for felt-files must at least define 1 existing parameter in the corresponding felt-file. Update your config-file (e.g. felt2nc_variables.xml) with the output from the felt-index, which can be read by felt-tools like '' | ||
+ | |||
+ | ==== CDMException: | ||
+ | |||
+ | This error-message is unfortuneatly very misleading. In many cases, the '' | ||
+ | |||
+ | ===== How can I read grib-message which are split over several files? ===== | ||
+ | |||
+ | since fimex 0.40.1, it is possible to give multiple optional file-sources: | ||
+ | < | ||
+ | fimex --input.file=/ | ||
+ | --input.type=grb --input.config=/ | ||
+ | --input.optional=/ | ||
+ | --input.optional=/ | ||
+ | --output.file=eceps.nc4 | ||
+ | </ | ||
+ | |||
+ | ===== How to use fimex as opendap-client? | ||
+ | |||
+ | ==== Install netcdf with integrated dap-support ==== | ||
+ | |||
+ | === Prerequisites === | ||
+ | |||
+ | * netcdf4 (4.1.1) from unidata | ||
+ | * curl library: '' | ||
+ | * make sure to remove the libnetcdf4-dev package (on ubuntu hardy): '' | ||
+ | |||
+ | === Build === | ||
+ | |||
+ | Configuring, | ||
+ | |||
+ | < | ||
+ | ./configure --prefix=/ | ||
+ | make -k # there might be errors building the man-pages depending on tex-installation | ||
+ | make check | ||
+ | make -k install | ||
+ | </ | ||
+ | |||
+ | ==== Fimex and OpENDAP ==== | ||
+ | |||
+ | === Build === | ||
+ | |||
+ | < | ||
+ | ./configure --with-netcdf=/ | ||
+ | </ | ||
+ | |||
+ | === Running an extraction from opendap === | ||
+ | |||
+ | < | ||
+ | fimex --input.file=http:// | ||
+ | --input.type=netcdf \ | ||
+ | --output.type=netcdf \ | ||
+ | --output.file=test.nc \ | ||
+ | \ | ||
+ | --extract.selectVariables=time \ | ||
+ | --extract.selectVariables=x \ | ||
+ | --extract.selectVariables=y \ | ||
+ | --extract.selectVariables=depth \ | ||
+ | --extract.selectVariables=longitude \ | ||
+ | --extract.selectVariables=latitude \ | ||
+ | --extract.selectVariables=temperature \ | ||
+ | \ | ||
+ | --extract.reduceTime.start=2010-10-05 \ | ||
+ | --extract.reduceTime.end=2010-10-07 \ | ||
+ | \ | ||
+ | --extract.reduceToBoundingBox.south 55.0 \ | ||
+ | --extract.reduceToBoundingBox.north 80.0 \ | ||
+ | --extract.reduceToBoundingBox.east 20.0 \ | ||
+ | --extract.reduceToBoundingBox.west -5.0 | ||
+ | |||
+ | </ | ||
+ | |||
+ | === OpENDAP and slicing === | ||
+ | |||
+ | OpENDAP servers are often limited on how much data can be read at per request, e.g. thredds allows only 500MB binary data / 50MB ascii data per request before failing with an error like | ||
+ | < | ||
+ | syntax error, unexpected WORD_STRING, | ||
+ | context: Error { code = 403; message = " | ||
+ | terminate called after throwing an instance of ' | ||
+ | what(): | ||
+ | Aborted | ||
+ | </ | ||
+ | |||
+ | Fimex tries to subdevide requests into one unlimited slice per variable. Unfortunately, | ||
+ | < | ||
+ | <?xml version=" | ||
+ | <netcdf xmlns=" | ||
+ | xmlns: | ||
+ | xsi: | ||
+ | < | ||
+ | </ | ||
+ | </ | ||
+ | |||
+ | |||
+ | ==== Authentication ==== | ||
+ | |||
+ | In theory, it should be possible to specify username and password | ||
+ | - in '' | ||
+ | - as part of the URL for example '' | ||
+ | |||
+ | Example '' | ||
+ | < | ||
+ | machine thredds.met.no | ||
+ | login XXXX | ||
+ | password YYYY | ||
+ | </ | ||
+ | |||
+ | Example '' | ||
+ | < | ||
+ | HTTP.NETRC=YOURHOME/ | ||
+ | |||
+ | HTTP.CREDENTIALS.USERNAME=XXXX | ||
+ | HTTP.CREDENTIALS.PASSWORD=YYYY | ||
+ | |||
+ | HTTP.USERNAME=XXXX | ||
+ | HTTP.PASSWORD=YYYY | ||
+ | </ | ||
+ | |||
+ | Unfortunately, | ||
+ | Known status of OpenDAP authentication support (applies to '' | ||
+ | * Specifying the password in the URL does not seem to work when the password contains a ''/'', | ||
+ | * For Ubuntu 16.04 (xenial, EOL), both URL and netrc seem to work. | ||
+ | * For Ubuntu 18.04 (bionic), username and password cannot be read from netrc, but can be specified in the URL. | ||
+ | * For Ubuntu 20.04 (focal), username and password can only be read from netrc, but cannot be specified in the URL. | ||
+ | * For Ubuntu 22.04 (jammy), the only solution known to us is to add a '' | ||