fimex:faq

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision 		Previous revision
fimex:faq [2013-10-03 12:59:24]
heikok [Vector-rotation gives error-message (>= 0.48) and wrong values due to different units] 		fimex:faq [2024-07-03 07:11:55] (current)
alexanderb opendap auth on jammy
Line 7: Line 7:
   * Fimex does not use file-locks for reading: Graphical interfaces like [[http://diana.met.no|diana]] which read via Fimex might keep the file open as long as it does not change, therefor locking the file for too long to allow writers to update the files. Writing additional data to an existing file is usually not a problem, in particularly when the reader re-opens the file when it has updated the time-stamp ''mtime''.   * Fimex does not use file-locks for reading: Graphical interfaces like [[http://diana.met.no|diana]] which read via Fimex might keep the file open as long as it does not change, therefor locking the file for too long to allow writers to update the files. Writing additional data to an existing file is usually not a problem, in particularly when the reader re-opens the file when it has updated the time-stamp ''mtime''.
   * Fimex does not implement file-locking for writers generally, though in particularly with the [[http://fimex.met.no/doc/fillWriterDoc.html|fillWriter]] interface this situation might change, in particular for netcdf-files   * Fimex does not implement file-locking for writers generally, though in particularly with the [[http://fimex.met.no/doc/fillWriterDoc.html|fillWriter]] interface this situation might change, in particular for netcdf-files
 +
 +
 +===== How can I change the units of a variable? =====
 +
 +If the original variable e.g. ''t2m'' has units-attribute Celsius and you want to have a new netcdf-file with unit Kelvin, you should use the ''--output.config=cdmWriterConfig.xml'' and define there the new units for the that variable and make sure you defined the datatype of that variable, too.
 +
 +If the input-file has the wrong units, e.g. Celsius while the data in fact is Kelvin, you should use the input.config=... and re-define the units-attribute. The data-values won't be changed.
 +
  
 ===== How to solve problems with a short variable? =====  ===== How to solve problems with a short variable? ===== 
Line 140: Line 148:
 </code> </code>
  
 +=== OpENDAP and slicing ===
  
 +OpENDAP servers are often limited on how much data can be read at per request, e.g. thredds allows only 500MB binary data / 50MB ascii data per request before failing with an error like
 +<code>
 +syntax error, unexpected WORD_STRING, expecting WORD_WORD
 +context: Error { code = 403; message = "Request too big=19011.0 Mbytes, max=500.0"^;};
 +terminate called after throwing an instance of 'MetNoFimex::CDMException'
 +  what():  CDMException: NetCDF: Malformed or inaccessible DAP DATADDS
 +Aborted
 +</code>
  
-===== Why takes linking with ''g++ -g''  ages? =====+Fimex tries to subdevide requests into one unlimited slice per variable. Unfortunately, OpENDAP often does not know about 'unlimited' dimension and one has to add that manually to fimex, e.g. for an unlimited time dimension: ''--input.config=unlimitedTime.ncml'' with contents like: 
 +<code> 
 +<?xml version="1.0" encoding="UTF-8"?> 
 +<netcdf xmlns="http://www.unidata.ucar.edu/namespaces/netcdf/ncml-2.2" 
 +        xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
 + xsi:schemaLocation="http://www.unidata.ucar.edu/namespaces/netcdf/ncml-2.2 http://www.unidata.ucar.edu/schemas/netcdf/ncml-2.2.xsd"> 
 +<dimension name="time" isUnlimited="true" /> 
 +</netcdf> 
 +</code>
  
-This happens with older versions of gcc, seen with 3.3 on debian sarge. 
  
-Use ''-gstabs'' or ''-ggdb'' instead of ''-g''. For debugging, this is better suited to c++ code and works with the GNU utils and most other debugger, too. (But unfortunately  not with /valgrind/.+==== Authentication ==== 
 + 
 +In theory, it should be possible to specify username and password 
 +  - in ''$HOME/.dodsrc'' and ''$HOME/.netrc'' (see below for an example) 
 +  - as part of the URL for example ''https://XXXX:YYYY@thredds.met.no/thredds/dodsC/....nc'' 
 + 
 +Example ''$HOME/.netrc'': 
 +<code> 
 +machine thredds.met.no 
 +    login XXXX 
 +    password YYYY 
 +</code> 
 + 
 +Example ''$HOME/.dodsrc'' (replace ''YOURHOME'' with the value og ''$HOME''): 
 +<code> 
 +HTTP.NETRC=YOURHOME/.netrc 
 + 
 +HTTP.CREDENTIALS.USERNAME=XXXX 
 +HTTP.CREDENTIALS.PASSWORD=YYYY 
 + 
 +HTTP.USERNAME=XXXX 
 +HTTP.PASSWORD=YYYY 
 +</code> 
 + 
 +Unfortunately, this does not always seem to work as expected. 
 +Known status of OpenDAP authentication support (applies to ''fimex'' and ''ncdump''): 
 +  * Specifying the password in the URL does not seem to work when the password contains a ''/'', at least for Ubuntu 18.04. 
 +  * For Ubuntu 16.04 (xenial, EOL), both URL and netrc seem to work. 
 +  * For Ubuntu 18.04 (bionic), username and password cannot be read from netrc, but can be specified in the URL. 
 +  * For Ubuntu 20.04 (focal), username and password can only be read from netrc, but cannot be specified in the URL. 
 +  * For Ubuntu 22.04 (jammy), the only solution known to us is to add a ''.dodsrc'' file in the working directory of the fimex/ncdump process
  
  • fimex/faq.1380805164.txt.gz
  • Last modified: 2022-05-31 09:23:15
  • (external edit)