noresm:norstoreesg

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision 		Previous revision
noresm:norstoreesg [2014-04-26 18:45:17]
ingo.bethke@gmail.com 		noresm:norstoreesg [2022-05-31 09:29:32] (current)
Line 1: Line 1:
 +
 +====== Simple online sharing ======
 +
 +===== Share data online =====
 +
 +For simple data sharing, use the script **/projects/NS2345K/tools/wwwpublish**: 
 +<file>
 +Usage:       
 +   wwwpublish <absolute path to data directory> <dataset name> 
 +   or 
 +   wwwpublish <absolute path to data file> <dataset name> 
 +
 +Example: 
 +   wwwpublish /projects/NS2345K/noresm/thredds/CORE2/AMOC CORE2-AMOC 
 +   or
 +   wwwpublish /projects/NS2345K/noresm/noresm2cmor.tbz noresm2cmor 
 +
 +Purpose: 
 +   Publish data online at http://ns2345k.web.nird.no 
 +
 +Description: 
 +   The input path must be absolute and start with /projects/NS2345K.
 +
 +   Only regular files are considered. Symbolic links are skipped.
 +          
 +   All date files must have unique names as the folder stucture is 
 +   flattened during publication. 
 +
 +   Reading permissions of the data files have to be open to everyone. 
 +   However, reading permissions of the folders containing the files 
 +   can be closed. 
 +              
 +   <dataset name> should be one word and not contain any special 
 +   characters other than "-" and "_".           
 +</file>
 +
 +The script provides following: 
 +  *local path to a folder that contains hard-links to the data files 
 +  *URL that points to the online-catalogue where the shared files are exposed 
 +  *download-URL to a wget-script that can be used to download the entire dataset (note that execution permissions have to be changed prior to running the script)   
 +
 +NOTE: The sharing method uses hard-links. This has the advantages that no additional disk space is used and that the sharing happens instantaneous. A removal of the hard-links will unshare the data, but will not affect the original files.   
 +
 +===== Unshare data =====
 +
 +If you don't want to share a dataset anymore, use the script **/projects/NS2345K/tools/wwwunpublish**: 
 +<file>
 +Usage: wwwunpublish <dataset name> 
 +
 +Example: wwwunpublish CORE2-AMOC 
 + 
 +Purpose: Unpublish a dataset from NorStore's www-server 
 +</file>
 +
 +NOTE: Unsharing data will not affect the original data. 
 +
 ====== Data sharing via the Norwegian ESG data portal ====== ====== Data sharing via the Norwegian ESG data portal ======
 +
 +====IMPORTANT: The portal is currently offline for maintenance and its operation is expected to be resumed sometime in October. For an alternative data sharing method see previous section "Simple data sharing". ====
 +
  
 [[http://noresg.norstore.no|{{ :noresm:noresg.png?nolink&512 |}}]] [[http://noresg.norstore.no|{{ :noresm:noresg.png?nolink&512 |}}]]
  
-===== Basic use (unofficial)=====+===== Simplified publishing (unofficial ESG publication)=====
  
 All users of the NorStore project ns2345k have the possibility to share their data via NorStore's ESG data portal [[http://noresg.norstore.no|http://noresg.norstore.no]].  All users of the NorStore project ns2345k have the possibility to share their data via NorStore's ESG data portal [[http://noresg.norstore.no|http://noresg.norstore.no]]. 
Line 15: Line 74:
 Note that the data sharing solution uses hard links to avoid additional use of disk resources. The original data is not touched/modified. Note that the data sharing solution uses hard links to avoid additional use of disk resources. The original data is not touched/modified.
  
-==== Publishing a simple dataset to NorStore's ESG data portal ====+==== Publish a simple dataset to NorStore's ESG data portal ====
  
-For simple data sharing, use the script **/projects/NS2345K/tools/esgpublish**: +For simple data sharing with ESG support, use the script **/projects/NS2345K/tools/esgpublish**: 
 <file> <file>
-Usage: esgpublish <absolute path to data directory> <dataset name> +Usage: esgpublish <absolute path to data directory> <dataset name> [unlisted]
        or         or 
-       esgpublish <absolute path to data file> <dataset name> +       esgpublish <absolute path to data file> <dataset name> [unlisted]
  
 Example: esgpublish /projects/NS2345K/noresm/thredds/CORE2/AMOC CORE2-AMOC  Example: esgpublish /projects/NS2345K/noresm/thredds/CORE2/AMOC CORE2-AMOC 
-         or  +         or 
-         esgpublish /projects/NS2345K/noresm/thredds/CORE2/AMOC/NOIIA_T62_tn11_sr10m60d_02_moc_z_annual_241-300.nc CORE2-MOCZ +         esgpublish /projects/NS2345K/noresm/noresm2cmor.tbz noresm2cmor unlisted  
-         +
 Purpose: Publish data via NorStore's online data portal http://noresg.norstore.no  Purpose: Publish data via NorStore's online data portal http://noresg.norstore.no 
  
-Description: The input path must be absolute and start with /projects/NS2345K or +Description: The input path must be absolute and start with /projects/NS2345K or
              /norstore_osl/projects/NS2345K.                /norstore_osl/projects/NS2345K.  
  
Line 41: Line 100:
              closed.               closed. 
                              
-             <dataset name> should be one word and not contain special +             <dataset name> should be one word and not contain any special 
              characters other than "-" and "_".               characters other than "-" and "_". 
 +           
 +             If a third argument is specified and set to "unlisted" then the 
 +             dataset will be served through the file server but not published 
 +             to the portal.             
 </file>  </file> 
  
-==== Removing a simple dataset from NorStore's ESG data portal ====+==== Remove a simple dataset from NorStore's ESG data portal ====
  
 To unpublish a dataset that previously has been published with esgpublish, use the script **/projects/NS2345K/tools/esgunpublish**:  To unpublish a dataset that previously has been published with esgpublish, use the script **/projects/NS2345K/tools/esgunpublish**: 
Line 56: Line 119:
 </file> </file>
  
-===== Advanced use (official) =====+===== Standard publishing (official ESG publication) =====
  
 Several projects (e.g, CMIP, GeoMIP, etc) require special post-processing and ESG publishing procedures.  Several projects (e.g, CMIP, GeoMIP, etc) require special post-processing and ESG publishing procedures. 
  
 Please contact Ingo Bethke (ingo.bethke[at]uni.no) or Alf Grini (alfg[at]met.no) for assistance. Please contact Ingo Bethke (ingo.bethke[at]uni.no) or Alf Grini (alfg[at]met.no) for assistance.
  • noresm/norstoreesg.1398537917.txt.gz
  • Last modified: 2022-05-31 09:23:24
  • (external edit)