A collection of high-level tape tools is available in **/projects/NS2345K/tools**.
A collection of high-level tape tools is available in **/projects/NS2345K/tools**.
-
==== disk2tape ====
+
==== noresm2tape ====
-
disk2tape is a script for copying a folder from NorStore's disk area to NorStore's tape resource.
+
noresm2tape copies the output of a NorESM case from NorStore's disk area to NorStore's tape resource. The output folder should be organised in the standard CCSM way (e.g., atmospheric output is expected in 'atm/hist'). If that is not the case, please use the command disk2tape instead.
-
Run ''disk2tape -h'' to print detailed instructions:
+
**IMPORTANT:** It is highly recommended to run the script in background using the nohup command, e.g.,
+
"''nohup /projects/NS2345K/tools/noresm2tape <path on disk> <path on tape> &''". Make sure not to forget the "&" at the end of the line. After submitting the script with nohup, it is safe to log out. You will receive a notification email when the transfer to tape is completed.
+
+
+
Run ''/projects/NS2345K/tools/noresm2tape -h'' to print detailed instructions:
<file>
<file>
-
Usage: disk2tape <path on disk> <path on tape> [none|gzip|bzip2]
+
Usage: noresm2tape <path on disk> <path on tape> <chunk size> <replication flag>
Example: noresm2tape /scratch/ingo/mycase /tape/NS2345K/cases/mycase auto replica
-
Purpose: Copies a folder from disk to tape.
+
Purpose: Copies NorESM case output from disk to tape.
-
Description: <path on disk> must be the absolute path to the input folder.
+
Description: <path on disk> and <path on tape> must be full path names (see example).
-
+
+
<chunk size> must be one of: auto, all, 1, 10, 100, or 1000 (numbers
+
indicate simulation years per chunk)
+
+
<replication flag> must be either 'replica' - resulting in two tape copies -
+
or 'noreplica'.
+
The output folder <path on tape> will be created - with parent
The output folder <path on tape> will be created - with parent
directories if necessary - and must not exist before running this
directories if necessary - and must not exist before running this
Line 29:
Line 39:
The input data is staged in form of tar-chuncks in
The input data is staged in form of tar-chuncks in
-
/scratch/ingo/copytotape. The tar-chunks are removed from scratch
+
/scratch/ingo/noresm2tape. The tar-chunks are removed from
-
after success transfer to tape.
+
scratch after successfull transfer to tape.
+
+
Checksums for all input files are computed and stored in a separate
+
checksum file. The checksums are used to verify the tar-chunks.
+
</file>
+
+
+
==== disk2tape ====
+
+
disk2tape copies a folder from NorStore's disk area to NorStore's tape resource. In contrast to noresm2tape, the data folder can have any structure and content.
+
+
**IMPORTANT:** It is highly recommended to run the script in background using the nohup command, e.g.,
+
"''nohup /projects/NS2345K/tools/disk2tape <path on disk> <path on tape> &''". Make sure not to forget the "&" at the end of the line. After submitting the script with nohup, it is safe to log out. You will receive a notification email when the transfer to tape is completed.
+
+
Run ''/projects/NS2345K/tools/disk2tape -h'' to print detailed instructions:
+
<file>
+
Usage: disk2tape <path on disk> <path on tape> [none|gzip|bzip2] [replicate]
Description: <path on disk> must be the absolute path to the input folder.
+
+
<path on tape> will be created - with parent directories if
+
necessary - and must not exist before running this script.
+
+
gzip or bzip2 compression will be performed if the third argument is
+
set to "gzip" or "bzip2", respectively. The default is no compression.
+
+
If the forth argument is set to "replicate" then a second copy of
+
the data will be stored on another physical tape medium.
+
The path of the additional copy differs from the path of the first
+
copy by that /tape is replaced with /replica.
+
+
The input data is staged in form of tar-chuncks in
+
/scratch/$USER/`basename $0`. The tar-chunks are removed from
+
scratch after success transfer to tape.
The size of a single tar-chunk is about 15Gb if no compression is
The size of a single tar-chunk is about 15Gb if no compression is
Line 42:
Line 89:
==== tape2disk ====
==== tape2disk ====
-
tape2disk is a script for retrieve a folder from NorStore's tape area. All tar-archieves are unpacked.
+
tape2disk copies a folder that had been put on tape with either noresm2tape or disk2tape. All tar-archieves found in the top-level folder are unpacked while tar-files stored within tar-files are not inflated.
-
Run ''tape2disk -h'' to print detailed instructions:
+
Run ''/projects/NS2345K/tools/tape2disk -h'' to print detailed instructions:
<file>
<file>
-
Usage: tape2disk <path on tape> <path on disk>
+
Usage: /projects/NS2345K/tools/tape2disk <path on tape> <path on disk>