noresm:modeldiagnostics

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
Next revision Both sides next revision
noresm:modeldiagnostics [2017-11-26 13:00:04]
johan.liakka@nersc.no [NorESM Diagnostic Packages]
noresm:modeldiagnostics [2018-08-20 18:03:45]
yanchun.he@nersc.no
Line 1: Line 1:
 ====== Model Diagnostic Tools====== ====== Model Diagnostic Tools======
  
-This page links to tools used for NorESM model evaluation.+This page links to tools used for the NorESM model evaluation.
  
 ===== NorESM Diagnostic Packages ===== ===== NorESM Diagnostic Packages =====
 +
 +Output from the latest NCAR diagnostic pages can be found on nird here:
 +
 +via the web:
 +http://ns2345k.web.sigma2.no/noresm_diagnostics/
 +
 +via the filesystem on nird here:
 +/projects/NS2345K/www/noresm_diagnostics/
 +
 +
  
 The diagnostics packages are currently available on NIRD. Each package can be run/configured from the command line using the program diag_run: The diagnostics packages are currently available on NIRD. Each package can be run/configured from the command line using the program diag_run:
Line 11: Line 21:
 Program: Program:
 /projects/NS2345K/noresm_diagnostics/bin/diag_run /projects/NS2345K/noresm_diagnostics/bin/diag_run
-Version: 1.0.0+Version: 5.1
 ------------------------------------------------- -------------------------------------------------
 Short description: Short description:
-diag_run is used to configure and run the NorESM diagnostic packages.+A wrapper script for NorESM diagnostic packages.
    
 Basic usage: Basic usage:
Line 24: Line 34:
 -m, --model=MODEL                             Specify the diagnostics package (REQUIRED). -m, --model=MODEL                             Specify the diagnostics package (REQUIRED).
                                               Valid arguments:                                               Valid arguments:
-                                                cam  : atmospheric package (AMWG) +                                                cam    : atmospheric package (AMWG) 
-                                                clm  : land package (LMWG) +                                                clm    : land package (LMWG) 
-                                                cice : sea-ice package +                                                cice   : sea-ice package 
-                                                all  : configure all available packages (currently cam, clm and cice; micom and pop are still in development).+                                                micom  : ocean package 
 +                                                hamocc : biogeochemistry package 
 +                                                all    : configure all available packages.
 -c, -c1, --case=CASE1, --case1=CASE1          Test case simulation (OPTIONAL). -c, -c1, --case=CASE1, --case1=CASE1          Test case simulation (OPTIONAL).
 -s, -s1, --start_yr=SYR1, --start_yr1=SYR1    Start year of test case climatology (OPTIONAL). -s, -s1, --start_yr=SYR1, --start_yr1=SYR1    Start year of test case climatology (OPTIONAL).
Line 38: Line 50:
 -i2, --input-dir2=DIR                         Specify the directory where the control case history files are located (OPTIONAL). -i2, --input-dir2=DIR                         Specify the directory where the control case history files are located (OPTIONAL).
                                               Default is --input-dir=/projects/NS2345K/noresm/cases                                               Default is --input-dir=/projects/NS2345K/noresm/cases
--o, --output-dir=DIR                          Specify the directory where the package(s) should be installed (OPTIONAL). +-o, --output-dir=DIR                          Specify the directory where the package(s) the climatology and time-series files should be stored (OPTIONAL). 
-                                              Default is --output-dir=/scratch/johiak/noresm_diagnostics+                                              Default is --output-dir=/projects/NS2345K/noresm_diagnostics/out/$USER
 -p, --passive-mode                            Run the script in passive mode: the diagnostic script will be configured but not executed (OPTIONAL). -p, --passive-mode                            Run the script in passive mode: the diagnostic script will be configured but not executed (OPTIONAL).
 -t, --type=TYPE                               Specify climatology or time series diagnostics (OPTIONAL): valid options are --type=climo and --type=time_series. -t, --type=TYPE                               Specify climatology or time series diagnostics (OPTIONAL): valid options are --type=climo and --type=time_series.
Line 45: Line 57:
 -w, --web-dir=DIR                             Specify the directory where the html should be published (OPTIONAL). -w, --web-dir=DIR                             Specify the directory where the html should be published (OPTIONAL).
                                               Default is --web-dir=/projects/NS2345K/www/noresm_diagnostics                                               Default is --web-dir=/projects/NS2345K/www/noresm_diagnostics
 +--no-atm                                      Run CLM diagnostics without CAM data. Must be used for offline CLM simulations.
    
 Examples: Examples:
-diag_run -m all -c N1850_f19_tn11_exp1 -s 21 -e 50 # model-obs diagnostics of N1850_f19_tn11_exp1 for all (available) components. +diag_run -m all -c N1850_f19_tn11_exp1 -s 21 -e 50 # model-obs diagnostics of case=N1850_f19_tn11_exp1 (climatology between yrs 21 and 50for all model components. 
-diag_run -m cam -c N1850_f19_tn11_exp1 -s 21 -e 50 -w /path/to/my/html # model-obs diagnostics for CAM, publish the html in /path/to/my/html.+diag_run -m cam -c N1850_f19_tn11_exp1 -s 21 -e 50 -w /path/to/my/html # model-obs diagnostics in CAM, publish the html in /path/to/my/html
 +diag_run -m micom -c N1850_f19_tn11_exp1 -t time_series # model-obs time-series diagnostics in MICOM for all years represented in the model output directory (/projects/NS2345K/noresm/cases/N1850_f19_tn11_exp1/ocn/hist/).
 diag_run -m cice -c N1850_f19_tn11_exp1 -s 21 -e 50 -p # configure (but do not run) model-obs diagnostics for CICE. diag_run -m cice -c N1850_f19_tn11_exp1 -s 21 -e 50 -p # configure (but do not run) model-obs diagnostics for CICE.
 diag_run -m clm -c N1850_f19_tn11_exp1 -s 21 -e 50 -i /input/directory1 -c2 N1850_f19_tn11_exp2 -s2 21 -e2 50 -i2 /input/directory2 # model1-model2 diagnostics for CLM with user-specified history file directories diag_run -m clm -c N1850_f19_tn11_exp1 -s 21 -e 50 -i /input/directory1 -c2 N1850_f19_tn11_exp2 -s2 21 -e2 50 -i2 /input/directory2 # model1-model2 diagnostics for CLM with user-specified history file directories
-diag_run -m clm -c N1850_f19_tn11_exp1 -s 21 -e 50 -t climo # model-obs climatology diagnostics (no time series) for CLM:+diag_run -m micom -c N1850_f19_tn11_exp1 -s 21 -e 50 -t climo # model-obs climatology diagnostics (no time series) for MICOM:
 diag_run -m cam -o /my/dir # install CAM diagnostics in /my/dir with minimal configuration. diag_run -m cam -o /my/dir # install CAM diagnostics in /my/dir with minimal configuration.
 +diag_run -m micom,hamocc -c N1850OC_f19_tn11_exp1 -s 21 -e 50 # model-obs diagnostics for MICOM and HAMOCC.
 +diag_run -m clm -c N1850_f19_tn11_clmexp1 -s 71 -e 100 --no-atm # model-obs time-series diagnostics for an offline (uncoupled) CLM simulation.
 +diag_run -m hamocc -c N1850OC_f19_tn11_exp1 -s 31 -e 100 -t time_series # model-obs time-series diagnostics in HAMOCC between yrs 31 and 100.
 </file> </file>
-Report any problems, comments or suggestions to Johan Liakka[[mailto:johan.liakka@nersc.no|johan.liakka@nersc.no]]+A comprehensive technical summary of diag_run (pdf){{ :noresm:diag_run_documentation.pdf |}}
  
 +Report any problems, comments or suggestions to Yanchun He: [[mailto:yanchun.he@nersc.no|yanchun.he@nersc.no]]
  
-=== Update to NCL version 6.4.0 === +=== Recent updates === 
-The diagnostics packages have been configured and developed using NCL version 6.4.0 (the most recent version on NIRD as of 23.11.2017)The user is strongly recommended to use this version, as some bugs have been reported for older versionsTo update NCLadd the following two lines of code to your $HOME/.bashrc on NIRD+  * 29.08.10. Update to v5.1: update NCO/NCL versions to support efficient process of compressed netcdf-files. 
-<file> +  * 29.06.18. Update to v5.0: new fields to HAMOCC and MICOM diagnostics; minor fixes to other issues. 
-export NCARG_ROOT=/opt/ncl64 +  * 20.04.18. Update to v4.3: added new fields to HAMOCC diagnostics. 
-export PATH=/opt/ncl64/bin/:${PATH} +  * 19.04.18Update to v4.2: included ability to do time-series diagnostics between two user-specified years. 
-</file> +  * 18.04.18. Update to v4.1: improved climatology and time-series calculations in CLMand introduction of the --no-atm option to enable diagnostics for offline CLM simulations. 
 +  * 09.04.18. Update to v4.0included the HAMOCC diagnostics package. 
 +  * 23.02.18. Update to v3.1: added monthly MLD, seasonal SST/SSS and annual meridional heat/salinity fluxes to the MICOM diagnostics. 
 +  * 17.01.18. Update to v3.0the first version of MICOM diagnostics has been included. 
 +  * 28.11.17. Update to v2.0: included a set of time series plots in CAM diagnostics, along with an html interface, which can be accessed from the index page (sets.htm). 
  
 ===Using diag_run with cron=== ===Using diag_run with cron===
-If you want to use diag_run with crontab, you first need to load $HOME/.bash_profile and export the variable RUN_BY_CRONe.g.:+If you want to use diag_run with crontab, you first need to load $HOME/.bash_profile, i.e.:
 <file> <file>
 #--------------------------------------- #---------------------------------------
 # Min Hour Day Month Weekday Command(s) # Min Hour Day Month Weekday Command(s)
 #--------------------------------------- #---------------------------------------
-50 09 23 11 * . $HOME/.bash_profile; export RUN_BY_CRON="TRUE"; /projects/NS2345K/noresm_diagnostics/bin/diag_run -m cam,cice -c N18_f19_tn11_080617 -s 21 -e 50 -o /scratch/johiak/noresm_diagnostics2 -w \ +50 09 23 11 * . $HOME/.bash_profile; /projects/NS2345K/noresm_diagnostics/bin/diag_run -m cam,cice -c N18_f19_tn11_080617 -s 21 -e 50 -o /scratch/$USER/noresm_diagnostics2 -w \ 
-/projects/NS2345K/www/test -t time_series >& /scratch/johiak/cron_out+/projects/NS2345K/www/test -t time_series >& /scratch/$USER/cron_out
 </file> </file>
  
Line 86: Line 107:
 The NorESM diagnostics packages and diag_run are included in the Git version control repository: The NorESM diagnostics packages and diag_run are included in the Git version control repository:
 [[https://github.com/johiak/NoresmDiagnostics]] [[https://github.com/johiak/NoresmDiagnostics]]
- 
 ====== Aerosol and Chemistry, Clouds and Forcing Diagnostics ====== ====== Aerosol and Chemistry, Clouds and Forcing Diagnostics ======
  
Line 260: Line 280:
 http://www.jasmin.ac.uk/services/jasmin-analysis-platform/ http://www.jasmin.ac.uk/services/jasmin-analysis-platform/
  
 +====== Post analysis and workup of CAM diagnostics output tables ======
 +
 +A tool for post analysis of (multiple) CAM diagnostics ASCII tables can be found in 
 +the following repository:
 +
 +GitHub https://github.com/jgliss/noresm_diag_postproc
 +
 +To get started, please follow the instructions in repository README (displayed in repository). Currently, the main 
 +analysis tool is a jupyter IPython notebook called 
 +
 +//**analysis_tool.ipynb**// (https://github.com/jgliss/noresm_diag_postproc/blob/master/analysis_tool.ipynb)
 +
 +which includes more detailed instructions about setup and options. 
 +
 +Use the notebook 
 +
 +https://github.com/jgliss/noresm_diag_postproc/blob/master/download_tables.ipynb
 +
 +to download local copies of result tables using a list of URL's.
 +
 +
 +**Short summary:** 
 +The notebook reads multiple diagnostics files (runs) into one long table and creates heatmap plots
 +of //Bias, RMSE and RMSE relative error// for a subset of variables (rows -> y-axis of heatmap) 
 +vs. the individual runs (columns -> xaxis). 
 +
 +**NOTE:** In the current version, you need to download all tables that you are interested in as csv or 
 +ascii into one directory, that is specified in the header of the notebook.
 +
 +Variable groups can be defined in this config file:
 +
 +https://github.com/jgliss/noresm_diag_postproc/blob/master/config/var_groups.ini
 +
 +**NOTE:** If you add groups to this file in your local copy of the repository, please consider sending
 +the updated to [[mailto:jonasg@met.no|jonasg@met.no]] or to submit a pull request, so that the remote repository 
 +remains up to date.
 +
 +**Troubleshooting**
 +
 +If you run into problems, please raise an issue in the repository or contact [[mailto:jonasg@met.no|jonasg@met.no]]
  
  • noresm/modeldiagnostics.txt
  • Last modified: 2022-05-31 09:29:32
  • (external edit)