User Tools

Site Tools


noresm:modeldiagnostics

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
noresm:modeldiagnostics [2017-11-23 20:36:09]
johan.liakka@nersc.no [NorESM Diagnostic Packages]
noresm:modeldiagnostics [2018-08-20 18:03:45] (current)
yanchun.he@nersc.no
Line 1: Line 1:
 ====== Model Diagnostic Tools====== ====== Model Diagnostic Tools======
  
-This page links to tools used for NorESM model evaluation.+This page links to tools used for the NorESM model evaluation.
  
 ===== NorESM Diagnostic Packages ===== ===== NorESM Diagnostic Packages =====
 +
 +Output from the latest NCAR diagnostic pages can be found on nird here:
 +
 +via the web:
 +http://​ns2345k.web.sigma2.no/​noresm_diagnostics/​
 +
 +via the filesystem on nird here:
 +/​projects/​NS2345K/​www/​noresm_diagnostics/​
 +
 +
  
 The diagnostics packages are currently available on NIRD. Each package can be run/​configured from the command line using the program diag_run: The diagnostics packages are currently available on NIRD. Each package can be run/​configured from the command line using the program diag_run:
Line 11: Line 21:
 Program: Program:
 /​projects/​NS2345K/​noresm_diagnostics/​bin/​diag_run /​projects/​NS2345K/​noresm_diagnostics/​bin/​diag_run
-Version: ​171123+Version: ​5.1
 ------------------------------------------------- -------------------------------------------------
 Short description:​ Short description:​
-diag_run is used to configure and run the NorESM diagnostic packages.+A wrapper script for NorESM diagnostic packages.
    
 Basic usage: Basic usage:
Line 24: Line 34:
 -m, --model=MODEL ​                            ​Specify the diagnostics package (REQUIRED). -m, --model=MODEL ​                            ​Specify the diagnostics package (REQUIRED).
                                               Valid arguments:                                               Valid arguments:
-                                                cam  : atmospheric package (AMWG) +                                                cam    : atmospheric package (AMWG) 
-                                                clm  : land package (LMWG) +                                                clm    : land package (LMWG) 
-                                                cice : sea-ice package +                                                cice   ​: sea-ice package 
-                                                ​all  : configure all available packages ​(currently cam, clm and cice; micom and pop are still in development).+                                                ​micom  : ocean package 
 +                                                hamocc : biogeochemistry package 
 +                                                all    ​: configure all available packages.
 -c, -c1, --case=CASE1,​ --case1=CASE1 ​         Test case simulation (OPTIONAL). -c, -c1, --case=CASE1,​ --case1=CASE1 ​         Test case simulation (OPTIONAL).
 -s, -s1, --start_yr=SYR1,​ --start_yr1=SYR1 ​   Start year of test case climatology (OPTIONAL). -s, -s1, --start_yr=SYR1,​ --start_yr1=SYR1 ​   Start year of test case climatology (OPTIONAL).
Line 38: Line 50:
 -i2, --input-dir2=DIR ​                        ​Specify the directory where the control case history files are located (OPTIONAL). -i2, --input-dir2=DIR ​                        ​Specify the directory where the control case history files are located (OPTIONAL).
                                               Default is --input-dir=/​projects/​NS2345K/​noresm/​cases                                               Default is --input-dir=/​projects/​NS2345K/​noresm/​cases
--o, --output-dir=DIR ​                         Specify the directory where the package(s) should be installed ​(OPTIONAL). +-o, --output-dir=DIR ​                         Specify the directory where the package(s) ​the climatology and time-series files should be stored ​(OPTIONAL). 
-                                              Default is --output-dir=/​scratch/johiak/​noresm_diagnostics+                                              Default is --output-dir=/​projects/NS2345K/​noresm_diagnostics/out/$USER
 -p, --passive-mode ​                           Run the script in passive mode: the diagnostic script will be configured but not executed (OPTIONAL). -p, --passive-mode ​                           Run the script in passive mode: the diagnostic script will be configured but not executed (OPTIONAL).
 -t, --type=TYPE ​                              ​Specify climatology or time series diagnostics (OPTIONAL): valid options are --type=climo and --type=time_series. -t, --type=TYPE ​                              ​Specify climatology or time series diagnostics (OPTIONAL): valid options are --type=climo and --type=time_series.
Line 45: Line 57:
 -w, --web-dir=DIR ​                            ​Specify the directory where the html should be published (OPTIONAL). -w, --web-dir=DIR ​                            ​Specify the directory where the html should be published (OPTIONAL).
                                               Default is --web-dir=/​projects/​NS2345K/​www/​noresm_diagnostics                                               Default is --web-dir=/​projects/​NS2345K/​www/​noresm_diagnostics
 +--no-atm ​                                     Run CLM diagnostics without CAM data. Must be used for offline CLM simulations.
    
 Examples: Examples:
-diag_run -m all -c N1850_f19_tn11_exp1 -s 21 -e 50 # model-obs diagnostics of N1850_f19_tn11_exp1 ​for all (available) components. +diag_run -m all -c N1850_f19_tn11_exp1 -s 21 -e 50 # model-obs diagnostics of case=N1850_f19_tn11_exp1 (climatology between yrs 21 and 50for all model components. 
-diag_run -m cam -c N1850_f19_tn11_exp1 -s 21 -e 50 -w /​path/​to/​my/​html # model-obs diagnostics ​for CAM, publish the html in /​path/​to/​my/​html.+diag_run -m cam -c N1850_f19_tn11_exp1 -s 21 -e 50 -w /​path/​to/​my/​html # model-obs diagnostics ​in CAM, publish the html in /​path/​to/​my/​html
 +diag_run -m micom -c N1850_f19_tn11_exp1 -t time_series # model-obs time-series diagnostics in MICOM for all years represented in the model output directory (/​projects/​NS2345K/​noresm/​cases/​N1850_f19_tn11_exp1/​ocn/​hist/​).
 diag_run -m cice -c N1850_f19_tn11_exp1 -s 21 -e 50 -p # configure (but do not run) model-obs diagnostics for CICE. diag_run -m cice -c N1850_f19_tn11_exp1 -s 21 -e 50 -p # configure (but do not run) model-obs diagnostics for CICE.
 diag_run -m clm -c N1850_f19_tn11_exp1 -s 21 -e 50 -i /​input/​directory1 -c2 N1850_f19_tn11_exp2 -s2 21 -e2 50 -i2 /​input/​directory2 # model1-model2 diagnostics for CLM with user-specified history file directories diag_run -m clm -c N1850_f19_tn11_exp1 -s 21 -e 50 -i /​input/​directory1 -c2 N1850_f19_tn11_exp2 -s2 21 -e2 50 -i2 /​input/​directory2 # model1-model2 diagnostics for CLM with user-specified history file directories
-diag_run -m clm -c N1850_f19_tn11_exp1 -s 21 -e 50 -t climo # model-obs climatology diagnostics (no time series) for CLM:+diag_run -m micom -c N1850_f19_tn11_exp1 -s 21 -e 50 -t climo # model-obs climatology diagnostics (no time series) for MICOM:
 diag_run -m cam -o /my/dir # install CAM diagnostics in /my/dir with minimal configuration. diag_run -m cam -o /my/dir # install CAM diagnostics in /my/dir with minimal configuration.
 +diag_run -m micom,​hamocc -c N1850OC_f19_tn11_exp1 -s 21 -e 50 # model-obs diagnostics for MICOM and HAMOCC.
 +diag_run -m clm -c N1850_f19_tn11_clmexp1 -s 71 -e 100 --no-atm # model-obs time-series diagnostics for an offline (uncoupled) CLM simulation.
 +diag_run -m hamocc -c N1850OC_f19_tn11_exp1 -s 31 -e 100 -t time_series # model-obs time-series diagnostics in HAMOCC between yrs 31 and 100.
 </​file>​ </​file>​
-Report any problems, comments or suggestions to Johan Liakka[[mailto:johan.liakka@nersc.no|johan.liakka@nersc.no]]+A comprehensive technical summary of diag_run (pdf){{ :noresm:​diag_run_documentation.pdf |}}
  
 +Report any problems, comments or suggestions to Yanchun He: [[mailto:​yanchun.he@nersc.no|yanchun.he@nersc.no]]
  
-=== **Update to NCL version 6.4.0** ​=== +=== Recent updates === 
-The diagnostics ​packages have been configured ​and developed using NCL version 6.4.0 (the most recent ​version ​on NIRD as of 23.11.2017)The user is strongly recommended ​to use this version, as some bugs have been reported for older versionsTo update NCLadd the following two lines of code to your $HOME/.bashrc on NIRD:+  ​29.08.10. ​Update to v5.1: update NCO/NCL versions to support efficient process of compressed netcdf-files. 
 +  * 29.06.18. Update to v5.0: new fields to HAMOCC and MICOM diagnostics;​ minor fixes to other issues. 
 +  ​20.04.18. Update to v4.3: added new fields to HAMOCC diagnostics. 
 +  ​19.04.18. Update to v4.2: included ability to do time-series diagnostics between two user-specified years. 
 +  * 18.04.18. Update to v4.1: improved climatology and time-series calculations in CLM, and introduction of the --no-atm option to enable ​diagnostics ​for offline CLM simulations. 
 +  * 09.04.18. Update to v4.0: included the HAMOCC diagnostics package. 
 +  * 23.02.18. Update to v3.1: added monthly MLD, seasonal SST/​SSS ​and annual meridional heat/​salinity fluxes to the MICOM diagnostics. 
 +  * 17.01.18. Update to v3.0the first version of MICOM diagnostics has been included. 
 +  * 28.11.17Update ​to v2.0: included a set of time series plots in CAM diagnostics,​ along with an html interfacewhich can be accessed from the index page (sets.htm).  
 + 
 +===Using diag_run with cron=== 
 +If you want to use diag_run with crontab, you first need to load $HOME/.bash_profile,​ i.e.:
 <​file>​ <​file>​
-export NCARG_ROOT=/opt/ncl64 +#​--------------------------------------- 
-export PATH=/opt/ncl64/bin/:${PATH}+# Min Hour Day Month Weekday Command(s) 
 +#​--------------------------------------- 
 +50 09 23 11 * . $HOME/.bash_profile; ​/projects/NS2345K/noresm_diagnostics/bin/diag_run -m cam,cice -c N18_f19_tn11_080617 -s 21 -e 50 -o /scratch/$USER/​noresm_diagnostics2 -w \ 
 +/​projects/​NS2345K/​www/​test -t time_series >& /​scratch/​$USER/​cron_out
 </​file>​ </​file>​
-===== SVN Repository for NorESM Diagnostic Packages ​ ===== 
- 
-(NEW 2015.04.23) 
- 
-The NorESM Diagnostic Packages can be downloaded via svn using the command: ​ 
-  svn export svn://​noresg.norstore.no/​NoresmDiagnosticPackages 
  
  
 +=== Other tips ===
 +It is useful to add diag_run as an alias in $HOME/​.bashrc,​ so that you do not need to write out the whole path every time you run it:
 +<​file>​
 +alias diag_run='/​projects/​NS2345K/​noresm_diagnostics/​bin/​diag_run'​
 +</​file>​
  
-A web-viewer is available at [[https://​webserver1.norstore.uio.no:​8443/​websvn/​wsvn/​NORESG-diagpacks]] (user: guestuser, passw: friendly). 
  
 +=== NorESM diagnostics on GitHub ===
 +The NorESM diagnostics packages and diag_run are included in the Git version control repository:
 +[[https://​github.com/​johiak/​NoresmDiagnostics]]
 ====== Aerosol and Chemistry, Clouds and Forcing Diagnostics ====== ====== Aerosol and Chemistry, Clouds and Forcing Diagnostics ======
  
Line 247: Line 280:
 http://​www.jasmin.ac.uk/​services/​jasmin-analysis-platform/​ http://​www.jasmin.ac.uk/​services/​jasmin-analysis-platform/​
  
 +====== Post analysis and workup of CAM diagnostics output tables ======
 +
 +A tool for post analysis of (multiple) CAM diagnostics ASCII tables can be found in 
 +the following repository:
 +
 +GitHub https://​github.com/​jgliss/​noresm_diag_postproc
 +
 +To get started, please follow the instructions in repository README (displayed in repository). Currently, the main 
 +analysis tool is a jupyter IPython notebook called ​
 +
 +//​**analysis_tool.ipynb**//​ (https://​github.com/​jgliss/​noresm_diag_postproc/​blob/​master/​analysis_tool.ipynb)
 +
 +which includes more detailed instructions about setup and options. ​
 +
 +Use the notebook ​
 +
 +https://​github.com/​jgliss/​noresm_diag_postproc/​blob/​master/​download_tables.ipynb
 +
 +to download local copies of result tables using a list of URL's.
 +
 +
 +**Short summary:​** ​
 +The notebook reads multiple diagnostics files (runs) into one long table and creates heatmap plots
 +of //Bias, RMSE and RMSE relative error// for a subset of variables (rows -> y-axis of heatmap) ​
 +vs. the individual runs (columns -> xaxis). ​
 +
 +**NOTE:** In the current version, you need to download all tables that you are interested in as csv or 
 +ascii into one directory, that is specified in the header of the notebook.
 +
 +Variable groups can be defined in this config file:
 +
 +https://​github.com/​jgliss/​noresm_diag_postproc/​blob/​master/​config/​var_groups.ini
 +
 +**NOTE:** If you add groups to this file in your local copy of the repository, please consider sending
 +the updated to [[mailto:​jonasg@met.no|jonasg@met.no]] or to submit a pull request, so that the remote repository ​
 +remains up to date.
 +
 +**Troubleshooting**
 +
 +If you run into problems, please raise an issue in the repository or contact [[mailto:​jonasg@met.no|jonasg@met.no]]
  
noresm/modeldiagnostics.1511469369.txt.gz · Last modified: 2017-11-23 20:36:09 by johan.liakka@nersc.no