User Tools

Site Tools


noresm:modeldiagnostics

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
noresm:modeldiagnostics [2017-11-23 19:05:14]
johan.liakka@nersc.no [Model Diagnostic Tools]
noresm:modeldiagnostics [2019-08-09 07:46:23] (current)
yanchun.he@nersc.no
Line 1: Line 1:
 ====== Model Diagnostic Tools====== ====== Model Diagnostic Tools======
  
-This page links to tools used for NorESM model evaluation.+This page links to tools used for the NorESM model evaluation.
  
 ===== NorESM Diagnostic Packages ===== ===== NorESM Diagnostic Packages =====
  
-Please, refer to GUIDE_ReadME_first and the ReadME files in each of the Model Components Diagnostic Tools (Packages) locations below for more details.+Output from the latest NCAR diagnostic pages can be found on nird here:
  
-<​file>​ +via the web: 
-Usagenohup [your work directory path]/[amwg/ice_diag]_template.csh >& out & +http://ns2345k.web.sigma2.no/​noresm_diagnostics/​
  
-PurposeModel solution validation, evaluation, and analysis ​ +via the filesystem on nird here
-         compared to observations and reference model simulation.  ​+/​projects/​NS2345K/​www/​noresm_diagnostics/​
  
-Description:​ Automated diagnostic packages for each of the NorESM model components (atmosphere,​ sea ice, land, ocean) 
-             based on the NCAR CESM diagnostics packages. ​ 
-             ​Written in NCL, NCO and C-Shell. Output is in the form of: .ps or .png plots and web-pages. 
  
-The packages are installed and stored in 4 locations; please use them on Cruncher or Grunch when possible: 
  
-On cruncher.uio.no/norstore.uio.no in:+The diagnostics packages are currently available on NIRDEach package can be run/configured from the command line using the program diag_run:
  
-/​projects/​NS2345K/​diagnostics/​Packages+<​file>​ 
 +------------------------------------------------- 
 +Program: 
 +/​projects/​NS2345K/​noresm_diagnostics/​bin/​diag_run 
 +Version: 6.0 
 +------------------------------------------------- 
 +Short description:​ 
 +A wrapper script for NorESM diagnostic packages. 
 +  
 +Basic usage: 
 +diag_run -m [model] -c [test case name] -s [test case start yr] -e [test case end yr] # Run model-obs diagnostics 
 +diag_run -m [model] -c [test case name] -s [test case start yr] -e [test case end yr] -c2 [cntl case name] -s2 [cntl case start yr] -e2 [cntl case end yr] # Run model1-model2 diagnostics 
 +nohup /​projects/​NS2345K/​noresm_diagnostics/​bin/​diag_run -m [model] -c [test case name] -s [test case start yr] -e [test case end yr] &> out & # Run model-obs diagnostics in the background with nohup 
 +  
 +Command-line options: 
 +-m, --model=MODEL ​                            ​Specify the diagnostics package (REQUIRED). 
 +                                              Valid arguments:​ 
 +                                                cam    : atmospheric package (AMWG) 
 +                                                clm    : land package (LMWG) 
 +                                                cice   : sea-ice package 
 +                                                micom  : ocean package 
 +                                                hamocc : biogeochemistry package 
 +                                                all    : configure all available packages. 
 +-c, -c1, --case=CASE1,​ --case1=CASE1 ​         Test case simulation (OPTIONAL). 
 +-s, -s1, --start_yr=SYR1,​ --start_yr1=SYR1 ​   Start year of test case climatology (OPTIONAL). 
 +-e, -e1, --end_yr=EYR1,​ --end_yr1=EYR1 ​       End year of test case climatology (OPTIONAL). 
 +-c2, --case2=CASE2 ​                           Control case simulation (OPTIONAL). 
 +-s2, --start_yr2=SYR2 ​                        Start year of control case climatology (OPTIONAL). 
 +-e2, --end_yr2=EYR2 ​                          End year of control case climatology (OPTIONAL). 
 +-i, -i1, --input-dir=DIR,​ --input-dir1=DIR ​   Specify the directory where the test case history files are located (OPTIONAL). 
 +                                              Default is --input-dir=/​projects/​NS2345K/​noresm/​cases 
 +-i2, --input-dir2=DIR ​                        ​Specify the directory where the control case history files are located (OPTIONAL). 
 +                                              Default is --input-dir=/​projects/​NS2345K/​noresm/​cases 
 +-o, --output-dir=DIR ​                         Specify the directory where the package(s) the climatology and time-series files should be stored (OPTIONAL). 
 +                                              Default is --output-dir=/​projects/​NS2345K/​noresm_diagnostics/​out/​$USER 
 +-p, --passive-mode ​                           Run the script in passive mode: the diagnostic script will be configured but not executed (OPTIONAL). 
 +-t, --type=TYPE ​                              ​Specify climatology or time series diagnostics (OPTIONAL): valid options are --type=climo and --type=time_series. 
 +                                              Default is to run both. Note that the time series are computed over the entire simulation. 
 +-w, --web-dir=DIR ​                            ​Specify the directory where the html should be published (OPTIONAL). 
 +                                              Default is --web-dir=/​projects/​NS2345K/​www/​noresm_diagnostics 
 +--no-atm ​                                     Run CLM diagnostics without CAM data. Must be used for offline CLM simulations. 
 +  
 +Examples: 
 +diag_run -m all -c N1850_f19_tn11_exp1 -s 21 -e 50 # model-obs diagnostics of case=N1850_f19_tn11_exp1 (climatology between yrs 21 and 50) for all model components. 
 +diag_run -m cam -c N1850_f19_tn11_exp1 -s 21 -e 50 -w /​path/​to/​my/​html # model-obs diagnostics in CAM, publish the html in /​path/​to/​my/​html. 
 +diag_run -m micom -c N1850_f19_tn11_exp1 -t time_series # model-obs time-series diagnostics in MICOM for all years represented in the model output directory (/​projects/​NS2345K/​noresm/​cases/​N1850_f19_tn11_exp1/​ocn/​hist/​). 
 +diag_run -m cice -c N1850_f19_tn11_exp1 -s 21 -e 50 -p # configure (but do not run) model-obs diagnostics for CICE. 
 +diag_run -m clm -c N1850_f19_tn11_exp1 -s 21 -e 50 -i /​input/​directory1 -c2 N1850_f19_tn11_exp2 -s2 21 -e2 50 -i2 /​input/​directory2 # model1-model2 diagnostics for CLM with user-specified history file directories 
 +diag_run -m micom -c N1850_f19_tn11_exp1 -s 21 -e 50 -t climo # model-obs climatology diagnostics (no time series) for MICOM: 
 +diag_run -m cam -o /my/dir # install CAM diagnostics in /my/dir with minimal configuration. 
 +diag_run -m micom,​hamocc -c N1850OC_f19_tn11_exp1 -s 21 -e 50 # model-obs diagnostics for MICOM and HAMOCC. 
 +diag_run -m clm -c N1850_f19_tn11_clmexp1 -s 71 -e 100 --no-atm # model-obs time-series diagnostics for an offline (uncoupled) CLM simulation. 
 +diag_run -m hamocc -c N1850OC_f19_tn11_exp1 -s 31 -e 100 -t time_series # model-obs time-series ​diagnostics ​in HAMOCC between yrs 31 and 100. 
 +</file> 
 +A comprehensive technical summary of diag_run (pdf): {{ :​noresm:​diag_run_documentation.pdf |}}
  
-On grunch.bccs.uib.no/hexagon.bccs.uib.no in:+Report any problems, comments or suggestions to Yanchun He: [[mailto:​yanchun.he@nersc.no|yanchun.he@nersc.no]]
  
-/work-common/shared/noresm/​diagnostics/​Packages+=== Recent updates === 
 +  * 08.08.19. Note: NS2345K is recovered now, please use the normal ​/projects/NS2345K/noresm_diagnostics/bin/​diag_run;​ put the webpage output to /​projects/​NS2345K/​ and view the results at: http://​ns2345k.web.sigma2.no/​noresm_diagnostics/​ 
 +  * 08.08.19. Update to v6.0: add support to TS-derived mlts, global averages: tempga,​salnga,​sstga and sssga, updated volume transports, support extend Atlantic-Arctic overturning (mmflxd) for the NorESM2 version. And other minor changes 
 +  * 29.08.18. Update to v5.1: update NCO/NCL versions to support efficient process of compressed netcdf-4 files. 
 +  * 29.06.18. Update to v5.0: new fields to HAMOCC and MICOM diagnostics; minor fixes to other issues. 
 +  * 20.04.18. Update to v4.3: added new fields to HAMOCC diagnostics. 
 +  * 19.04.18. Update to v4.2: included ability to do time-series diagnostics between two user-specified years. 
 +  * 18.04.18. Update to v4.1: improved climatology and time-series calculations in CLM, and introduction of the --no-atm option to enable diagnostics for offline CLM simulations. 
 +  * 09.04.18. Update to v4.0: included the HAMOCC diagnostics package. 
 +  * 23.02.18. Update to v3.1: added monthly MLD, seasonal SST/SSS and annual meridional heat/​salinity fluxes to the MICOM diagnostics. 
 +  * 17.01.18. Update to v3.0: the first version of MICOM diagnostics has been included. 
 +  * 28.11.17. Update to v2.0: included a set of time series plots in CAM diagnostics,​ along with an html interface, which can be accessed from the index page (sets.htm). ​
  
-On vilje.hpc.ntnu.no in:+===Using diag_run with cron=== 
 +If you want to use diag_run with crontab, you first need to load $HOME/.bash_profile,​ i.e.: 
 +<​file>​ 
 +#​--------------------------------------- 
 +# Min Hour Day Month Weekday Command(s) 
 +#​--------------------------------------- 
 +50 09 23 11 * . $HOME/​.bash_profile;​ /​projects/​NS2345K/​noresm_diagnostics/​bin/​diag_run -m cam,cice -c N18_f19_tn11_080617 -s 21 -e 50 -o /​scratch/​$USER/​noresm_diagnostics2 -w \ 
 +/​projects/​NS2345K/​www/​test -t time_series >& /​scratch/​$USER/​cron_out 
 +</​file>​
  
-/​work/​shared/​noresm/​diagnostics/​Packages 
- 
-There are 6 diagnostic tools CAM_DIAG (ready for use), CICE_DIAG (ready for use), CLM_DIAG (in testing), 
-POP_DIAG (in testing), MICOM_DIAG (in development) and CVDP_DIAG (the new NCAR Climate Package, in testing) 
- 
-Currently only the CAM (AMWG) and the CICE packages are ported and ready to use.  
-You will find for each of those c-shell macroscript *_template.csh.[machine] 
-Copy it in your working directory, change the paths and file names in the user-defined part of it and run. 
  
 +=== Other tips ===
 +It is useful to add diag_run as an alias in $HOME/​.bashrc,​ so that you do not need to write out the whole path every time you run it:
 +<​file>​
 +alias diag_run='/​projects/​NS2345K/​noresm_diagnostics/​bin/​diag_run'​
 </​file>​ </​file>​
-Report any problems or suggestions to Detelina Ivanova: [[mailto:​detelina.ivanova@nersc.no|detelina.ivanova@nersc.no]] 
- 
- 
-===== MICOM Diagnostic Package ===== 
- 
- (NEW 2015.04.23) ​ 
-First version of MICOM diagnostic package is available on grunch/​norstore/​hexagon/​vilje. Please find more details ​ {{:​noresm:​detelinaivanova_noresm_diagnostics_rg1_032515.pdf|here}}. 
- 
-===== SVN Repository for NorESM Diagnostic Packages ​ ===== 
- 
-(NEW 2015.04.23) 
- 
-The NorESM Diagnostic Packages can be downloaded via svn using the command: ​ 
-  svn export svn://​noresg.norstore.no/​NoresmDiagnosticPackages 
- 
- 
  
-A web-viewer is available at [[https://​webserver1.norstore.uio.no:​8443/​websvn/​wsvn/​NORESG-diagpacks]] (user: guestuser, passw: friendly). 
  
 +=== NorESM diagnostics on GitHub ===
 +The NorESM diagnostics packages and diag_run are included in the Git version control repository:
 +[[https://​github.com/​NordicESMhub/​NoresmDiagnostics]]
 ====== Aerosol and Chemistry, Clouds and Forcing Diagnostics ====== ====== Aerosol and Chemistry, Clouds and Forcing Diagnostics ======
  
Line 231: Line 282:
 http://​www.jasmin.ac.uk/​services/​jasmin-analysis-platform/​ http://​www.jasmin.ac.uk/​services/​jasmin-analysis-platform/​
  
 +====== Post analysis and workup of CAM diagnostics output tables ======
 +
 +A tool for post analysis of (multiple) CAM diagnostics ASCII tables can be found in 
 +the following repository:
 +
 +GitHub https://​github.com/​jgliss/​noresm_diag_postproc
 +
 +To get started, please follow the instructions in repository README (displayed in repository). Currently, the main 
 +analysis tool is a jupyter IPython notebook called ​
 +
 +//​**analysis_tool.ipynb**//​ (https://​github.com/​jgliss/​noresm_diag_postproc/​blob/​master/​analysis_tool.ipynb)
 +
 +which includes more detailed instructions about setup and options. ​
 +
 +Use the notebook ​
 +
 +https://​github.com/​jgliss/​noresm_diag_postproc/​blob/​master/​download_tables.ipynb
 +
 +to download local copies of result tables using a list of URL's.
 +
 +
 +**Short summary:​** ​
 +The notebook reads multiple diagnostics files (runs) into one long table and creates heatmap plots
 +of //Bias, RMSE and RMSE relative error// for a subset of variables (rows -> y-axis of heatmap) ​
 +vs. the individual runs (columns -> xaxis). ​
 +
 +**NOTE:** In the current version, you need to download all tables that you are interested in as csv or 
 +ascii into one directory, that is specified in the header of the notebook.
 +
 +Variable groups can be defined in this config file:
 +
 +https://​github.com/​jgliss/​noresm_diag_postproc/​blob/​master/​config/​var_groups.ini
 +
 +**NOTE:** If you add groups to this file in your local copy of the repository, please consider sending
 +the updated to [[mailto:​jonasg@met.no|jonasg@met.no]] or to submit a pull request, so that the remote repository ​
 +remains up to date.
 +
 +**Troubleshooting**
  
 +If you run into problems, please raise an issue in the repository or contact [[mailto:​jonasg@met.no|jonasg@met.no]]
noresm/modeldiagnostics.1511463914.txt.gz · Last modified: 2017-11-23 19:05:14 by johan.liakka@nersc.no