User Tools

Site Tools


noresm:modeldiagnostics

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
noresm:modeldiagnostics [2017-11-28 10:01:30]
johan.liakka@nersc.no [NorESM Diagnostic Packages]
noresm:modeldiagnostics [2019-08-09 07:46:23]
yanchun.he@nersc.no
Line 1: Line 1:
 ====== Model Diagnostic Tools====== ====== Model Diagnostic Tools======
  
-This page links to tools used for NorESM model evaluation.+This page links to tools used for the NorESM model evaluation.
  
 ===== NorESM Diagnostic Packages ===== ===== NorESM Diagnostic Packages =====
 +
 +Output from the latest NCAR diagnostic pages can be found on nird here:
 +
 +via the web:
 +http://​ns2345k.web.sigma2.no/​noresm_diagnostics/​
 +
 +via the filesystem on nird here:
 +/​projects/​NS2345K/​www/​noresm_diagnostics/​
 +
 +
  
 The diagnostics packages are currently available on NIRD. Each package can be run/​configured from the command line using the program diag_run: The diagnostics packages are currently available on NIRD. Each package can be run/​configured from the command line using the program diag_run:
Line 11: Line 21:
 Program: Program:
 /​projects/​NS2345K/​noresm_diagnostics/​bin/​diag_run /​projects/​NS2345K/​noresm_diagnostics/​bin/​diag_run
-Version: ​2.0+Version: ​6.0
 ------------------------------------------------- -------------------------------------------------
 Short description:​ Short description:​
-diag_run is used to configure and run the NorESM diagnostic packages.+A wrapper script for NorESM diagnostic packages.
    
 Basic usage: Basic usage:
Line 24: Line 34:
 -m, --model=MODEL ​                            ​Specify the diagnostics package (REQUIRED). -m, --model=MODEL ​                            ​Specify the diagnostics package (REQUIRED).
                                               Valid arguments:                                               Valid arguments:
-                                                cam  : atmospheric package (AMWG) +                                                cam    : atmospheric package (AMWG) 
-                                                clm  : land package (LMWG) +                                                clm    : land package (LMWG) 
-                                                cice : sea-ice package +                                                cice   ​: sea-ice package 
-                                                ​all  : configure all available packages ​(currently cam, clm and cice; micom and pop are still in development).+                                                ​micom  : ocean package 
 +                                                hamocc : biogeochemistry package 
 +                                                all    ​: configure all available packages.
 -c, -c1, --case=CASE1,​ --case1=CASE1 ​         Test case simulation (OPTIONAL). -c, -c1, --case=CASE1,​ --case1=CASE1 ​         Test case simulation (OPTIONAL).
 -s, -s1, --start_yr=SYR1,​ --start_yr1=SYR1 ​   Start year of test case climatology (OPTIONAL). -s, -s1, --start_yr=SYR1,​ --start_yr1=SYR1 ​   Start year of test case climatology (OPTIONAL).
Line 38: Line 50:
 -i2, --input-dir2=DIR ​                        ​Specify the directory where the control case history files are located (OPTIONAL). -i2, --input-dir2=DIR ​                        ​Specify the directory where the control case history files are located (OPTIONAL).
                                               Default is --input-dir=/​projects/​NS2345K/​noresm/​cases                                               Default is --input-dir=/​projects/​NS2345K/​noresm/​cases
--o, --output-dir=DIR ​                         Specify the directory where the package(s) should be installed ​(OPTIONAL). +-o, --output-dir=DIR ​                         Specify the directory where the package(s) ​the climatology and time-series files should be stored ​(OPTIONAL). 
-                                              Default is --output-dir=/​scratch/johiak/​noresm_diagnostics+                                              Default is --output-dir=/​projects/NS2345K/​noresm_diagnostics/out/$USER
 -p, --passive-mode ​                           Run the script in passive mode: the diagnostic script will be configured but not executed (OPTIONAL). -p, --passive-mode ​                           Run the script in passive mode: the diagnostic script will be configured but not executed (OPTIONAL).
 -t, --type=TYPE ​                              ​Specify climatology or time series diagnostics (OPTIONAL): valid options are --type=climo and --type=time_series. -t, --type=TYPE ​                              ​Specify climatology or time series diagnostics (OPTIONAL): valid options are --type=climo and --type=time_series.
Line 45: Line 57:
 -w, --web-dir=DIR ​                            ​Specify the directory where the html should be published (OPTIONAL). -w, --web-dir=DIR ​                            ​Specify the directory where the html should be published (OPTIONAL).
                                               Default is --web-dir=/​projects/​NS2345K/​www/​noresm_diagnostics                                               Default is --web-dir=/​projects/​NS2345K/​www/​noresm_diagnostics
 +--no-atm ​                                     Run CLM diagnostics without CAM data. Must be used for offline CLM simulations.
    
 Examples: Examples:
-diag_run -m all -c N1850_f19_tn11_exp1 -s 21 -e 50 # model-obs diagnostics of N1850_f19_tn11_exp1 ​for all (available) components. +diag_run -m all -c N1850_f19_tn11_exp1 -s 21 -e 50 # model-obs diagnostics of case=N1850_f19_tn11_exp1 (climatology between yrs 21 and 50for all model components. 
-diag_run -m cam -c N1850_f19_tn11_exp1 -s 21 -e 50 -w /​path/​to/​my/​html # model-obs diagnostics ​for CAM, publish the html in /​path/​to/​my/​html.+diag_run -m cam -c N1850_f19_tn11_exp1 -s 21 -e 50 -w /​path/​to/​my/​html # model-obs diagnostics ​in CAM, publish the html in /​path/​to/​my/​html
 +diag_run -m micom -c N1850_f19_tn11_exp1 -t time_series # model-obs time-series diagnostics in MICOM for all years represented in the model output directory (/​projects/​NS2345K/​noresm/​cases/​N1850_f19_tn11_exp1/​ocn/​hist/​).
 diag_run -m cice -c N1850_f19_tn11_exp1 -s 21 -e 50 -p # configure (but do not run) model-obs diagnostics for CICE. diag_run -m cice -c N1850_f19_tn11_exp1 -s 21 -e 50 -p # configure (but do not run) model-obs diagnostics for CICE.
 diag_run -m clm -c N1850_f19_tn11_exp1 -s 21 -e 50 -i /​input/​directory1 -c2 N1850_f19_tn11_exp2 -s2 21 -e2 50 -i2 /​input/​directory2 # model1-model2 diagnostics for CLM with user-specified history file directories diag_run -m clm -c N1850_f19_tn11_exp1 -s 21 -e 50 -i /​input/​directory1 -c2 N1850_f19_tn11_exp2 -s2 21 -e2 50 -i2 /​input/​directory2 # model1-model2 diagnostics for CLM with user-specified history file directories
-diag_run -m clm -c N1850_f19_tn11_exp1 -s 21 -e 50 -t climo # model-obs climatology diagnostics (no time series) for CLM:+diag_run -m micom -c N1850_f19_tn11_exp1 -s 21 -e 50 -t climo # model-obs climatology diagnostics (no time series) for MICOM:
 diag_run -m cam -o /my/dir # install CAM diagnostics in /my/dir with minimal configuration. diag_run -m cam -o /my/dir # install CAM diagnostics in /my/dir with minimal configuration.
 +diag_run -m micom,​hamocc -c N1850OC_f19_tn11_exp1 -s 21 -e 50 # model-obs diagnostics for MICOM and HAMOCC.
 +diag_run -m clm -c N1850_f19_tn11_clmexp1 -s 71 -e 100 --no-atm # model-obs time-series diagnostics for an offline (uncoupled) CLM simulation.
 +diag_run -m hamocc -c N1850OC_f19_tn11_exp1 -s 31 -e 100 -t time_series # model-obs time-series diagnostics in HAMOCC between yrs 31 and 100.
 </​file>​ </​file>​
-Report any problems, comments or suggestions to Johan Liakka[[mailto:johan.liakka@nersc.no|johan.liakka@nersc.no]]+A comprehensive technical summary of diag_run (pdf){{ :noresm:​diag_run_documentation.pdf |}}
  
 +Report any problems, comments or suggestions to Yanchun He: [[mailto:​yanchun.he@nersc.no|yanchun.he@nersc.no]]
  
-=== Update to NCL version 6.4.0 === +=== Recent updates ​=== 
-The diagnostics packages have been configured ​and developed using NCL version 6.4.0 (the most recent ​version ​on NIRD as of 23.11.2017)The user is strongly recommended ​to use this version, as some bugs have been reported for older versionsTo update NCL, add the following two lines of code to your $HOME/.bashrc on NIRD+  * 08.08.19. Note: NS2345K is recovered now, please use the normal /​projects/​NS2345K/​noresm_diagnostics/​bin/​diag_run;​ put the webpage output to /​projects/​NS2345K/ ​and view the results at: http://​ns2345k.web.sigma2.no/​noresm_diagnostics/​ 
-<​file>​ +  * 08.08.19. Update to v6.0: add support to TS-derived mlts, global averages: tempga,​salnga,​sstga and sssga, updated volume transports, support extend Atlantic-Arctic overturning ​(mmflxd) for the NorESM2 ​version. ​And other minor changes 
-export NCARG_ROOT=/​opt/​ncl64 +  * 29.08.18. Update ​to v5.1: update ​NCO/NCL versions to support efficient process ​of compressed netcdf-4 files. 
-export PATH=/​opt/​ncl64/​bin/​:${PATH} +  * 29.06.18. Update ​to v5.0new fields to HAMOCC and MICOM diagnostics;​ minor fixes to other issues. 
-</file> +  * 20.04.18. Update to v4.3: added new fields to HAMOCC diagnostics. 
 +  * 19.04.18. Update to v4.2: included ability to do time-series diagnostics between two user-specified years. 
 +  * 18.04.18. Update to v4.1improved climatology and time-series calculations in CLM, and introduction of the --no-atm option to enable diagnostics for offline CLM simulations. 
 +  * 09.04.18. Update to v4.0: included the HAMOCC diagnostics package. 
 +  * 23.02.18. Update to v3.1: added monthly MLD, seasonal SST/SSS and annual meridional heat/​salinity fluxes to the MICOM diagnostics. 
 +  * 17.01.18. Update to v3.0: the first version of MICOM diagnostics has been included. 
 +  * 28.11.17. Update to v2.0: included a set of time series plots in CAM diagnostics,​ along with an html interface, which can be accessed from the index page (sets.htm). ​
  
 ===Using diag_run with cron=== ===Using diag_run with cron===
-If you want to use diag_run with crontab, you first need to load $HOME/​.bash_profile ​and export the variable RUN_BY_CRONe.g.:+If you want to use diag_run with crontab, you first need to load $HOME/​.bash_profile, ​i.e.:
 <​file>​ <​file>​
 #​--------------------------------------- #​---------------------------------------
 # Min Hour Day Month Weekday Command(s) # Min Hour Day Month Weekday Command(s)
 #​--------------------------------------- #​---------------------------------------
-50 09 23 11 * . $HOME/​.bash_profile; export RUN_BY_CRON="​TRUE"​; /​projects/​NS2345K/​noresm_diagnostics/​bin/​diag_run -m cam,cice -c N18_f19_tn11_080617 -s 21 -e 50 -o /scratch/johiak/​noresm_diagnostics2 -w \ +50 09 23 11 * . $HOME/​.bash_profile;​ /​projects/​NS2345K/​noresm_diagnostics/​bin/​diag_run -m cam,cice -c N18_f19_tn11_080617 -s 21 -e 50 -o /scratch/$USER/​noresm_diagnostics2 -w \ 
-/​projects/​NS2345K/​www/​test -t time_series >& /scratch/johiak/cron_out+/​projects/​NS2345K/​www/​test -t time_series >& /scratch/$USER/cron_out
 </​file>​ </​file>​
  
Line 85: Line 108:
 === NorESM diagnostics on GitHub === === NorESM diagnostics on GitHub ===
 The NorESM diagnostics packages and diag_run are included in the Git version control repository: The NorESM diagnostics packages and diag_run are included in the Git version control repository:
-[[https://​github.com/​johiak/​NoresmDiagnostics]] +[[https://​github.com/​NordicESMhub/​NoresmDiagnostics]]
 ====== Aerosol and Chemistry, Clouds and Forcing Diagnostics ====== ====== Aerosol and Chemistry, Clouds and Forcing Diagnostics ======
  
Line 260: Line 282:
 http://​www.jasmin.ac.uk/​services/​jasmin-analysis-platform/​ http://​www.jasmin.ac.uk/​services/​jasmin-analysis-platform/​
  
 +====== Post analysis and workup of CAM diagnostics output tables ======
 +
 +A tool for post analysis of (multiple) CAM diagnostics ASCII tables can be found in 
 +the following repository:
 +
 +GitHub https://​github.com/​jgliss/​noresm_diag_postproc
 +
 +To get started, please follow the instructions in repository README (displayed in repository). Currently, the main 
 +analysis tool is a jupyter IPython notebook called ​
 +
 +//​**analysis_tool.ipynb**//​ (https://​github.com/​jgliss/​noresm_diag_postproc/​blob/​master/​analysis_tool.ipynb)
 +
 +which includes more detailed instructions about setup and options. ​
 +
 +Use the notebook ​
 +
 +https://​github.com/​jgliss/​noresm_diag_postproc/​blob/​master/​download_tables.ipynb
 +
 +to download local copies of result tables using a list of URL's.
 +
 +
 +**Short summary:​** ​
 +The notebook reads multiple diagnostics files (runs) into one long table and creates heatmap plots
 +of //Bias, RMSE and RMSE relative error// for a subset of variables (rows -> y-axis of heatmap) ​
 +vs. the individual runs (columns -> xaxis). ​
 +
 +**NOTE:** In the current version, you need to download all tables that you are interested in as csv or 
 +ascii into one directory, that is specified in the header of the notebook.
 +
 +Variable groups can be defined in this config file:
 +
 +https://​github.com/​jgliss/​noresm_diag_postproc/​blob/​master/​config/​var_groups.ini
 +
 +**NOTE:** If you add groups to this file in your local copy of the repository, please consider sending
 +the updated to [[mailto:​jonasg@met.no|jonasg@met.no]] or to submit a pull request, so that the remote repository ​
 +remains up to date.
 +
 +**Troubleshooting**
  
 +If you run into problems, please raise an issue in the repository or contact [[mailto:​jonasg@met.no|jonasg@met.no]]
noresm/modeldiagnostics.txt · Last modified: 2019-08-09 07:46:23 by yanchun.he@nersc.no