...
References
...
| View file | ||||
|---|---|---|---|---|
|
Getting the ICON Framework Code
...
| View file | ||||
|---|---|---|---|---|
|
Prerequisites
HDF5: https://www.hdfgroup.org/downloads/hdf5/source-code
...
Optional: https://developer.nvidia.com/networking/hpc-x or other MPI (HPC-X will be provided in a USB drive with input files).
Check this link to learn how to install/use HPC-X, Profiling using IPM and HPC-X
...
The executable file of ICON will be created to /path/to/icon-scc/build/bin. OpenMPI/ HPC-X was more stable than Intel MPIis available from Nvidia, https://developer.nvidia.com/networking/hpc-x .
Testing Icon
Interactive Mode Test Example
...
| Code Block |
|---|
Timer report
------------------------------- ------- ------------ ------------ ------------ ------------- ------------- -------------
name # calls t_min t_avg t_max total min (s) total max (s) total avg (s)
------------------------------- ------- ------------ ------------ ------------ ------------- ------------- -------------
total 1 03m29s 03m29s 03m29s 209.619 209.619 209.619
L solve_ab 240 0.15084s 0.26483s 27.0229s 63.559 63.559 63.559
L ab_expl 240 0.09686s 0.09740s 0.10715s 23.375 23.375 23.375
L extra1 240 0.03408s 0.03435s 0.03754s 8.244 8.244 8.244
L extra2 240 0.02285s 0.02301s 0.02559s 5.523 5.523 5.523
L extra3 240 0.00792s 0.00799s 0.00887s 1.917 1.917 1.917
L extra4 240 0.03187s 0.03204s 0.03514s 7.690 7.690 7.690
L ab_rhs4sfc 240 0.01947s 0.01957s 0.02058s 4.697 4.697 4.697
L triv-t init 1 0.00013s 0.00013s 0.00013s 0.000 0.000 0.000
... |
...
Batch Mode Test Example
For running an experiment in batch mode you first need to create the appropriate run script. You can do this by executing the script make_runscripts in ICON’s root directory. The experiment you can prepare and run is named test_ocean_omip_10days and the template with the experiment specific data already exists.
...
Running ICON for the competition
Input will be handed out using a USB drive.Download the input data from : TBD
The input data contain the 3 files: restart.tar.gz, hd.tar.gz and grids.tar.gz
Unpack them into the directory, /path/to/icon-scc/build/data. 2.
Configuration of the experiment.
Go to the directory where the configuration files of the experiment are located:
| Code Block |
|---|
$ cd /path/to/icon-scc/build/run/standard_experiments In file DEFAULT.config on line 33 set MODEL_ROOT to the directory, where you have icon-scc installed with plenty of space for IP. Set EXP_SUBDIR = build/$EXPERIMENTS_SUBDIR/$EXP_ID on line 46. In file scc.config on line 6 you have to set your ACCOUNT. The file scc.config also contains the tunables described in the next paragraph. |
...
Go to the main directory of ICON and run the experiment script:
| Code Block |
|---|
$ cd /path/to/icon-scc $ pip install --user six jinja2 $ ./RUN_ISC_SCC_EXPERIMENT If you encounter the below error, add "export mpi_total_procs=1" to the script as a workaround and rerun it. Oops: missing option "mpi_total_procs" in interpolation while reading key 'use_mpi_startrun' $ $ ./RUN_ISC_SCC_EXPERIMENT |
...
| View file | ||
|---|---|---|
|
| View file | ||
|---|---|---|
|
Tasks and Submissions
|
| View file | ||||
|---|---|---|---|---|
|
Tasks and Submissions
Input files
on Bridges-2 cluster, copy the input files from the following location:
/ocean/projects/cis210088p/davidcho/Icon-input
to your home directory.
| Code Block |
|---|
[maor@bridges2-login011 ~]$ ll /ocean/projects/cis210088p/davidcho/Icon-input
total 8458004
-rw-r--r-- 1 davidcho cis210088p 8356853152 Mar 1 13:06 grids.tar.gz
-rw-r--r-- 1 davidcho cis210088p 451058 Mar 1 13:06 hd.tar.gz
-rw-r--r-- 1 davidcho cis210088p 303674079 Mar 1 13:06 restart.tar.gz
[maor@bridges2-login011 ~]$ cp /ocean/projects/cis210088p/davidcho/Icon-input . |
On Niagara cluster, copy the input files from the following location
/scratch/i/iscscc-scinet/davidcho01/Icon-input
to your home directory.
GOAL : is to speedup the execution time of this experiment setup by porting and tuning ICON to the newer architectures of Niagara and Bridges-2 cluster. Run ICON with the coupled atmosphere ocean experiment as described in section “Running ICON for the competition” and submit the results, assuming 4 node cluster. The simulated time is 1 model year and the reference Wallclock time is around 30 minutes with 4 nodes30 minutes. Your main task is to beat the reference time, i.e. make ICON run 1 simulated year in less than 30 minutes.
Porting and tuning for specific processor architecture . For single node/processor performance optimization the parameter nproma can be used to tune the model towards better vectorization (AVX2/AVX512).
Load Balancing: Upon successful completion you will see in the log file 2 “Timer reports”, one for the atmosphere and one for the ocean. By inspecting these reports identify the most CPU time consuming parts. The timer coupling_1stget gives an indication of the extent of load imbalance between atmosphere and ocean. Can you think of a better load balancing schemean indication of the extent of load imbalance between atmosphere and ocean. Can you think of a better load balancing scheme?
Run IPM profile for the application and submit the results as PDF. Based on this profile make an analysis of the MPI-Communication pattern. What are the main bottlenecks ?
Your optimizations need to yield acceptable results. Check the correctness of your results by using the python script scc_plots.ipynb, as described above in the section “Postprocessing”.
Files to be returned are all log files the results in the subdirectory outdata of your experiment and your scc_plots.ipynb. The profiles you have produced and your analysis in a PDF document.
In case the team have a twitter account publish the figure (or video) with the hashtags: #ISC22, #ISC22_SCC (mark/tag the figure or video with your team name/university).