...
| Code Block |
|---|
$ export WRFIO_NCD_LARGE_FILE_SUPPORT=1
$ export NETCDF=$NETCDF_DIR
$ export HDF5=$HDF5_DIR |
4. Configure
| Code Block |
|---|
./configure |
...
After the configure, you may edit the configure.wrf file to adjust as needed.
For exampleAMD nodes:
Remove -xHost - will not work on AMD
change to -xCORE-AVX2 -to -march=core-avx2
For Skylake Nodes
-xCORE-AVX2 may need to change to AVx512 -xCORE-AVX512 for Intel Skylake hosts
Other
Edit mpicc, remove the “-f90=$(SFC)” option from mpif90 and the “-cc=$(SCC)” option from mpicc for HPC-X (Open MPI) as it doesn’t need those options.
5. Compile
| Code Block |
|---|
$ ./compile wrf |
...
-j8 wrf |
Note: if you compiled before, you need to clean the old compilation by running, and renaming again the configure.wrf.
| Code Block |
|---|
$ ./clean -a
...
$ mv configure.wrf.backup configure.wrf |
6. Check the run directory
For input, make sure you have
...
| Code Block |
|---|
$ tail -f rsl.out.0000 WRF V3.9.1.1 MODEL ************************************* Parent domain ids,ide,jds,jde 1 1501 1 1201 ims,ime,jms,jme -4 195 -4 82 ips,ipe,jps,jpe 1 188 1 75 ************************************* DYNAMICS OPTION: Eulerian Mass Coordinate alloc_space_field: domain 1 , 299838928 bytes allocated RESTART run: opening wrfrst_d01_2005-06-04_06_00_00 for reading Timing for processing restart file for domain 1: 54.97731 elapsed seconds Max map factor in domain 1 = 1.02. Scale the dt in the model accordingly. INITIALIZE THREE Noah LSM RELATED TABLES LANDUSE TYPE = USGS FOUND 27 CATEGORIES INPUT SOIL TEXTURE CLASSIFICATION = STAS SOIL TEXTURE CLASSIFICATION = STAS FOUND 19 CATEGORIES d01 2005-06-04_06:00:00 WRF restart, LBC starts at 2005-06-04_00:00:00 and restart starts at 2005-06-04_06:00:00 LBC for restart: Starting valid date = 2005-06-04_00:00:00, Ending valid date = 2005-06-04_03:00:00 LBC for restart: Restart time = 2005-06-04_06:00:00 LBC for restart: Looking for a bounding time LBC for restart: Starting valid date = 2005-06-04_03:00:00, Ending valid date = 2005-06-04_06:00:00 LBC for restart: Starting valid date = 2005-06-04_06:00:00, Ending valid date = 2005-06-04_09:00:00 LBC for restart: Found the correct bounding LBC time periods for restart time = 2005-06-04_06:00:00 Timing for processing lateral boundary for domain 1: 0.29368 elapsed seconds Tile Strategy is not specified. Assuming 1D-Y WRF TILE 1 IS 1 IE 188 JS 1 JE 75 WRF NUMBER OF TILES = 1 Timing for main: time 2005-06-04_06:00:15 on domain 1: 3.07004 elapsed seconds Timing for main: time 2005-06-04_06:00:30 on domain 1: 1.74579 elapsed seconds Timing for main: time 2005-06-04_06:00:45 on domain 1: 1.72297 elapsed seconds Timing for main: time 2005-06-04_06:01:00 on domain 1: 1.73288 elapsed seconds Timing for main: time 2005-06-04_06:01:15 on domain 1: 1.73057 elapsed seconds Timing... for main: time... d01 2005-06-04_06:0110:30 on domain 100 wrf: SUCCESS COMPLETE 1.72477 elapsed seconds Timing for main: time 2005-06-04_06:01:45 on domain 1: 1.73379 elapsed seconds Timing for main: time 2005-06-04_06:02:00 on domain 1: 1.73048 elapsed seconds Timing for main: time 2005-06-04_06:02:15 on domain 1: 1.73523 elapsed seconds Timing for main: time 2005-06-04_06:02:30 on domain 1: 1.72220 elapsed seconds Timing for main: time 2005-06-04_06:02:45 on domain 1: 1.72852 elapsed seconds Timing for main: time 2005-06-04_06:03:00 on domain 1: 1.73229 elapsed seconds WRF taskid: 0 hostname: thor001.hpcadvisorycouncil.com |
8. For actual forecast output and video creating, you need to enable history output in the namelist.input file.
Performance Measurements
As WRF is heavy in IO (reading the optionally writing the files), there are two ways to measure performance here.
1. Use Linux time command to measure the total time of the mpirun, that includes IO time and compute time. For example:
| Code Block |
|---|
$ time -p mpirun -np 512 wrf.exe
...
real 72.00
user 2178.06
sys 30.18 |
Check the real total time of the application.
2. In case you are not interested on the IO time, but only on the compute time, you can sum all the rows in the rsl.out.0000 files with the time measurements. For example:
| Code Block |
|---|
Timing for main: time 2005-06-04_06: |
...
00: |
...
30 on domain 1: 1. |
...
74579 elapsed seconds |
...
A run command could be:
| Code Block |
|---|
$ cat rsl.out.0000 | grep "Timing for main |
...
" | awk '{ SUM += $9} END { print "Total Time is: " SUM }'
...
Total Time is: 16.9808 |