...
Compiling MILC (CPU only)
| Code Block |
|---|
#git clone fall3d-8.1.2.tar.gz will be provided during competition. module load intel/2021.4 module load compiler/2021.4.0 module load mkl/2021.4.0 # Build netcdf and set include and lib paths ./configure -prefix=<install path> --enable-parallel make clean make -j 32 make install--branch develop https://github.com/milc-qcd/milc_qcd.git cd milc_qcd/ks_imp_rhmc cp ../Makefile . # Edit compile_su3_rhmd_hisq_quda.sh # Remove QUDA, CUDA, GPU flags and set compilers, arch etc. # module load intel/2022.3.1 compiler mkl module load hpcx/2.14.0 export OMPI_MPICC=icx ./compile_su3_rhmd_hisq_quda.sh |
Compiling MILC (GPU support)
Running FALL3D
A sample example is available under Example directory.
| Code Block |
|---|
cd Example
NPROC=160
mpirun -np $NPROC <MPI Flags> Fall3d.r8.x Example.inp 8 5 4 -nens 1
# Make sure the process grid matches with NPROC, i.e. 8 * 5 * 4 = 160 |
Tasks and Submissions
Input is related Mount St. Helens eruption, https://www.usgs.gov/media/videos/mount-st-helens-eruption-may-18-1980.
| Code Block |
|---|
# Copy input from USB to a folder.
NPROC=160
mpirun -np $NPROC <MPI Flags> Fall3d.r8.x fall3d helens.inp 8 5 4
# Make sure the process grid matches with NPROC, i.e. 8 * 5 * 4 = 160 |
Try different processor grids for better performance.
For debugging or quick testing, you can decrease the duration of simulation by changing the below parameter in helens.inp but make sure to use the original value for the submission.
| Code Block |
|---|
RUN_END_(HOURS_AFTER_00) = 55 |
...
Check the below link.
https://github.com/lattice/quda/wiki/MILC-with-QUDA
Running MILC
Input file and instructions are available at https://github.com/lattice/quda/wiki/Running-the-NERSC-MILC-Benchmarks
We will be using 24x24x24x60.chklat for the competition.
| Code Block |
|---|
wget https://portal.nersc.gov/project/m888/apex/MILC_160413.tgz
tar xvzf MILC_160413.tgz
cd MILC-apex/benchmarks
wget https://portal.nersc.gov/project/m888/apex/MILC_lattices/24x24x24x60.chklat
cp medium/run_medium.sh run-milc.sh
# Change problem size in run-milc.sh to 24 24 24 60 and adjust other parameters.
# Execute run-milc.sh |
Submissions
Submit your build & run script and the output file.