...
To build Neko, you will need a Fortran compiler supporting the Fortran-08 standard, autotools, pkg-config, a working MPI installation supporting the Fortran 2008 bindings (mpi_f08), BLAS/LAPACK and JSON-Fortran. Detailed installation instructions can be found in the Neko manual.
Sample build script on PSC:
| Code Block |
|---|
#!/bin/bash source /jet/packages/oneapi/v2023.2.0/compiler/2023.2.1/env/vars.sh source /jet/packages/oneapi/v2023.2.0//mpi/2021.10.0/env/vars.sh export MPIFC=mpiifortmpif90 export CC=mpiiccmpicc export FC=$MPIFC export PKG_CONFIG_PATH=/path/to/jsonfortran/lib/pkgconfig:${PKG_CONFIG_PATH} ./regen.sh ./configure --prefix=<path> make make install |
...
Sample run scriptoutput: (Update)
| Code Block |
|---|
#!/bin/bash
#SBATCH -p RM
#SBATCH --nodes=4
#SBATCH --ntasks-per-node=128
#SBATCH -J neko
#SBATCH --time=4:00:00
#SBATCH --exclusive
module purge
source /jet/packages/oneapi/v2023.2.0/compiler/2023.2.1/env/vars.sh
source /jet/packages/oneapi/v2023.2.0/mkl/2023.2.0/env/vars.sh
HCA=mlx5_0:1
source /jet/packages/oneapi/v2023.2.0//mpi/2021.10.0/env/vars.sh
USE_UCX=1
MPIFLAGS=""
if [ $USE_UCX -ne 0 ]; then
MPIFLAGS+="-genv USE_UCX=$USE_UCX "
MPIFLAGS+="-genv UCX_NET_DEVICES ${HCA} "
MPIFLAGS+="-genv FI_PROVIDER=mlx "
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:<path to ucx>/lib
else
MPIFLAGS+="-genv FI_PROVIDER=^mlx "
fi
cd tgv
<path to neko>/bin/makeneko ${TEST}.f90
mpirun -np <# procs> $MPIFLAGS ./neko tgv_Re1600.case |
Sample output: (Update)
| Code Block |
|---|
_ __ ____ __ __ ____ / |/ / / __/ / //_/ / __ \ / / / _/ / ,< / /_/ / /_/|_/ /___/ /_/|_| \____/ (version: 0.6.1) (build: 2023-09-14 on x86_64-pc-linux-gnu using intel) -------Job Information-------- Start time: 18:44 / 2023-09-18 Running on: 128 MPI ranks CPU type : Intel(R) Xeon(R) Platinum 8280 CPU @ 2.70GHz Bcknd type: CPU Real type : double precision _ __ ____ __ __ ____ / |/ / / __/ / //_/ / __ \ / / / _/ / ,< / /_/ / /_/|_/ /___/ /_/|_| \____/ (version: 0.8.0-rc1) (build: 2024-04-10 on x86_64-pc-linux-gnu using cray) -------Job Information-------- Start time: 10:41 / 2024-04-10 Running on: 256 MPI ranks CPU type : AMD EPYC 7742 64-Core Processor Bcknd type: CPU Real type : double precision -------------Case------------- Reading case file tgv_Re1600.case -------------Mesh------------- Reading a binary Neko file 32768.nmsh ... -----Material properties------ Read non-dimensional values: ... -----Starting simulation------ ... -----Starting simulation------ T : [ 0.0000000E+00, 0.2000000E+01) dt : 0.5000000E-03 ... ---------------------------------------------------------------- t = 0.0000000E+00 [ 0.00% ] ---------------------------------------------------------------- Time-step: 1 CFL: 0.3970812E-01 dt: 0.5000000E-03 ... ---------------------------------------------------------------- t = 0.2000000E+01 [ 100.00% ] ----------------------------------Case--------------- Reading case file tgv.case ... --------Postprocessing------- Time-step: 4001 CFL: 0.4271451E-01 dt: 0.5000000E-03 ... --------Writer output--------- File name: field.fld Output number: 210 Writing at time: 2.000500 Output time (s): 2.501281 Normal end. |
...
Note: A Neko installation can only support one backend. Thus, to run both CPU and GPU experiments, two different builds and installations are necessary.
Tasks & Submissions
These tasks aim to solve the given flow case as fast as possible (time to solution); this is reported as the Elapsed time in the log files and includes everything from solving the equations to performing I/O.
Run Neko with the given input
(View file name tgv.zip neko tgv_Re1600.case) on 4 CPU nodes and submit the results to the team’s folder (standard output,field0.f00000,field0.f00001andfield0.nek5000)
Note: the small input is for you to play around.Run Neko with the given input on 4 a couple of GPUs (the memory requirements of the case is roughly 2 A100 or 4 V100) and submit the results to the team’s folder (see instructions for GPU support above, and the additional information provided with the input file)
...