...

Building and Running example

Download Neko v0.68.10-rc1, https://github.com/ExtremeFLOW/neko/archive/refs/tags/v0.68.10-rc1.tar.gz.

To build Neko, you will need a Fortran compiler supporting the Fortran-08 standard, autotools, pkg-config, a working MPI installation supporting the Fortran 2008 bindings (mpi_f08), BLAS/LAPACK and JSON-Fortran. Detailed installation instructions can be found in the Neko manual.

Sample build script on PSC:

...

Note: Intel MPI is recommended, OpenMPI may not be stable, but you are welcome to try.

 Sample run script: (Update)

#!/bin/bash
#SBATCH -p RM
#SBATCH --nodes=4
#SBATCH --ntasks-per-node=128
#SBATCH -J neko
#SBATCH --time=4:00:00
#SBATCH --exclusive
module purge
source /jet/packages/oneapi/v2023.2.0/compiler/2023.2.1/env/vars.sh
source /jet/packages/oneapi/v2023.2.0/mkl/2023.2.0/env/vars.sh

HCA=mlx5_0:1

source /jet/packages/oneapi/v2023.2.0//mpi/2021.10.0/env/vars.sh
USE_UCX=1
MPIFLAGS=""
if [ $USE_UCX -ne 0 ]; then
        MPIFLAGS+="-genv USE_UCX=$USE_UCX "
        MPIFLAGS+="-genv UCX_NET_DEVICES ${HCA} "
        MPIFLAGS+="-genv FI_PROVIDER=mlx "
        export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:<path to ucx>/lib
else
        MPIFLAGS+="-genv FI_PROVIDER=^mlx "
fi

cd tgv
<path to neko>/bin/makeneko ${TEST}.f90
mpirun -np <# procs> $MPIFLAGS ./neko tgv_Re1600.case

 Sample output: (Update)

    _  __  ____  __ __  ____
   / |/ / / __/ / //_/ / __ \
  /    / / _/  / ,<   / /_/ /
 /_/|_/ /___/ /_/|_|  \____/

 (version: 0.6.1)
 (build: 2023-09-14 on x86_64-pc-linux-gnu using intel)


 -------Job Information--------
 Start time: 18:44 / 2023-09-18
 Running on: 128 MPI ranks
 CPU type  : Intel(R) Xeon(R) Platinum 8280 CPU @ 2.70GHz
 Bcknd type: CPU
 Real type : double precision

 -------------Case-------------
 Reading case file tgv.case
...

    --------Postprocessing--------

       --------Writer output---------
       File name: field.fld
       Output number:     2
       Writing at time:   2.000500 Output time (s):  2.501281
 Normal end.

...

To compile Neko with GPU support, please follow either the generic CUDA instructions at https://github.com/ExtremeFLOW/neko/discussions/540 instructions in the manual (Compiling Neko for NVIDIA GPUs and Compiling Neko for AMD GPUs).

Note: A Neko installation can only support one backend. Thus, to run both CPU and GPU experiments, two different builds and installations are necessary.

Tasks & Submissions (Update)

1. Run Neko with the given input

   View file
   name tgv.zip

   (neko tgv_Re1600.case) on 4 CPU nodes and submit the results to the team’s folder (standard output, field0.f00000, field0.f00001 and field0.nek5000)
   Note: the small input is for you to play around.

2. Run Neko with the given input on 4 GPUs (V100) and submit the results to the team’s folder (see instructions for GPU support above, and the additional information provided with the input file)

3. Run IPM profile or any other MPI profile on 4 nodes, and find the 3 most used MPI calls, show your work in the team interview.

4. Create a visualization short video with Paraview and show it on your team interview

   1. Change nsamples to e.g. 100 in tgv_Re1600.case

   2. Download all field0.* files to a machine with Paraview (e.g. your laptop)

   3. Open field0.nek5000 in Paraview, don’t forget to select all “Point Arrays” before pressing “Apply”

   4. Try various filters and representations, for example, iso-surfaces, glyphs or volume rendering

      1. The flow field will show more interesting features if the simulation is allowed to run longer (controlled by T_end in the case file).

   5. To save the final animation, follow the instructions at https://docs.paraview.org/en/latest/UsersGuide/savingResults.html#saving-animation

...