...

Note: The page may be changed until the competition stats, maybe sure to follow up until the opening ceremony.

Neko presentation to the teams:

Presentation file:

Building and Running example

Download Neko v0.6.1, https://github.com/ExtremeFLOW/neko/archive/refs/tags/v0.6.1.tar.gz.

Sample build script on PSC:

tar xfp v0.6.1.tar.gz
cd neko-0.6.1/
# Load intel compilers and mpi#!/bin/bash
source /jet/packages/oneapi/v2023.2.0/compiler/2023.2.1/env/vars.sh
source /jet/packages/oneapi/v2023.2.0//mpi/2021.10.0/env/vars.sh
export MPIFC=mpiifort
export CC=mpiicc
export FC=$MPIFC

./regen.sh
./configure CC=mpiicc FC=mpiifort --prefix=<path>
make
make install

...

 Sample run script:

# Load intel compilers and mpi
mpirun -np 128 -genv I_MPI_PIN on neko tgv!/bin/bash
#SBATCH -p RM
#SBATCH --nodes=4
#SBATCH --ntasks-per-node=128
#SBATCH -J neko
#SBATCH --time=4:00:00
#SBATCH --exclusive
module purge
source /jet/packages/oneapi/v2023.2.0/compiler/2023.2.1/env/vars.sh
source /jet/packages/oneapi/v2023.2.0/mkl/2023.2.0/env/vars.sh

HCA=mlx5_0:1

source /jet/packages/oneapi/v2023.2.0//mpi/2021.10.0/env/vars.sh
USE_UCX=1
MPIFLAGS=""
if [ $USE_UCX -ne 0 ]; then
        MPIFLAGS+="-genv USE_UCX=$USE_UCX "
        MPIFLAGS+="-genv UCX_NET_DEVICES ${HCA} "
        MPIFLAGS+="-genv FI_PROVIDER=mlx "
        export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:<path to ucx>/lib
else
        MPIFLAGS+="-genv FI_PROVIDER=^mlx "
fi

cd tgv
<path to neko>/bin/makeneko ${TEST}.f90
mpirun -np <# procs> $MPIFLAGS ./neko tgv_Re1600.case

 Sample output:

    _  __  ____  __ __  ____
   / |/ / / __/ / //_/ / __ \
  /    / / _/  / ,<   / /_/ /
 /_/|_/ /___/ /_/|_|  \____/

 (version: 0.6.1)
 (build: 2023-09-14 on x86_64-pc-linux-gnu using intel)


 -------Job Information--------
 Start time: 18:44 / 2023-09-18
 Running on: 128 MPI ranks
 CPU type  : Intel(R) Xeon(R) Platinum 8280 CPU @ 2.70GHz
 Bcknd type: CPU
 Real type : double precision

 -------------Case-------------
 Reading case file tgv.case
...

    --------Postprocessing--------

       --------Writer output---------
       File name: field.fld
       Output number:     2
       Writing at time:   2.000500 Output time (s):  2.501281
 Normal end.

...

Note: A Neko installation can only support one backend. Thus, to run both CPU and GPU experiments, two different builds and installations are necessary.Input file: View filenametgv_Re1600.tar.gz

Tasks & Submissions

1. Run Neko with the given input

   View file
   name tgv.zip

   (neko tgv_Re1600.case) on 4 CPU nodes and submit the results to the team’s folder (standard output, field0.f00000, field0.f00001 and field0.nek5000)
   Note: the small input is for you to play around.

2. Run Neko with the given input on 4 GPUs (V100) and submit the results to the team’s folder (see instructions for GPU support above, and the additional information provided with the input file)

3. Run IPM profile or any other MPI profile on 4 nodes, and find the 3 most used MPI calls, show your work in the team interview.

4. Create a visualization short video with Paraview and show it on your team interview

   1. Change nsamples to e.g. 100 in tgv_Re1600.case

   2. Download all field0.* files to a machine with Paraview (e.g. your laptop)

   3. Open field0.nek5000 in Paraview, don’t forget to select all “Point Arrays” before pressing “Apply”

   4. Try various filters and representations, for example, iso-surfaces, glyphs or volume rendering

      1. The flow field will show more interesting features if the simulation is allowed to run longer (controlled by T_end in the case file).

   5. To save the final animation, follow the instructions at https://docs.paraview.org/en/latest/UsersGuide/savingResults.html#saving-animation

...