Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 15 Next »

Overview

Neko is a portable framework for high-order spectral element flow simulations. Written in modern Fortran, Neko adopts an object-oriented approach, allowing multi-tier abstractions of the solver stack and facilitating various hardware backends ranging from general-purpose processors, CUDA and HIP enabled accelerators to SX-Aurora vector processors. Neko has its roots in the spectral element code Nek5000 from UChicago/ANL, from where many of the namings, code structure and numerical methods are adopted.

For more information, please visit https://neko.cfd.

Note: The page may be changed until the competition stats, maybe sure to follow up until the opening ceremony.

Building and Running example

Download Neko v0.6.1, https://github.com/ExtremeFLOW/neko/archive/refs/tags/v0.6.1.tar.gz.

Sample build script on PSC:

#!/bin/bash
source /jet/packages/oneapi/v2023.2.0/compiler/2023.2.1/env/vars.sh
source /jet/packages/oneapi/v2023.2.0//mpi/2021.10.0/env/vars.sh
export MPIFC=mpiifort
export CC=mpiicc
export FC=$MPIFC

./regen.sh
./configure --prefix=<path>
make
make install

Note: Intel MPI is recommended, OpenMPI may not be stable, but you are welcome to try.

 Sample run script:

#!/bin/bash
#SBATCH -p RM
#SBATCH --nodes=4
#SBATCH --ntasks-per-node=128
#SBATCH -J neko
#SBATCH --time=4:00:00
#SBATCH --exclusive
module purge
source /jet/packages/oneapi/v2023.2.0/compiler/2023.2.1/env/vars.sh
source /jet/packages/oneapi/v2023.2.0/mkl/2023.2.0/env/vars.sh

HCA=mlx5_0:1

source /jet/packages/oneapi/v2023.2.0//mpi/2021.10.0/env/vars.sh
USE_UCX=1
MPIFLAGS=""
if [ $USE_UCX -ne 0 ]; then
        MPIFLAGS+="-genv USE_UCX=$USE_UCX "
        MPIFLAGS+="-genv UCX_NET_DEVICES ${HCA} "
        MPIFLAGS+="-genv FI_PROVIDER=mlx "
        export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:<path to ucx>/lib
else
        MPIFLAGS+="-genv FI_PROVIDER=^mlx "
fi

cd tgv
<path to neko>/bin/makeneko ${TEST}.f90
mpirun -np <# procs> $MPIFLAGS ./neko tgv_Re1600.case

 Sample output:

    _  __  ____  __ __  ____
   / |/ / / __/ / //_/ / __ \
  /    / / _/  / ,<   / /_/ /
 /_/|_/ /___/ /_/|_|  \____/

 (version: 0.6.1)
 (build: 2023-09-14 on x86_64-pc-linux-gnu using intel)


 -------Job Information--------
 Start time: 18:44 / 2023-09-18
 Running on: 128 MPI ranks
 CPU type  : Intel(R) Xeon(R) Platinum 8280 CPU @ 2.70GHz
 Bcknd type: CPU
 Real type : double precision

 -------------Case-------------
 Reading case file tgv.case
...

    --------Postprocessing--------

       --------Writer output---------
       File name: field.fld
       Output number:     2
       Writing at time:   2.000500 Output time (s):  2.501281
 Normal end.

GPU support:

To compile Neko with GPU support, please follow the generic CUDA instructions at https://github.com/ExtremeFLOW/neko/discussions/540

Note: A Neko installation can only support one backend. Thus, to run both CPU and GPU experiments, two different builds and installations are necessary.

Tasks & Submissions

  1. Run Neko with the given input on 4 CPU nodes and submit the results to the team’s folder (standard output, field0.f00000, field0.f00001 and field0.nek5000)

  2. Run Neko with the given input on 4 GPUs (V100) and submit the results to the team’s folder (see instructions for GPU support above, and the additional information provided with the input file)

  3. Run IPM profile or any other MPI profile on 4 nodes, and find the 3 most used MPI calls, show your work in the team interview.

  4. Create a visualization short video with Paraview and show it on your team interview

    1. Change nsamples to e.g. 100 in tgv_Re1600.case

    2. Download all field0.* files to a machine with Paraview (e.g. your laptop)

    3. Open field0.nek5000 in Paraview, don’t forget to select all “Point Arrays” before pressing “Apply”

    4. Try various filters and representations, for example, iso-surfaces, glyphs or volume rendering

      1. The flow field will show more interesting features if the simulation is allowed to run longer (controlled by T_end in the case file).

    5. To save the final animation, follow the instructions at https://docs.paraview.org/en/latest/UsersGuide/savingResults.html#saving-animation

  • No labels