Getting started with Neko for ISC24 SCC (In-Person)

Owned by Ophir Maor
Last updated: May 14, 2024 by Niclas Jansson (Unlicensed)
4 min read

Overview

Neko is a portable framework for high-order spectral element flow simulations. Written in modern Fortran, Neko adopts an object-oriented approach, allowing multi-tier abstractions of the solver stack and facilitating various hardware backends ranging from general-purpose processors, CUDA and HIP enabled accelerators to SX-Aurora vector processors. Neko has its roots in the spectral element code Nek5000 from UChicago/ANL, from where many of the namings, code structure and numerical methods are adopted.

For more information, please visit https://neko.cfd.

Neko presentation to the teams:

 

Presentation file:

Building and Running example

Download Neko v0.8.0-rc1, https://github.com/ExtremeFLOW/neko/archive/refs/tags/v0.8.0-rc1.tar.gz.

To build Neko, you will need a Fortran compiler supporting the Fortran-08 standard, autotools, pkg-config, a working MPI installation supporting the Fortran 2008 bindings (mpi_f08), BLAS/LAPACK and JSON-Fortran. Detailed installation instructions can be found in the Neko manual.

Note: For the in-person event, a Fortran compiler supporting the entire Fortran 2008 standard is required; compilers with partial support are not supported. It is not allowed to modify src/common/signal.f90, with the exception of provided patches located in patches/.

Sample build script:

Build json-fortran:

git clone --depth=1 https://github.com/jacobwilliams/json-fortran.git cd json-fortran && mkdir b && cd b cmake -DCMAKE_INSTALL_PREFIX=/path/to/installation -DUSE_GNU_INSTALL_CONVENTION=ON .. make install

Build Neko:

#!/bin/bash # Load Intel Compilers and MPI libraries. export MPIFC=mpiifort export CC=mpiicc export FC=$MPIFC export PKG_CONFIG_PATH=/path/to/jsonfortran/lib64/pkgconfig:${PKG_CONFIG_PATH} ./regen.sh ./configure --prefix=<path> make make install

Sample run script:

# Untar tgv.zip cd tgv # Load Intel Compilers and MPI libraries. export MPIFC=mpiifort export FC=$MPIFC export LD_LIBRARY_PATH=<path>/json-fortran/lib64:$LD_LIBRARY_PATH <path>/neko/bin/makeneko ${TEST}.f90 mpirun -np <NPROC> -genv USE_UCX=1 -genv UCX_NET_DEVICES mlx5_0:1 -genv FI_PROVIDER=mlx ./neko tgv_Re1600.case

Sample output:

 

GPU support:

To compile Neko with GPU support, please follow either the instructions in the manual (Compiling Neko for NVIDIA GPUs and Compiling Neko for AMD GPUs).

Note: A Neko installation can only support one backend. Thus, to run both CPU and GPU experiments, two different builds and installations are necessary.

Task & Submission

The aim of this task is to compute as many time steps as possible for the given flow case within a time limit of maximum 15 minutes.

  1. Run Neko with the given input (neko tgv_Re1600.case) on either CPUs or GPUs.

  2. Submit your best result (standard output). Do not submit binary output files nor multiple results.

 

Note: You are allowed to experiment with different linear solvers in Neko (see the manual) to achieve the fastest runtime. However, all of them are not guaranteed to work for the given case or support all hardware backends in Neko. Furthermore, it is allowed to experiment with different configuration options when building Neko, e.g., enabling device-aware MPI. However, it is not allowed to use single precision, changing the time-step size or any other parameters related to the case.

Â