Getting Started with FluTAS for ISC23 SCC

Owned by Ophir Maor
Last updated: Mar 07, 2023
4 min read

 

 

Overview

FluTAS (Fluid Transport Accelerated Solver) is an open-source code targeting multiphase fluid dynamics simulations. The key feature of FluTAS is the ability to efficiently run both on many-CPUs and many-GPUs in a single and unified framework.
This framework is also designed to be modular so that it can be extended in a sustainable manner.

 

Presentation

Here are the introduction presentation:

 

Slides:

Downloading and compiling FluTAS

  • Download the application:

git clone https://github.com/Multiphysics-Flow-Solvers/FluTAS.git

Sample build script for Fritz:

cd FluTAS/src # Edit FC name and FFTW path in targets/target.generic-intel. # MPI=intelmpi-2021.7.0 MPI=openmpi-4.1.2-intel2021.4.0 if [[ "$MPI" =~ ^intel ]]; then module load intel/2022.1.0 module load fftw/3.3.10-impi export I_MPI_F90=ifort elif [[ "$MPI" =~ ^openmpi ]]; then module load fftw/3.3.10-ompi export OMPI_MPIF90=ifort fi module load $(echo $MPI | sed -e "s/\-/\//") make ARCH=generic-intel APP=two_phase_ht DO_DBG=0 DO_POSTPROC=0

Sample build script for PSC:

Download HPC-X from ISC23 SCC Getting Started with Bridges-2 Cluster.

cd FluTAS/src # Edit FC name and FFTW path in targets/target.generic-intel. source /jet/packages/intel/oneapi/compiler/2022.1.0/env/vars.sh source /jet/packages/intel/oneapi/mkl/2022.1.0/env/vars.sh MPI=hpcx MPI=intelmpi if [[ "$MPI" =~ ^hpcx ]]; then module use $HOME/hpcx-2.13.1/modulefiles module load hpcx export OMPI_MPICC=icc export OMPI_MPICXX=icpc export OMPI_MPIFC=ifort export OMPI_MPIF90=ifort export FC=mpif90 else source /jet/packages/intel/oneapi/mpi/2021.6.0/env/vars.sh export FC=mpiifort fi export FFTW_HOME=<path>/fftw-3.3.10-$MPI make ARCH=generic-intel APP=two_phase_ht DO_DBG=0 DO_POSTPROC=0

Running Example

Before you start, make sure to change the process grid in dns.in:

The example below will work with 256cores (16x16):

 

Example:

 

Task and submission

Use this input, two_layer_rb.

  1. Profile the input
    Use any of the remote clusters to run an MPI profile (such as IPM profile or any other profiler) over 4 nodes, full PPN to profile the given input.

    1. Submit the profile as PDF to the team's folder.

    2. Add to your presentation the 3 main MPI calls that are being used plus the MPI time being used.

  2. Run the FluTAS on both PSC bridges-2 and FAU Fritz CPU clusters for four node runs.

    • Submit the results to the team's folder (4 node run only, 1 result per cluster).

    • experiment with 1,2,4 node runs. Add to your presentation a scalability graph based on your results and any conclusions you came up with. No need to submit here the results, just show your work on the ppt for the team interview.

  3. Submission and Presentation:
    - Submit all the build scripts, run scripts and stdout/logs.
    - Do not submit the output data or the source code.
    - Prepare slides for the team’s interview based on your work for this application

 

 

Â