Getting Started with FluTAS for ISC23 SCC

Overview

FluTAS (Fluid Transport Accelerated Solver) is an open-source code targeting multiphase fluid dynamics simulations. The key feature of FluTAS is the ability to efficiently run both on many-CPUs and many-GPUs in a single and unified framework.
This framework is also designed to be modular so that it can be extended in a sustainable manner.

Downloading and compiling FluTAS

• Download the application:

git clone https://github.com/Multiphysics-Flow-Solvers/FluTAS.git

• Builld FFTW library

• Choose a built script from src/targets and build the application.

cd FluTAS
BASE=$PWD
# Edit src/targets/target.generic-intel with your fftw path
module load intel/2022.1.2
module load compiler/2022.1.0
module load mkl/2022.0.2
module load hpcx/2.13.0

export CXX=mpicxx
export FC=mpif90
export FFTW_DIR=$BASE/fftw-3.3.10

make clean APP=single_phase
make ARCH=generic-intel APP=single_phase DO_DBG=0 -j 16 

Running Example

# Copy one of examples
mkdir run
cp -r examples/two_phase_ht/dhc run
cd run/dhc
# Edit the following line in dns.in to change the process grid. Below will work with 128 cores (2x64 cores)
2 64                                 ! dims(1:2)

EXE=<path>/flutas.two_phase_ht
/usr/bin/time mpirun -np 128 $MPIFLAGS $EXE"

Task and submission

1. Use the input under <TBD>.

2. Profile the input
   Use any of the remote clusters to run an MPI profile (such as IPM profile or any other profiler) over 4 nodes, full PPN to profile the given input.

   • Submit the profile as PDF to the team's folder.

   • Add to your presentation the 3 main MPI calls that are being used plus the MPI time being used.

3. Run the FluTAS on both PSC bridges-2 and FAU Fritz CPU clusters.

4. Submission and Presentation:
   - Submit all the build scripts, run scripts and stdout/logs.
   - Do not submit the data output.
   - Prepare slides for the team’s interview based on your work for this application