Overview

ASCOT5 is a simulation tool used in fusion energy science and engineering. It solves test particle trajectories in magnetized plasmas using the Monte Carlo method, providing statistical analysis on the overall behavior of a particle population.

The code is frequently applied to study fast particles: fusion-born alpha particles (He²⁺) inherently present in any fusion device, energetic ions produced by external heating schemes such as neutral beam injection and ion cyclotron resonance heating, and runaway electrons often generated during plasma disruptions. A key feature of ASCOT5 is its full 3D geometry capability, which enables the study of how fast particles are affected by magnetic field perturbations, and assessment of whether fast particle losses pose a risk to plasma-facing components within the reactor chamber.

The code is extensively parallelized with hybrid MPI+OpenMP scheme with GPU support, and it is capable of large simulations with ten million markers or more.

GitHub: https://github.com/ascot4fusion/ascot5

Documentation: https://ascot4fusion.github.io/ascot5/

Presentation:

Build on CPU (Iris cluster)

Install the requirements:
- Any compiler (GNU, NVHPC, INTEL)
- HDF5
- MPI
- Python3.10+ for postprocessing
Download the source:
git clone https://github.com/ascot4fusion/ascot5.git
Sample build script on iris cluster. The executables will be located at build/:
module load intel/2023.2 module load compiler mkl MPI=hpcx-2.25.1 module load $(echo $MPI | sed 's/x-/x\//') export OMPI_CC=icc module load hdf5/1.14.6-$MPI module load python make clean make ascot5_main -j MPI=1 OMP=1 NSIMD=8 CC=mpicc make libascot -j MPI=1 OMP=1 NSIMD=8 CC=mpicc

MPI=1 → enables MPI support

OMP=1 → enable OpenMP
NSIMD=8 → SIMD vector length used for CPU vectorization.
CC=mpicc → MPI C compiler wrapper

Build with NVHPC for NVIDIA GPU

Allocate a Grace node like below before you build Ascot5 because the login node has a AMD CPU.

salloc -N 1 -p instant --account=<...> -t 1:00:00 --mem=16G --constraint=armgpu

Sample build script on Romeo cluster using nvhpc.

module use /gpfs/projet/r250119/tools/nvhpc/modulefiles
module load nvhpc-hpcx-cuda12/26.1
module load hdf5-1.14.6

make ascot5_main -j MPI=1 ACC=1 GPU=1 CC=mpicc
make libascot -j MPI=1 ACC=1 GPU=1 CC=mpicc

ACC=1 → enable OpenACC
GPU=1 → enable computation on GPU

The test case

The test case is based on the upcoming SPARC fusion experiment, currently under construction by Commonwealth Fusion Systems in Devens, Massachusetts. SPARC is expected to begin operations in 2027, with the primary objective of demonstrating net energy production by achieving Q > 1. The design point of SPARC, as published in Creely et al. J. Plasma Phys. (2020), is intended to produce up to 140 MW of fusion power.

This test case represents a modeled reference scenario similar to that design point, with an assumed 140 MW of fusion power. The fusion energy is carried by alpha particles (He⁴ ions) and energetic neutrons: alpha particles carry approximately 28 MW, while the remaining energy is carried by neutrons. The neutrons, being uncharged, promptly escape to the vessel wall, where—in a fusion power plant—they would both breed new fuel and provide heat for electricity generation. The alpha particles, however, remain confined in the plasma, where they transfer their energy through collisions, thereby sustaining the plasma temperature and reducing the need for external heating.

The role of ASCOT5 in this test case is to model the slowing-down process of these alpha particles as they deposit their energy into the surrounding plasma. Because alphas are sensitive to magnetic field perturbations, they can be lost from confinement, potentially creating localized heat loads on the vessel wall—conditions that must be avoided. ASCOT5 traces alpha particle trajectories using detailed magnetic field and plasma profiles as input, and computes wall losses. The magnetic field model includes realistic perturbations arising from the finite number of toroidal field coils, while the plasma profiles—comprising deuterium, tritium, some impurity species, and electrons—are obtained from transport simulations, with the electron density of approx. 3 x 10^20 m^-3 and temperature of approx. 2 x 10^8 K. The wall is represented by a simplified geometric model of the actual structure.

To ensure it finishes in reasonable time, the simulation does not cover the full alpha population over the entire slowing-down time (~0.5 s). Instead, it focuses on alphas born near the plasma edge, which are most prone to prompt losses, and the simulation time is limited to cover only the prompt losses. The results indicate moderate peak wall loads of approximately 0.5 MW/m², well within the tolerances of the wall material.

Running and accessing results

The simulation is executed as follows:

./ascot5_main --in=<path/to/input.h5>

From stdout, you can confirm that the code is running in MPI parallel by finding the lines

Initialized MPI, rank X, size Y.

Likewise for OpenMP (CPU) you should see

Simulation begins; 24 threads.

If the simulation completed successfully, the end of log will look like this (note: the results shown here are for the 1M marker case):

Summary of results:
   648600 markers had end condition Sim time limit
   351400 markers had end condition Wall collision

          No markers were aborted.

Done.
Simulation complete

The same inputs can be used for both CPU and GPU simulations.

The post-processing is done using Python 3.10+:

import matplotlib.pyplot as plt
from a5py import Ascot
from matplotlib.collections import LineCollection
a5 = Ascot("<path/to/input.h5>")

# Plot side-view of the tokamak showing how markers were lost along the wall
fig, ax = plt.subplots()
wall_segments = a5.data.wall.wall_2D_3905771333.getwallcontour()
ax.add_collection(LineCollection(wall_segments, colors="k"))
a5.data.active.plotstate_scatter("end r", "end z", endcond="WALL", axes=ax)
ax.autoscale()
ax.set_aspect("equal")
plt.show()

Positions of the lost markers (blue) on the reactor wall (black).

If pyvista is installed, you can also plot 3D view of the wall loads:

a5.data.active.plotwall_3dstill(
    data="eload",
    log=False,
    clim=[1e0,1e5],
    cfoc=[ 2.0,  0.0, -0.5 ],
    cpos=[ 2.3, -0.3, -0.4]
    )

Power loads from prompt alpha particle losses on the reactor wall showing no concentrated hot spots.

The figures of merit (wetted area and peak power load) can be obtained with:

a5.data.active.getwall_figuresofmerit()
>>(unyt_quantity(41.52707139, 'm**2'), unyt_quantity(460678.55567314, 'W/m**2'))

Since there is no randomness in this test case (collisions are disabled and marker input is fixed), the results should always be the same.

Tasks and Submissions

Inputs are available under /gpfs/projet/r250119/inputs/ISC26/ascot5 on Romeo and /global/exafs/groups/rdmaworkshop/inputs/ISC26/ascot5 on Iris.

Run the ASCOT5 application with the given input (1M) on 1, 2, 4 Romeo GPU nodes (each node has 4 GraceHopper CPU-GPU). Analyze the scalability of the results. Submit your best performance, build script and run script. Do not submit multiple results nor binary files.
Run the ASCOT5 application with the given input (1M) on 1, 2, 4 nodes on Iris cluster Analyze the scalability of the results. submit the results to the teams folder. Play with number of MPI ranks and OpenMP for best results. Submit your best performance, build script and run script. Do not submit multiple results nor binary files.
Compare 1M input to 5M input on single GPU node (4 GPUs), show your results in the teams presentation.
Perform MPI Profile (any profiler) to the application on Romeo and Iris clusters on 4 nodes, show the results in the teams presentation, what are the 3 most used MPI calls.
Visualize the results using any tool, create a short video or several photos.