This post will walk you through the procedure to build and test ChaNGa over Charm++ using UCX, or Open MPI/Intel MPI machine layers over InfiniBand Interconnect and Intel architecture (Skylake).
...
| Code Block |
|---|
$ ./build ChaNGa ucx-linux-x86_64 ompipmix icc ifort --enable-error-checking --with-production -j16 ... ChaNGa built successfully. Next, try out a sample program like ucx-linux-x86_64-ifort-ompipmix-icc/tests/charm++/simplearrayhello |
Note: In case you are using OpenUCX, you need to include the CPATH and the LIBRARY_PATH parameters in your environment.
| Code Block |
|---|
$ CPATH=/where/ucx/was/installed/include LIBRARY_PATH=/where/ucx/was/installed/lib ./build ChaNGa ucx-linux-x86_64 ompipmix icc ifort --enable-error-checking --with-production -j16 |
and before running, make sure to pre-load the library path as well.
| Code Block |
|---|
$ LD_LIBRARY_PATH=/where/ucx/was/installed/lib:$LD_LIBRARY_PATH ./charmrun ... |
MPI Machine Layer (OpenMPI/HPC-X )
...
In this example, we will use small dwarf star dwf1 (dwf1.2048.param and dwf1.2048.00384) is a 5 million particle zoom-in simulation. It is cosmological, but the particle sampling focuses on a single halo of roughly 1e11 solar masses. This is a dark matter only version of the DWF1 model that demonstrates how disk galaxies can form in a cosmological context.
To test ChaNGa, download dwf files into ChaNGa directory (where the executable is).
| Code Block |
|---|
$ ls dwf1.2048.* dwf1.2048.00384 dwf1.2048.param |
...