Get Started with HPC-X
- Ophir Maor
HPC-X is a comprehensive software package that includes MPI and SHMEM communications libraries. HPC-X also includes various acceleration packages to improve both the performance and scalability of applications running on top of these libraries, including UCX (Unified Communication X), which accelerate the underlying point to point. It also includes HCOLL which accelerates the underlying collective operations used by the MPI/PGAS languages. In addition, HPC-X contains SHARP library. For more information about HPC-X see here.
HPC-X is installed by default as a part of Mellanox OFED, or can be installed by itself.
HPC-X installation as a part of Mellanox OFED
1. Download and Mellanox OFED on your server(s), untar and install.
./mlnxofedinstall [options]See here for example.
2. Export libraries for OpenMPI and OSU benchmarks (if needed).
For example:
$ export PATH=/usr/mpi/gcc/openmpi-4.0.2rc3/bin:/usr/mpi/gcc/openmpi-4.0.2rc3/tests/osu-micro-benchmarks-5.3.2:$PATH
$ export LD_LIBRARY_PATH=/usr/mpi/gcc/openmpi-4.0.2rc3/lib:$LD_LIBRARY_PATH
3. Check that you can use mpirun and osu:
$ which mpirun
/usr/mpi/gcc/openmpi-4.0.2rc3/bin/mpirun
$ which osu_latency
/usr/mpi/gcc/openmpi-4.0.2rc3/tests/osu-micro-benchmarks-5.3.2/osu_latency
4. Check HPC-X versions per package
4. Allocate nodes and test
HPC-X installation as a separate package
Download and Mellanox OFED on your server(s), untar and install.
See here for example.
2. Download and untar HPC-X package , for example, HPC-X 2.7 for RHEL 7.6
3. Install environment modules
4. exit and ssh again to the session.
5. Use hpcx module
6. Run basic OSU test
For more info refer to HPC-X documentation here.