MetaHipMer 2.0 for ISC21

Owned by Ophir Maor
Last updated: May 20, 2021
14 min read

 

References

 

MHM2 Lecture

Download the Slides here:

Recording:

 

Getting Started

 

Build and Install upcxx

 

  1. Download MHM2 and UPCXX from https://bitbucket.org/berkeleylab/mhm2/src/master/ and .

$ ls mhm2-2.0.0.tar.gz upcxx-2020.3.2.tar.gz

 

2. uncompress

$ tar xf mhm2-2.0.0.tar.gz $ tar xf upcxx-2020.3.2.tar.gz $ ls mhm2-2.0.0 mhm2-2.0.0.tar.gz upcxx-2020.3.2 upcxx-2020.3.2.tar.gz

 

3. Follow the guidelines here to build, make and install UPC++, load relevant compilers and environment variables, make sure the compilers you use are above the basic version required:

$ gcc -v Using built-in specs. COLLECT_GCC=gcc COLLECT_LTO_WRAPPER=/usr/libexec/gcc/x86_64-redhat-linux/8/lto-wrapper OFFLOAD_TARGET_NAMES=nvptx-none OFFLOAD_TARGET_DEFAULT=1 Target: x86_64-redhat-linux Configured with: ../configure --enable-bootstrap --enable-languages=c,c++,fortran,lto --prefix=/usr --mandir=/usr/share/man --infodir=/usr/share/info --with-bugurl=http://bugzilla.redhat.com/bugzilla --enable-shared --enable-threads=posix --enable-checking=release --enable-multilib --with-system-zlib --enable-__cxa_atexit --disable-libunwind-exceptions --enable-gnu-unique-object --enable-linker-build-id --with-gcc-major-version-only --with-linker-hash-style=gnu --enable-plugin --enable-initfini-array --with-isl --disable-libmpx --enable-offload-targets=nvptx-none --without-cuda-driver --enable-gnu-indirect-function --enable-cet --with-tune=generic --with-arch_32=x86-64 --build=x86_64-redhat-linux Thread model: posix gcc version 8.3.1 20191121 (Red Hat 8.3.1-5) (GCC) $ g++ -v Using built-in specs. COLLECT_GCC=g++ COLLECT_LTO_WRAPPER=/usr/libexec/gcc/x86_64-redhat-linux/8/lto-wrapper OFFLOAD_TARGET_NAMES=nvptx-none OFFLOAD_TARGET_DEFAULT=1 Target: x86_64-redhat-linux Configured with: ../configure --enable-bootstrap --enable-languages=c,c++,fortran,lto --prefix=/usr --mandir=/usr/share/man --infodir=/usr/share/info --with-bugurl=http://bugzilla.redhat.com/bugzilla --enable-shared --enable-threads=posix --enable-checking=release --enable-multilib --with-system-zlib --enable-__cxa_atexit --disable-libunwind-exceptions --enable-gnu-unique-object --enable-linker-build-id --with-gcc-major-version-only --with-linker-hash-style=gnu --enable-plugin --enable-initfini-array --with-isl --disable-libmpx --enable-offload-targets=nvptx-none --without-cuda-driver --enable-gnu-indirect-function --enable-cet --with-tune=generic --with-arch_32=x86-64 --build=x86_64-redhat-linux Thread model: posix gcc version 8.3.1 20191121 (Red Hat 8.3.1-5) (GCC)

 

4. cd and configure

 

5. Check the Makefile the was created in the build directory:

 

6. Make and check

 

7. Make install

 

8. Export upcxx bin to PATH

 

Build and Install mhm2

1. cd to mhm2 directory

 

2. Follow the guidelines here to build, make and install MHM2.

 

3. Export UPCXX network variables, not mandatory, as part of the build scripts supplied with the code, “Release” option should over that.

 

4. Continue with the build script supplied with mhm2 sources

 

Run mhm2

  1. In this example, we used slurm script to spawn the executable (Using this input file)

 

Output

Output Example for arctic_sample_0.fq on 4 skylake nodes:

 

We will look for the total time it took:

And the accuracy of the results.

Tuning Options

You are allowed to change any gasnet or upcxx configuration as long as the results come out correctly. We will supply a tool to check that.

 

Input File

The input file for the competition can be downloaded here.

ISC21 SCC Tasks

  1. Build and Install UPCXX

  2. Build and Install MHM2

  3. Run MHM2 for the given input (to be given to you at the competition day)

  4. Submit the output file and the results folder.

 

Note: You will need to supply results on both clusters, Niagara and Aspire-1 (4 CPU nodes only).

 

Note:

The output of MHM2 is probabilistic, so we don't expect identical results each time. However, the results should not differ dramatically from one run to the next. For validation, you can run a script ci_asm_qual_test.sh, found in the ci directory in the mhm2 repo. That script will download a small dataset, assemble it with MHM2, and compare the results with expected results. you can look at the contents of the script to get an idea of how the validation is done.

For this competition MHM2 should be running only on CPUs.