NEK5000

The version of Nek5000 that will be used for the Student Cluster Competition at ISC18 is 17.0, and it is available from [insert location].  You will also need the dataset tarball, available from [insert location].

Here is a step by step example of building Nek5000 for use on a cluster with x86_64 processors.  Replace all absolute paths with whatever is appropriate for your system.

1. Create a directory where you want to do your build, and change into it:

    cd /global/home/users/gerardo/benchmarks/isc18scc

2. Set your environment to use the appropriate compilers and MPI implementation (in this example, Intel compilers and HPC-X):

   module purge
   module load intel/2018.2.199 hpcx/2.1.0
   

3. Get the Nek5000 and cases tarballs from [], extract the contents of the Nek5000 tarball, go down into the Nek5000 directory, and extract the contents of the cases tarball:
   
   

   tar zxvf Nek5000-v17.0.tar.gz
   cd Nek5000
   tar zxvf ../cases.tgz

4. Nek5000 comes with a set of scripts that need to be made available by adding the appropriate path to the PATH environment variable.  SOURCE_ROOT may also need setting, because the default is $HOME/Nek5000.

   export PATH=/global/home/users/gerardo/benchmarks/isc18scc/Nek5000/bin:$PATH
   export SOURCE_ROOT=/global/home/users/gerardo/benchmarks/isc18scc/Nek5000

5. Use case1 to build and run a small test case:

   cd cases/case1
   makenek case1
   mpirun -np 40 ./nek5000 > case1.log.40

   You may choose to edit the SIZE file to match your target number of MPI ranks.  As distributed, it will run with at most 128 ranks.   This test case runs in 253s on 40 Intel SKL 6138 cores.

6. For the main competition test case, move into the case2 subdirectory and make the executable:

   cd ../case2
   ## SIZE comes configured for a minimum of 32 and a maximum of 2048 MPI ranks
   makenek case2

   The case2 SIZE file comes configured for a minimum of 32 and a maximum of 2048 MPI ranks.

7. If case2 does not include a SESSION.NAME file, just create it.  It should have to lines, containing the case name and the full path to the case2 directory, for instance:

   case2
   /global/home/users/gerardo/benchmarks/isc18scc/Nek5000/cases/case2/

8. Execute nek5000 with the appropriate mpirun command or a suitably built script for your batch scheduler. Case2 runs in 685s on 320 Intel SKL 6138 cores.