Initial Commit

This package contains scripts intended to run OMB and IMB
benchmarks on certain collectives. It will use SGE to
gather relevant data and form a decision file.

The best and detailed output will be contained for each
collective separately.

Signed-off-by: William Zhang <wilzhang@amazon.com>

LICENSE

@@ -0,0 +1,50 @@
+Most files in this release are marked with the copyrights of the
+organizations who have edited them.  The copyrights below are in no
+particular order and generally reflect members of the Open MPI core
+team who have contributed code to this release.  The copyrights for
+code used under license from other parties are included in the
+corresponding files.
+
+Copyright (c) 2020      Amazon.com, Inc. or its affiliates.  All Rights
+                        reserved.
+
+$COPYRIGHT$
+
+Additional copyrights may follow
+
+$HEADER$
+
+Redistribution and use in source and binary forms, with or without
+modification, are permitted provided that the following conditions are
+met:
+
+- Redistributions of source code must retain the above copyright
+  notice, this list of conditions and the following disclaimer.
+
+- Redistributions in binary form must reproduce the above copyright
+  notice, this list of conditions and the following disclaimer listed
+  in this license in the documentation and/or other materials
+  provided with the distribution.
+
+- Neither the name of the copyright holders nor the names of its
+  contributors may be used to endorse or promote products derived from
+  this software without specific prior written permission.
+
+The copyright holders provide no reassurances that the source code
+provided does not infringe any patent, copyright, or any other
+intellectual property rights of third parties.  The copyright holders
+disclaim any liability to any recipient for claims brought against
+recipient by any third party for infringement of that parties
+intellectual property rights.
+
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

README.md

@@ -1 +1,92 @@
+Copyright (c) 2020      Amazon.com, Inc. or its affiliates.  All Rights
+                        reserved.
 
+$COPYRIGHT$
+
+Additional copyrights may follow
+
+$HEADER$
+
+===========================================================================
+
+Collectives Tuning
+
+===========================================================================
+
+Prerequisites:
+
+        Python2.7
+        SGE scheduler
+        OSU Micro Benchmarks
+        Intel Micro Benchmarks
+
+===========================================================================
+
+Installing OSU Micro Benchmarks:
+
+Run these commands to install osu micro benchmarks. Change $INSTALL_PATH
+to be your desired install path. Change $MPI_INSTALL_PATH to your Open MPI
+install path.
+
+```
+wget http://mvapich.cse.ohio-state.edu/download/mvapich/osu-micro-benchmarks-5.6.2.tar.gz
+tar -xvf osu-micro-benchmarks-5.6.2.tar.gz
+cd osu-micro-benchmarks-5.6.2
+./configure --prefix=$INSTALL_PATH CC=$MPI_INSTALL_PATH/bin/mpicc CXX=$MPI_INSTALL_PATH/bin/mpicxx
+make
+make install
+```
+
+===========================================================================
+
+Installing Intel Micro Benchmarks:
+
+Run these commands to install IMB-MPI1. Change $MPI_INSTALL_PATH to your
+Open MPI install path.
+
+```
+git clone https://github.com/intel/mpi-benchmarks.git
+cd mpi-benchmarks
+make IMB-MPI1 CC=$MPI_INSTALL_PATH/bin/mpicc CXX=$MPI_INSTALL_PATH/bin/mpicxx
+```
+
+===========================================================================
+
+This repository is intended to create scripts and analyze results of
+collectives to create a tuning decision file for Open MPI.
+
+Currently, the only binaries supported are for OSU Micro Benchmarks and
+Intel Micro Benchmarks for the following collectives:
+
+        allgather
+        allgatherv
+        allreduce
+        alltoall
+        alltoallv
+        barrier
+        bcast
+        gather
+        reduce
+        reduce_scatter_block
+        reduce_scatter
+        scatter
+
+Currently, you need to create a config file - see "./examples/config" in order
+to choose collectives, OMB collectives directory, IMB-MPI1 binary path,
+cluster sizes, number of ranks, number of nodes, number of ranks per node,
+and number of runs.
+
+If you need to adjust the number of algorithms or exclude certain
+algorithms, please adjust the file "./collective_jobs/<collective>.job"
+
+In order to run the scripts, please run inside this directory
+"./run_and_analyze.sh -c <your config file>"
+
+This script will run and analyze all collectives specified. The output
+will be saved under the ./output directory.
+
+A decision file will be written under ./output/decision.file
+
+Each collective will have a detailed output and a best output file under
+./output/<collective>/detail.out and ./output/<collective>/best.out
+respectively.

analyze.sh

@@ -0,0 +1,37 @@
+#!/bin/sh
+#
+#$ -j y
+#
+#$ -cwd
+#
+# Copyright (c) 2020      Amazon.com, Inc. or its affiliates.  All Rights
+#                         reserved.
+#
+# Additional copyrights may follow
+#
+
+config_file=""
+
+OPTIONS=$(getopt -o :c: --long config-file: -- "$@")
+if [[ $? -ne 0 ]]; then
+        usage
+        exit 1
+fi
+
+eval set -- "$OPTIONS"
+while true; do
+        case "$1" in
+                '-c'|'--config-file')
+                        case "$2" in
+				"") shift 2 ;;
+				*) config_file=$2; shift 2;;
+			esac;;
+                '--')
+                        shift
+                        break
+                ;;
+                *) echo "Internal error!"; usage; exit 1 ;;
+	esac
+done
+
+python coltune_analyze.py $config_file

collective_jobs/allgather.job

@@ -0,0 +1,3 @@
+number_of_algorithms : 6
+exclude_algorithms :
+two_proc_alg : 6

collective_jobs/allgatherv.job

@@ -0,0 +1,3 @@
+number_of_algorithms : 5
+exclude_algorithms :
+two_proc_alg : 5

collective_jobs/allreduce.job

@@ -0,0 +1,3 @@
+number_of_algorithms : 6
+exclude_algorithms :
+two_proc_alg :

collective_jobs/alltoall.job

@@ -0,0 +1,3 @@
+number_of_algorithms : 5
+exclude_algorithms :
+two_proc_alg : 5

collective_jobs/alltoallv.job

@@ -0,0 +1,3 @@
+number_of_algorithms : 2
+exclude_algorithms :
+two_proc_alg :

collective_jobs/barrier.job

@@ -0,0 +1,3 @@
+number_of_algorithms : 6
+exclude_algorithms :
+two_proc_alg : 5

collective_jobs/bcast.job

@@ -0,0 +1,3 @@
+number_of_algorithms : 9
+exclude_algorithms :
+two_proc_alg :

collective_jobs/gather.job

@@ -0,0 +1,3 @@
+number_of_algorithms : 3
+exclude_algorithms :
+two_proc_alg :

collective_jobs/reduce.job

@@ -0,0 +1,3 @@
+number_of_algorithms : 7
+exclude_algorithms :
+two_proc_alg :

collective_jobs/reduce_scatter.job

@@ -0,0 +1,3 @@
+number_of_algorithms : 4
+exclude_algorithms :
+two_proc_alg :

collective_jobs/reduce_scatter_block.job

@@ -0,0 +1,3 @@
+number_of_algorithms : 4
+exclude_algorithms :
+two_proc_alg :

collective_jobs/scatter.job

@@ -0,0 +1,3 @@
+number_of_algorithms : 2
+exclude_algorithms :
+two_proc_alg :