AMD Opteron 6100 Series platform cluster environment settings file.

This flags file provides additional information about how to run SPEC CPU2006 across multiple 6100 Series systems in a cluster environment using a perl script and numactl to distribute jobs across systems and cores.

System and Other Tuning Information

submit= submit-mc.pl $copies $SPECCOPYNUM %{cores_per_socket} %{useautopar} [0|1] "$command"

For rate jobs spanning multiple systems, the submit-mc.pl script will distribute the copies evenly across nodes and control placement of the copies to cores by using the numactl command.

The submit-mc.pl script takes the following as input:
- Number of copies being run in the test
- The copy number being submitted
- The number of cores per socket
- Whether or not autopar environment settings are needed
- Whether or not one copy per numa node is needed for the autopar environment (the number of numa nodes on a system can be seen by running numactl --hardware)
- The command to run

The script also needs to have which systems to run on defined in the systems variable.

***** START OF SCRIPT *****

#!/usr/bin/perl
#
# Submit script for clustered SPEC CPU2006 runs on
# Magny-Cours (Opteron 6100 series) platforms
#

use Cwd;

# Particular testbed used today:
my @systems = qw ( c6145-1 c6145-2 );

my $rundir        = getcwd;
my $copies        = shift @ARGV;
my $copynum       = shift @ARGV;
my $cores_per_socket = shift @ARGV;
# This variable will be 0 for int-rate and 1 for fp-rate submissions when --define autopar is set
my $autoparenv       = shift @ARGV;
# When running one copy per node
my $one_per_node  = shift @ARGV;

# Figure out how many copies to run per system, number the systems and
# what cpu number to use
my $copies_per_system = $copies / ($#systems+1);
my $systemnum = int($copynum / $copies_per_system);
my $cpunum = $copynum % $copies_per_system;

open DOBMK, "> dobmk" or die "Eh?";
print DOBMK "cd $rundir\n";

# This part is done only for fp-rate submissions when --define autopar is set
if (( $autoparenv ) && ( $one_per_node )) {
   my $whichnode = $copynum - ($systemnum * $copies_per_system);
# At most there could be 8 nodes in a system so distribute depending
# on how many cores per socket there are (which translates to cores to use per node)
   if ( $cores_per_socket eq 12 ) {
      if ( $whichnode == 0) { print DOBMK "numactl -m 0 --physcpubind=0,1,2,3,4,5 @ARGV \n"; }
      elsif ( $whichnode == 1 ) { print DOBMK "numactl -m 1 --physcpubind=6,7,8,9,10,11 @ARGV \n"; }
      elsif ( $whichnode == 2 ) { print DOBMK "numactl -m 2 --physcpubind=12,13,14,15,16,17 @ARGV \n"; }
      elsif ( $whichnode == 3 ) { print DOBMK "numactl -m 3 --physcpubind=18,19,20,21,22,23 @ARGV \n"; }
      elsif ( $whichnode == 4 ) { print DOBMK "numactl -m 4 --physcpubind=24,25,26,27,28,29 @ARGV \n"; }
      elsif ( $whichnode == 5 ) { print DOBMK "numactl -m 5 --physcpubind=30,31,32,33,34,35 @ARGV \n"; }
      elsif ( $whichnode == 6 ) { print DOBMK "numactl -m 6 --physcpubind=36,37,38,39,40,41 @ARGV \n"; }
      elsif ( $whichnode == 7 ) { print DOBMK "numactl -m 7 --physcpubind=42,43,44,45,46,47 @ARGV \n"; }
   } elsif ( $cores_per_socket eq 8 ) {
      if ( $whichnode == 0 ) { print DOBMK "numactl -m 0 --physcpubind=0,1,2,3 @ARGV \n"; }
      elsif ( $whichnode == 1 ) { print DOBMK "numactl -m 1 --physcpubind=4,5,6,7 @ARGV \n"; }
      elsif ( $whichnode == 2 ) { print DOBMK "numactl -m 2 --physcpubind=8,9,10,11 @ARGV \n"; }
      elsif ( $whichnode == 3 ) { print DOBMK "numactl -m 3 --physcpubind=12,13,14,15 @ARGV \n"; }
      elsif ( $whichnode == 4 ) { print DOBMK "numactl -m 4 --physcpubind=16,17,18,19 @ARGV \n"; }
      elsif ( $whichnode == 5 ) { print DOBMK "numactl -m 5 --physcpubind=20,21,22,23 @ARGV \n"; }
      elsif ( $whichnode == 6 ) { print DOBMK "numactl -m 6 --physcpubind=24,25,26,27 @ARGV \n"; }
      elsif ( $whichnode == 7 ) { print DOBMK "numactl -m 7 --physcpubind=28,29,30,31 @ARGV \n"; }
   }
}
else {
   print DOBMK "numactl -l --physcpubind=$cpunum @ARGV\n"; 
}
close DOBMK;
system 'chmod +x dobmk';
system  'ssh', "-n", $systems[$systemnum], "bash", "$rundir/dobmk";

***** END OF SCRIPT *****