Building the SPEC CPU2017 Toolset

$Id: tools-build.html 6049 2018-08-03 14:09:17Z JohnHenning $ Latest: www.spec.org/cpu2017/Docs/

Contents

1. Introduction

a. What are the tools?

b. Pre-compiled binaries

c. When to build the tools yourself

d. Turn on session recorder

2. How to build the tools

a. Notes specific to Unix systems

b. Notes specific to Windows systems

3. How to verify that your build succeeded

4. Describing your newly-built tools

a. Toolset presentation

1. Excluding otherwise valid candidates

2. Affecting presentation order

5. Packagetools

a. Example: use packagetools

b. Example: test the newly packaged tools

6. Troubleshooting: What to do if something goes wrong

a. Try it by hand

b. Known problems

c. Executing just part of buildtools

7.Support

1. Introduction

SPEC supplies various tools that are used to ensure consistent operation of benchmarks across a variety of platforms. In order to generate a valid result which can be used in public and/or submitted to SPEC, you must use the SPEC supplied tools.

The tools are supplied with your copy of SPEC CPU2017, in both source form and binary form. Usually, the binaries will be all that you need (and you won't need this document).

1.a. What are the tools?

The SPEC CPU2017 tools include (but are not limited to):

rawformat Report writer for all SPEC CPU2017 resuls
runcpu The primary user interface for running the benchmarks.
specdiff Examines benchmark outputs for acceptable answers.
sysinfo Inventories various aspects of a System Under Test
specinvoke Runs and times commands, after runcpu sets them up.
(Some of the above are Perl scripts, which are always used in source form; some need to be compiled.)

Redistributable tools. SPEC gratefully acknowledges the many free and open source software contributions that are used in CPU2017. The table below provides a few examples; for the complete list, please see SPEC CPU2017 Licenses. Most are built as spectoolname to avoid possible conflicts with versions that may may already be on your system. SPEC requires that published results use the versions of the tools that it supplies, to improve benchmark result comparability.

specmake GNU make Builds binary versions of benchmark source files.
specxz XZ Utils Compression
specsha512sum From GNU coreutils Print or verify checksums.
spectar GNU tar Save or restore from file archives
specperl Perl The Perl 5 language interpreter
And many more.

1.b. Pre-compiled binaries

You will find pre-compiled binaries for the tools on your distribuion media, in the directories:

tools/bin/<archname> (Unix)
tools\bin\<archname> (Microsoft Windows)

After installation, the above directory will be found underneath $SPEC (Unix) or %SPEC% (Windows).

The appropriate set of precompiled binaries is automatically installed when you run install.sh (Unix) or install.bat (Windows).

1.c. When to build the tools yourself

Sometimes it may be necessary to rebuild the tools, for example if a change in an operating system renders a precompiled binary inoperable, or if you are the first person to add support for a new architecture.

It is generally not possible to do cross-compilation of Perl, as Perl's build process involves running binaries generated during the build. Therefore, getting a toolset onto an embedded platform without a native compiler is most likely impossible.

1.d. Turn on session recorder

Please turn on your operating system's session recorder (e.g. in Unix, typically the script(1) command) prior to using buildtools. If you do not have a session recorder, make your terminal window 999999 lines tall and be prepared to cut-and-paste the full output. There are two reasons to record your session:

  1. In case something goes wrong, and you want to ask SPEC for technical support.
  2. If case everything goes right, and you want to use your results in public.

NOTICE: If you want to generate results that can be used in public, you must ask SPEC to review your tool build prior to publishing (whether at SPEC or elsewhere.) You will need to provide the log and the output from runcpu -V and runcpu --test on a system OTHER than the build system, where you have installed your new tools build.

2. How to build the tools

If the directory $SPEC/tools/src (Unix) or %SPEC%\tools\src (Windows) does not yet exist, look in the directory install_archives on your SPEC CPU2017 distribution media, for a file with a name similar to tools-src.tar. (The name might vary slightly, depending on your version of CPU2017.) Expand it into a writable directory local to the build system. (Building tools on networked filesystems might work, but it may make some of GNU tar's tests fail, so it's safer to build on a local filesystem.)

In the example below, the user notes that the tools source directory is not present, locates the tar file on the distribution media, unpacks it, and then lists the contents of the newly-unpacked tools/src. (Note: the set of directories, and the versions of the tools, may vary depending on your version of SPEC CPU2017.)

$ cd /Users/cloyce/spec/cpu2017/v1.0.0
$ ls $SPEC/tools
bin
$ 
$ ls /Volumes/CPU2017v1.0.0/install_archives/*tar
/Volumes/CPU2017v1.0.0/install_archives/tools-src.tar
$ tar xf /Volumes/CPU2017v1.0.0/install_archives/tools-src.tar 
$ ls tools/src
Algorithm-Diff-1.1903    MailTools-2.14               XML-SAX-0.99           makedepf90-2.8.8
Exporter-Tiny-0.042      PDF-API2-2.025               XML-SAX-Base-1.08      ntzsh
File-NFSLock-1.27        SVG-2.64                     XML-SAX-ExpatXS-1.33   perl-5.24.0
Font-AFM-1.20            String-ShellQuote-1.04       XSLoader-0.22          redistributable
Font-TTF-1.05            Test-Deep-0.119              buildtools             rxp-1.5.0
IO-String-1.08           Text-CSV_XS-1.20             buildtools.bat         specinvoke
IO-stringy-2.111         TimeDate-2.30                buildtools.log         specsum
List-MoreUtils-0.416     URI-1.69                     expat-2.1.0            tar-1.28
MIME-tools-5.507         XML-NamespaceSupport-1.11    make-4.2.1             xz-5.2.2
$ 

The scripts $SPEC/tools/src/buildtools (Unix) or %SPEC%\tools\src\buildtools.bat (Windows) will build the tools. You may need to invoke the buildtools script with appropriate environment variables set first - see the sections immediately following that provide notes about specific platforms.

You will need a C99 compiler. Most of the tools are C++ or C89, but XZ is C99.

If everything goes right, you won't have to do very much at all, other than watch build commands fly by.

2.a. Notes specific to Unix systems

When building the tools under Unix, the following warnings appear to be harmless as of July 2011:

Check out those who have gone before. If the system for which you're attempting to build tools is similar to one for which a tools build exists, you may gain insight by looking at the top of the build logs in tools/src/buildtools.log. Even if your system is very unusual and very different than previous systems, it is probably still worth your while to look over the previous logs. You don't have to read all 20,000 lines; just check out the environment variables and path settings and so forth from the top of previous builds.

For convenience, and to give you ideas as you may be thinking about some new system, here are the variables set by the builds from SPEC CPU2017 v1.0.0 (lines have been wrapped for human readability).

aix-ppc-0.102.0.buildlog.txt

 buildtools controls:
  SKIPPERLOPTIMIZE=1

 configure flags:
  PERLFLAGS=-Dcc=xlc_r -Duse64bitall -Dusethreads -Uuseshrplib -Uoptimize= -Aoptimize=-O2 
            -Aoptimize=-qarch=pwr4 -Aoptimize=-qtune=balanced:balanced -Accflags=-U__MATH__
  XZFLAGS=--enable-static --disable-shared
  ac_cv_func_realpath=no
  ac_cv_header_stdbool_h=yes

 build options:
  CC=xlc
  CFLAGS=-O3 -q64 -qsuppress=1500-036 -qlanglvl=extc99 -qarch=pwr4 -qtune=balanced:balanced
  CXX=xlC
  LDFLAGS=-b64

hpux11iv3-ipf-0.101.0.buildlog.txt

 configure flags:
  PERLFLAGS=-Doptimize=+O1 -Dlibpth=/usr/lib/hpux32 -Dldflags=-lm -Dcppstdin=cc 
            -Acppstdin=-E -Acppstdin=-Aa -Acppstdin=-D__STDC_EXT__ -Acppstdin=-AC89 
            -Aoptimize=-Ae -Aoptimize=+O2 -Aoptimize=+z -Aoptimize=-AC99

 build options:
  CFLAGS=-Ae +O2 +z -D_FILE_OFFSET_BITS=64 -AC99
  CXX=CC
  XZCFLAGS=-AC99

linux-aarch64-0.904.0.buildlog.txt

 configure flags:
  PERLFLAGS=-Uplibpth= -Aplibpth=/lib/aarch64-linux-gnu -Aplibpth=/lib 
            -Aplibpth=/usr/lib/aarch64-linux-gnu -Aplibpth=/usr/lib  -Uoptimize= -Aoptimize=-O3
  XZFLAGS=--disable-assembler

 build options:
  CC=gcc
  CFLAGS= -O3 
  CXX=g++
  LDFLAGS= 

linux-armv7l-0.102.0.buildlog.txt

 configure flags:
  PERLFLAGS=-Uplibpth= -Aplibpth=/lib/arm-linux-gnueabihf -Aplibpth=/lib 
            -Aplibpth=/usr/lib/arm-linux-gnueabihf -Aplibpth=/usr/lib  -Uoptimize= -Aoptimize=-O3
  XZFLAGS=--disable-assembler

 build options:
  CC=gcc
  CFLAGS= -O3 
  CXX=g++
  LDFLAGS= 

 make options:
  MAKEFLAGS=-j4

linux-ppc-0.102.0.buildlog.txt

 configure flags:
  PERLFLAGS=-Uplibpth= -Aplibpth=/lib/powerpc-linux-gnu -Aplibpth=/lib -Aplibpth=/usr/lib/powerpc-linux-gnu 
            -Aplibpth=/usr/lib  -Uoptimize= -Aoptimize=-m32 -Aoptimize=-O3 -Aoptimize=-m32
  XZFLAGS=--disable-assembler

 build options:
  CC=gcc
  CFLAGS=-m32 -O3 -m32 -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64
  CXX=g++
  LDFLAGS=-m32 -m32

linux-ppc64-0.903.0.buildlog.txt

 configure flags:
  PERLFLAGS=-Uplibpth= -Aplibpth=/usr/lib64 -Aplibpth=/usr/lib  -Uoptimize= -Aoptimize=-m64 
            -Aoptimize=-O3 -Aoptimize=-m64
  XZFLAGS=--disable-assembler

 build options:
  CC=gcc
  CFLAGS=-m64 -O3 -m64
  CXX=g++
  LDFLAGS=-m64 -m64

linux-ppc64le-0.903.0.buildlog.txt

 configure flags:
  PERLFLAGS=-Uplibpth= -Aplibpth=/usr/lib64 -Aplibpth=/usr/lib  -Uoptimize= -Aoptimize=-m64 
            -Aoptimize=-O3 -Aoptimize=-m64
  XZFLAGS=--disable-assembler

 build options:
  CC=gcc
  CFLAGS=-m64 -O3 -m64
  CXX=g++
  LDFLAGS=-m64 -m64

linux-sparc-0.102.0.buildlog.txt

 configure flags:
  PERLFLAGS=-Uplibpth= -Aplibpth=/lib64 -Aplibpth=/usr/lib64 -Aplibpth=/lib -Aplibpth=/usr/lib  
            -Uoptimize= -Aoptimize=-m64 -Aoptimize=-O3 -Aoptimize=-m64
  XZFLAGS=--disable-assembler

 build options:
  CC=gcc
  CFLAGS=-m64 -O3 -m64
  CXX=g++
  LDFLAGS=-m64 -m64

linux-x86_64-0.102.0.buildlog.txt

 configure flags:
  PERLFLAGS=-Uplibpth= -Aplibpth=/lib64 -Aplibpth=/usr/lib64 -Aplibpth=/lib 
            -Aplibpth=/usr/lib  -Uoptimize= -Aoptimize=-m64 -Aoptimize=-O3 -Aoptimize=-m64
  XZFLAGS=--disable-assembler

 build options:
  CC=gcc
  CFLAGS=-m64 -O3 -m64
  CXX=g++
  LDFLAGS=-m64 -m64

macosx-x86-0.102.0.buildlog.txt

 configure flags:
  PERLFLAGS=-Doptimize=-O3 -Aoptimize=-g -Aoptimize=-m64
  XZFLAGS=--disable-assembler

 build options:
  CC=cc
  CFLAGS=-m64
  CPPFLAGS=-no-cpp-precomp -D_DARWIN_C_SOURCE
  CXX=CC
  LDFLAGS=-m64
  SUMCFLAGS=-O3

solaris-sparc-0.904.0.buildlog.txt

 configure flags:
  PERLFLAGS=-Doptimize=-xO4 -Aoptimize=-xchip=generic -Aoptimize=-xtarget=generic 
            -Aoptimize=-g -Dcc=cc -Dlibswanted=socket -Alibswanted=nsl -Alibswanted=dl 
            -Alibswanted=m -Aoptimize=-m64 -Aoptimize=-xpagesize=4M
  TARFLAGS=-with-rmt=/usr/sbin/rmt
  XZFLAGS=--disable-assembler

 build options:
  CC=cc
  CFLAGS=-m64 -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -xpagesize=4M
  CXX=CC
  LDFLAGS=-m64
  SPECINVOKECFLAGS=-xc99
  SUMCFLAGS=-xO4 -xchip=generic -xtarget=generic
  XZCFLAGS=-fast -xchip=generic -xtarget=generic

solaris-x86-0.102.0.buildlog.txt


 configure flags:
  PERLFLAGS=-Doptimize=-xO4 -Aoptimize=-xchip=generic -Aoptimize=-xtarget=generic 
            -Aoptimize=-g -Dcc=cc -Dlibswanted=socket -Alibswanted=nsl -Alibswanted=dl 
            -Alibswanted=m -Aoptimize=-m32 -Aoptimize=-xpagesize=2M
  TARFLAGS=-with-rmt=/usr/sbin/rmt
  XZFLAGS=--disable-assembler

 build options:
  CC=cc
  CFLAGS=-m32 -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -xpagesize=2M
  CXX=CC
  LDFLAGS=-m32
  SPECINVOKECFLAGS=-xc99
  SUMCFLAGS=-xO4 -xchip=generic -xtarget=generic
  XZCFLAGS=-fast -xchip=generic -xtarget=generic

tru64-alpha-0.903.0.buildlog.txt

 configure flags:
  XZFLAGS=ac_cv_prog_cc_c99=

 build options:
  SPECINVOKECFLAGS=-DHAVE_STDBOOL_H -I.
  SUMCFLAGS=-O4 -DPRIuMAX=\"llu\" -DPRIx64=\"llx\" -I/tmp/tmpinclude
  XZCFLAGS=-pthread -DPRIx64=\"llx\" -DPRIx32=\"x\"

2.b. Notes specific to Windows systems

Because XZ is written in C99, no version of Visual Studio (as of July 2011) can be used to build the tools. Instead, the Windows version of buildtools is designed to use the MinGW GCC compiler. It has been tested successfully with both 4.3 and 4.5 versions of GCC; the latest version available should be used. The tools are not yet ported to be built with a 64-bit compiler; use the 32-bit tools instead.

The builds use makefiles to be processed with either mingw-make.exe (which is GNU make) or dmake.exe (which is built in the process of building tools). The full MSYS environment is not used, so all of the makefiles are pre-generated; the GNU autoconf system is not used. Compiler flags are defined in those pre-generated makefiles.
Building tools under Cygwin is not supported and probably won't work.

If it is necessary to unzip and untar the sources of the tools in Windows, open a Command-Prompt window, cd to %SPEC% (i.e. the top directory of SPEC). Then execute:

  cd tools
  bin\windows-i386\specxz.exe -d install_archives/cpu2017.tar.xz
  ooo\windows-i386\spectar.exe -xvf install_archives/cpu2017.tar

Then go ahead and execute

%SPEC%\tools\src\buildtools.bat

You can ignore warnings about objects that are not found, especially at the beginning of each tool build, when cleanup is attempted from previous builds. Some tools may not know how to make 'clean', 'distclean', or 'realclean'; don't worry about it.

In Windows XP with Service Pack 2 or later, there may be a Security Alert, depending on the Firewall settings. "Windows Firewall has blocked this program from accepting connections from the Internet or a network." You may ignore this warning.

All compiler warnings seen as of July 2011 appear to be harmless.

3. How to verify that your build succeeded

After a tool build, you should:

  cd $SPEC (Unix) or %SPEC% (Windows)

  shrc.bat (Windows)
  . ./shrc (Unix, if you are in an sh-compatible shell.  
                  If not, start one!)  

  See if you can at least get as far as asking the major tools
  to identify themselves:

       runcpu -V 

  Or you can ask individual tools about themselves:

       specmake -v 
       specxz -h
       specperl -v
       specdiff -h
       runcpu -h 
       specinvoke -h 

Running runcpu --test will run more comprehensive tests on specperl.

4. Describing your newly-built tools

As a courtesy to others who might like to know which systems you intended your new tools to work on, and to describe the environment in which they were built, you may write a short description file. The contents of this file will be displayed when install.sh is run. Windows users only have one choice, and thus no description is printed. The file is $SPEC/tools/bin/<archname>/description. If that directory does not exist, you should create it. (If the directory does not exist, packagetools will create it, but in order for the description to be packaged, it must be present before packagetools is run.) See the "Packagetools" section below for guidance about picking "<archname>".

The format is very simple; in order to avoid wrapping on an 80-column screen, the first line must be no longer than 50 characters. In order to line up properly, subsequent lines should begin with 30 spaces and be no longer than 80 columns total.

For example, given $SPEC/tools/bin/turboblaster-m68k/description with the following contents

For TurboBlaster v2.1+ systems running on M68010
                             hardware.
                             Built on TurboBlaster v2.0 with GCC 1.81.

A user using a TurboBlaster system when running install.sh would see in the list of toolset choices, something that looks approximately like

turboblaster-m68k            For TurboBlaster v2.1+ systems running on M68010
                             hardware.
                             Built on TurboBlaster v2.0 with GCC 1.81.

There are plenty of examples on the install media in tools/bin/*/description.

4.a. Toolset presentation

If the toolset you're building isn't the only one that will work on a particular system, you may want to adjust where in the list of toolsets it appears, or even whether it appears at all. By default toolsets are presented in lexical ASCII order. However, it is possible to both adjust the position of a toolset in the list of candidates, as well as keeping it from appearing at all.

Let's assume that the TurboBlaster systems mentioned in the last section have migrated to PowerPC and have an emulation layer so that they can run old M68K binaries. Given two toolsets called turboblaster-m68k and turboblaster-ppc, normally the turboblaster-m68k would be presented and attempted first. And the emulation layer is so good that it would work, but it'd be slower than it needs to be.

4.a.1. Excluding otherwise valid candidates

There are several files that can be used to prevent install.sh from even attempting to use a toolset. The exclusion is silent, and for all intents and purposes makes the affected toolset behave just as one from another OS or architecture — it doesn't even show up on the candidate list.

To effect this exclusion, install.sh looks for signatures of the current system in files in each toolset's directory. A toolset is excluded from consideration if any of the following are true:

Output from... Appears in...
uname -m tools/bin/<archname>/excludearch
uname -s tools/bin/<archname>/excludearch
uname -r tools/bin/<archname>/excluderev
uname -p tools/bin/<archname>/excludeproc

On a PPC-based TurboBlaster system, uname -m returns "PowerPC". Putting "PowerPC" into tools/bin/turboblaster-m68k/excludearch would ensure that TurboBlaster/PPC users never even saw the TurboBlaster/M68K tools.

This is most useful for preventing cross-OS confusion. FreeBSD, for example, used to try to execute Solaris binaries. Putting "FreeBSD" (the output from uname -s on a FreeBSD system) into tools/bin/solaris-x86/excludearch fixed that problem and kept the unmaskable and alarming failure messages from appearing during installation.

4.a.2. Affecting presentation order

In the case where there may be two equally valid toolsets (as might be the case when there's a 32-bit and 64-bit build for the same OS), it's possible to change the order in which they're presented. Each toolset has a file named order in its directory. If it doesn't exist, packagetools will create this file with contents that are simply:

1:<archname>

(See the packagetools section for guidelines on picking <archname>.) The "priority" of a toolset decreases as that number increases. Continuing with the TurboBlaster example, if we wanted the PowerPC toolset to be preferred, we'd put

2:turboblaster-m68k

into tools/bin/turboblaster-m68k/order. That would ensure that turboblaster-ppc would appear first in the menu, as the number in its order file is "1". Since it appears first in the menu, it's installed before turboblaster-m68k.

5. Packagetools

If everything has succeeded, and you intend to submit results using your new tools, you should submit the tools to SPEC. To do so:

     cd $SPEC (Unix) or cd %SPEC% (Windows)
     packagetools <archname>

Pick an architecture name that other users will recognize. Check on the install media in tools/bin for some examples.

The packagetools script will create:

    $SPEC/tools/bin/<archname>/specxz
    $SPEC/tools/bin/<archname>/spectar
    $SPEC/tools/bin/<archname>/specmd5sum
    $SPEC/tools/bin/<archname>/tools-<archname>.tar.xz

Having created a large tarfile with everything in it, packagetools will then proceed to create an even larger tarfile with specxz and spectar it in too. This even bigger file is known as:

    $SPEC/<archname>-<version>.tar

and is to be submitted to SPEC. (See also the example below.)

You can optionally add components to your platform's toolset. For example, if you would like $SPEC/config/default.cfg to be set in an appropriate way, you can add the relative path to default.cfg as a parameter to packagetools:

(on non-Windows systems)
     cd $SPEC
     packagetools <archname> config/default.cfg

More than one file may be specified in this way.

Operation on Windows is substantially similar; just provide the relative paths with backslashes instead of forward slashes.

Please submit the resulting tarfile to SPEC for review, along with the recording of your tool build session. SPEC will review your tools, and assuming that they pass review, will add the tools you have built to its patch library, for possible distribution to future users of your interesting new architecture.

NOTE 1: If your operating system is unable to execute the packagetools script, please have a look at what the script does and enter the corresponding commands by hand. Again, you will need to submit the results to SPEC.

NOTE 2: Be sure to test your packaged tools on a different system, preferably one with a different disk layout. If the destination system is unable to invoke libperl.so, check that libperl.so exists in one of the locations where shrc expects to find it.

5.a. Example: use packagetools

Here is an example use of packagetools. In the example below, notice that:

  1  $ . ./shrc
  2  $ which runcpu
  3  /spec/cpu2017/kit117/bin/runXXX

XXX this example must be regenerated
4 $ 5 $ export MYTOOLS=myOS-myHW 6 $ mkdir tools/bin/$MYTOOLS 7 $ echo \ 8 "For systems running myOS. Note: works only with V12 or later" \ 9 tools/bin/$MYTOOLS/description 10 $ 11 $ cp typescript $MYTOOLS.buildlog.txt 12 $ runXXX -V > $MYTOOLS.runXXX-V.txt 2>&1 13 $ runXXX --test > $MYTOOLS.runXXX--test.txt 2>&1 14 $ 15 $ packagetools $MYTOOLS $MYTOOLS.buildlog.txt $MYTOOLS.runsp*txt 16 Preparing /spec/cpu2017/kit117/tools/bin/myOS-myHW 17 Generating checksums for the tools 18 Generating the tools installation tarball 19 100 % 12.0 MiB / 80.1 MiB = 0.149 1.0 MiB/s 1:17 20 Making the distribution tarball 21 tools/bin/myOS-myHW/ 22 tools/bin/myOS-myHW/order 23 tools/bin/myOS-myHW/tools-myOS-myHW.tar.xz 24 tools/bin/myOS-myHW/description 25 tools/bin/myOS-myHW/unbundled 26 tools/bin/myOS-myHW/specxz 27 tools/bin/myOS-myHW/specmd5sum 28 tools/bin/myOS-myHW/spectar 29 30 ******************************************************************* 31 32 The tarball to submit for inclusion in the distribution is 33 34 myOS-myHW-117.tar 35 36 ABSOLUTELY DO NOT submit the tarball in tools/bin/myOS-myHW 37 as it is not complete. 38 39 ******************************************************************* 40 41 $ specmd5sum myOS-myHW-117.tar > myOS-myHW-117.tar.md5

5.b. Example: test the newly packaged tools

Testing a newly-built toolset on the system where it was built is not enough to ensure basic sanity of the tools. Test for unintended dependencies by installing on an entirely different system. For example:

  1  $ cd /spec/cpu2017
  2  $ cp /net/(buildsys)/spec/cpu2017/kit117/myOS-myHW-117.tar* .
  3  $ cp /net/(buildsys)/spec/cpu2017/kit117/bin/spectar .
  4  $ cp /net/(buildsys)/spec/cpu2017/kit117/bin/specmd5sum .
  5  $ ls
  6  myOS-myHW-117.tar      myOS-myHW-117.tar.md5  specmd5sum             spectar
  7  $ ./specmd5sum -c myOS-myHW-117.tar.md5 
  8  myOS-myHW-117.tar: OK
  9  $ ./spectar -xvf myOS-myHW-117.tar
 10  tools/bin/myOS-myHW/
 11  tools/bin/myOS-myHW/spectar
 12  tools/bin/myOS-myHW/tools-myOS-myHW.tar.xz
 13  tools/bin/myOS-myHW/unbundled
 14  tools/bin/myOS-myHW/description
 15  tools/bin/myOS-myHW/specxz
 16  tools/bin/myOS-myHW/order
 17  tools/bin/myOS-myHW/specmd5sum
 18  $
 19  $ install.sh -u myOS-myHW
 20  .
 21  .
 22  .
 23  Installation successful.  Source the shrc or cshrc in
 24  /spec/cpu2017/kit117
 25  to set up your environment for the benchmark.
 26  
 27  
 28  $ . ./shrc
 29  $ runXXX -V > newsys.runXXX-V.txt 2>&1

XXX this example must be regenerated
30 $ runXXX --test > newsys.runXXX--test.txt 2>&1 31 $ 32 $ grep -c "...ok" *runXXX--test* 33 myOS-myHW.runXXX--test.txt:215 34 newsys.runXXX--test.txt:215 35 $ 36 $ spectar cvf - newsys.runXXX* | specxz > newsys.tests.tar.xz

6. Troubleshooting: What to do if something goes wrong

If something goes wrong, unfortunately, you're probably just going to have to take it apart and figure out what. Here are some hints on how to go about doing that.

6.a. Try it by hand

If something goes wrong, you probably do NOT want to make some random adjustment (like: reinstall a compiler, fix an environment variable, or adjust your path) and start all over again. That's going to be painful and take a lot of your time. Instead, you should temporarily abandon the buildtools script at that point and just try to build the offending tool, until you understand exactly why that particular tool is failing.

Consider turning on verbose diagnostics if your system has a way to do that. Make a huge terminal window (e.g. 200 columns wide by 84 lines tall, with 9999 lines recorded off the top), so you can see what is going on.

Read what buildtools (or buildtools.bat) does for you, then cd to tools/src/<tool> and try the commands by hand. For example, you might do something like this:

    cd $SPEC/tools/src/<toolname>
    ./configure
    make (or build.sh or whatever you note buildtools would have done)

Now, try fixing that environment variable or reinstalling that compiler, and rebuild the single tool. Does it look better?

If not, have a close look at the error messages and the Makefile. Does the Makefile use a feature that is not present in your version of make? If so, can you get it to work with GNU make?

Note that for GNU configure based tools (everything except Perl and its modules) you may specify your compiler by setting the CC environment variable. For compiler flags, set CFLAGS.

When building perl, note that:

Many of the build logs in tools/src/buildtools.log have examples of settings used for PERLFLAGS.

If you want to see more about what buildtools is doing for you, turn on your shell's verbose mode. For example:

   sh -x ./buildtools

6.b. Known problems

Try doing a web search to see if there are known problems with the tool on your architecture.

If SPEC supplies Version X.Y of a tool and it just won't build on your operating system, you might check whether there is a new Version X.Y+1 available. If so, download the new version to a scratch directory outside of the SPEC tree and try building it there. If that version succeeds, try to deduce why. Narrow it down to a one-line fix, won't you please? Then tell SPEC that you'd like the same one-line fix applied to its variant of the tool. Or, if you just can't narrow the fix down, ask SPEC whether it will allow use of Version X.Y+1 instead of X.Y on your system.

6.c. Executing just part of buildtools

Once you believe that you understand how to fix the problem tool, and can build it by hand, see whether the buildtools script can build it. You can execute just a portion of buildtools by defining environment variables. Please read the script itself to see what variables are allowed; the following are just some examples:

  SKIPALL      - turns off everything.  If you like, set this, then
                   turn individual phases on by setting them.
  DOTOOLSRM    - Remove previously installed tools
  DOCLEAN      - Run 'make clean', 'make distclean', and
                   'make realclean' in all source directories.
  DOMAKE       - build make
  DOXZ         - build xz
  DOTAR        - build tar
  DOSUM        - build specsha512sum, specsha256sum, and specmd5sum
  DOSPECINVOKE - build specinvoke
  DORXP        - build rxp
  DOEXPAT      - build the expat XML parser, used by XML::SAX::ExpatXS
  DOPERL       - build perl
  DOPERL2      - build perl modules
  DOCOPY       - copy the results to $SPEC/bin and fix shbangs

It doesn't matter what you set the environment variables to -- any non-zero-length string will do. In all cases, "DO" can be replaced with "SKIP" to reverse the sense of the setting.

If you are using Windows, be sure to note the WARNING in buildtools.bat about how the variables work.

If you can now build the (formerly misbehaving) tool using just part of buildtools, then you can try building the whole set (i.e. a build of all tools from start to finish). It should work now, so be sure to turn on your logging facility.

7. Support

You'll find information about technical support in techsupport.html, either in the same directory where you are found the document you are reading now, or at www.spec.org/cpu2017.

Building the SPEC CPU®2017 Toolset: Copyright © 2017 Standard Performance Evaluation Corporation (SPEC)