Last updated: 7-Feb-2007 bw/jh/ya
To check for possible updates to this document, please see http://www.spec.org/mpi2007/Docs/
Contents
Editing result description fields
Generating a rate result from a speed result
This document describes various useful utilities that ship with SPEC MPI2007. It is assumed that you have already read runspec.html and config.html.
Note: links to SPEC MPI2007 documents on this web page assume that you are reading the page from a directory that also contains the other SPEC MPI2007 documents. If by some chance you are reading this web page from a location where the links do not work, try accessing the referenced documents at one of the following locations:
Most of the examples in this document have been written in Unix syntax. That means you'll need to use your imagination just a bit as you read this document:
| If you see: | Just pretend you saw: |
| cat | type |
|---|---|
| cp | copy |
| diff | fc |
| grep | findstr |
| ls | dir |
| / | \ |
| $SPEC | %spec% |
Also, some of the scripts cannot be invoked directly, but instead must be invoked by putting
specperl %spec%\bin\
in front of the script name. This is noted in the sections where it is needed.
"configpp" is a short way to say:
$ runspec --action configpp
That switch is mentioned in runspec.html but you'll find much more about the config file preprocessor in config.html. The configuration file is specified using the --config switch, or -c for short, at the command line. For example, the following two command are equivalent:
$ runspec --action configpp -c try_utilities.cfg $ configpp -c try_utilities.cfg
There are a couple of points worth mentioning here:
The configpp utility is a good way to check your configuration file for fatal errors, as the config file is completely parsed before it's output.
If you have a rawfile, you can extract the config file that was used with it. To do so, use the extract_config utility.
For example, starting with a freshly-installed SPEC MPI2007 directory tree:
% runspec --config=try_utilities --size=test --iterations=1 \ --tuning=base --output_format=text,raw bzip2 > /dev/null % cd result % ls MPIM2007.001.test.rsf MPIM2007.001.test.txt MPI2007.001.log MPI2007.lock
Note that we have just the single result. Let's make a temporary directory and copy the rawfile there:
% mkdir tmp % cd tmp % cp ../MPIM2007.001.test.rsf . % ls MPIM2007.001.test.rsf
(On Windows, that ls command would be spelt dir. You'll find some more Windows notes in the introduction.)
Now, attempt to extract the config file:
% extract_config MPIM2007.001.test.rsf > tmp.cfg
Does the extracted file match the original?
% diff -u ../../config/try_utilities.cfg tmp.cfg
--- ../../config/try_utilities.cfg 2006-05-22 14:41:42.000000000 -0700
+++ tmp.cfg 2006-05-22 14:43:16.000000000 -0700
@@ -1,3 +1,6 @@
+# Invocation command line:
+# /Volumes/storage/WorkRelated/SPEC/MPI2007/bin/runspec
--config=try_utilities --size=test --iterations=1 --tuning=base
--output_format=asc,raw bzip2
+############################################################################
# This is a sample config file. It was tested with:
#
# Compiler name/version: Apple GCC v4.0.1 build 5245
@@ -128,15 +131,8 @@
sw_state = Default
sw_other = None
-__MD5__
-104.milc=base=try_utilities=default:
-# Last updated Mon May 22 14:41:42 2007
-optmd5=0666e60587f33e2aa13eb6ba0917aa60
-baggage=
-compile_options=\
-@eNqNj00LgkAQhu/7K4a9b5duoUKuJpa6C+qlS9QmYqAjaYf+faMF2cfBYQ8Dsy/P80oV6xWUxoCg\
-h4BtX2HTLRCEl2pfHqTOaU08380DoBFqCaI+35tjXRnRoGgrCpbwNVaHt6spHCZXIKXNicDHXblb\
-pTObf+A4IwkibaJ1kNLtF82ZGsI6C+Nw79OX/xqcRWGyexaa4Wrh6VKYvnMm5m+rwTfyXu4z8WNE\
-5UPBSbsHPPNh9Q==
-exemd5=d828496b92983e96408f680b55393225
%
Yes. The only differences are:
Instead of saying diff just above, you would say fc (and, as already mentioned, dir is the Windows spelling for what Unix calls ls. You can invoke extract_config as:
specperl %spec%\bin\extract_config rawfile
If you have a pdf, html, or ps output file, you can extract the rawfile that was used to generate it. To do so, use extract_raw.
For example:
% runspec --config=try_utilities --size=test --iterations=1 \ --tuning=base --output_format=ps bzip2 > /dev/null % cd $SPEC/result % ls MPIM2007.001.test.rsf MPIM2007.001.test.ps MPI2007.001.log MPI2007.lock % mkdir tmp % cd tmp % cp ../*ps . % ls MPIM2007.001.test.ps
Note that at this point we have just the postscript output in the current directory. Now, attempt to extract the raw file:
% extract_raw *ps Wrote "MPIM2007.001.test.rsf" % ls MPIM2007.001.test.ps MPIM2007.001.test.rsf
Does the extracted file match the original?
% diff -u ../MPI2007.001.rsf ./MPI2007.001.rsf %
Yes.
You can invoke extract_raw as:
specperl %spec%\bin\extract_raw file-to-extract-from
Note that the file you extract from can be pdf, ps, or html, just as on Unix systems, with one important exception: if you are using a browser that attempts to add value to the html when you save it, that browser may break the embedded rawfile. For example, with Internet Explorer Version 5.00.3103.1000, if you are viewing a SPEC MPI2007 html file and you select File -> Save As, the default for "Save as type" is shown as
"Web Page, complete (*.htm,*.html)"
Do not accept that default if you want to be able to retrieve the rawfile later on. If you take the default, your saved file will be incomplete. To save the web page INCLUDING the embedded rawfile, you must change "Save as type" to
"Web Page, HTML only (*.htm,*.html)"
If you have a rawfile, you can extract the XML flags description file that the result uses. To do so, use extract_flags. Usage is identical to extract_raw, but unlike extract_raw, may not produce an flags description file identical to the input. This is because EOLs (end-of-line) characters are normalized to Unix-style LF (line feed) only.
Note: If you have used a flags description file to format your result, the entire flags file is stored in the raw file (not just the part needed for result).
For example, if the flags description file "flags-macosx-ia32-iccifortv91.xml" was used to format the MPIM2007.002.rsf raw file, then you can extract the original flags file using the following command
% pwd
/mpi2007/mpi2007-kit92/result
% ls
MPIM2007.002.rsf images
% extract_flags MPIM2007.002.rsf > tmp.xml
% ls ../*xml
../flags-macosx-ia32-iccifortv91.xml
% diff -u ../flags-macosx-ia32-iccifortv91.xml tmp.xml
%
We see absolutely no difference between the extraced flags description file "tmp.xml" and the original flags description file "flags-macosx-ia32-iccifortv91.xml"
Note: If a raw file has not been formatted with a flags description file, extract_flags does nothing.
If you have an XML flag description file, such as one extracted using extract_flags, or one downloaded from SPEC, flag_dump will create a more human-readable HTML version from it.
For example:
% cd $SPEC/Docs
% ls *xml
flags-advanced.xml flags-simple.xml
% cp flags-advanced.xml chris_flags.xml
% flag_dump chris_flags.xml
Processing chris_flags.xml...wrote chris_flags.html
% ls chris_flags.*
chris_flags.html chris_flags.xml
Note that flag_dump utility assumes that you are formatting flags files written by users. These files are not allowed to contain mandatory flags. If you attempt to use the flag_dump utility to format any of the suite-provided flag files such as flags_mandatory.xml or the benchmark flag files that reside in $SPEC/benchspec/MPI2007/*/Spec/ directory, the utility will exit with an error message such as the one below.
% cd $SPEC/benchspec
% flag_dump flags_mandatory.xml
Processing flags_mandatory.xml...
ERROR: The flag file at
flags_mandatory.xml
contains one or more flags in the class "mandatory", which is not
allowed.
%
When you source the shrc, as described in runspec.html, the 'go' shell alias is created. go is a handy utility to navigate the directory structure of a SPEC source tree. Here are the various things it understands and the places they will take you:
| If you say | You'll end up in |
| go
go top | $SPEC |
|---|---|
| go bin | $SPEC/bin (where the tools live) |
| go config | $SPEC/config |
| go result
go results | $SPEC/result |
| go doc
go docs | $SPEC/Docs |
| go cpu | $SPEC/benchspec/MPI2007 |
| go benchmark | The top level directory for the named benchmark, if it can be found. (Examples follow) |
| go 126 | $SPEC/benchspec/MPI2007/126.lammps |
| go zeus | $SPEC/benchspec/MPI2007/132.zeusmp2 Notice that abbreviated names are accepted. But if the abbreviation is ambiguous, the command will pick the first match, so pay attention to the reply: $ go G /spec/mpi2007/kit91/benchspec/MPI2007/113.GemsFDTD $ go GA /spec/mpi2007/kit91/benchspec/MPI2007/128.GAPgeofem $ |
| go nada | no change: there's no benchmark named nada* |
| go benchmark subdir | The named subdirectory of the directory tree for the referenced benchmark, for example "go tera run". You can also name more than one level of subdirectory: "go 129 data train". |
| go subdir | If you are already in a benchmark subtree, then you can move to a different branch of the same benchmark, by naming the branch and not mentioning a benchmark. For example: $ pwd /ronl/benchspec/MPI2007/127.wrf2/run/build_base_a3.0000 $ go exe /ronl/benchspec/MPI2007/127.wrf2/exe $ |
Note: If you leave the shell where you sourced shrc, the alias is no longer available! So if you want to use this alias, do not start a sub-shell.
Note: 'go' does not know about your settings for output_root or expid, and will not be able to take you to your run, exe, or result directories if you use those features. See ogo for something that can help.
If you use a csh-like shell, please note that after you say source cshrc, there is a go alias that is set up for you, which implements the same features as described above.
Sorry, the go feature does not exist for Windows systems.
When you source shrc, as described in runspec.html, the 'ogo' shell alias is created. Its function is similar to the go alias with one important difference: if you use the output_root setting in your config file (also discussed in runspec.html) you can use ogo as a shorthand for moving around both the original installed benchmark tree and your output tree.
To do so, simply set the GO environment variable to the same value as your output_root. Then, 'ogo' will use the contents of that variable as the top level of the benchmark tree when looking for run directories and results.
For example, you could say something like this:
$ go config /spec/config $ grep output_root cds.cfg output_root = /cds_stuff $ GO=/cds_stuff $ ogo run perl Using value in GO for output_root: /cds_stuff /cds_stuff/benchspec/MPI2007/104.milc/run $
In this example, the benchmarks are installed in "/spec", and output_root is set to "/cds_stuff" in the config file. Any runs using that config file will write their results, executables, and run directories under "/cds_stuff". Continuing with other examples:
| If you say | You'll end up in |
| ogo result | /cds_stuff/result
Your own private result directory |
|---|---|
| ogo 104 run | /cds_stuff/benchspec/MPI2007/104.milc/run
Your own private run directories |
| ogo wrf2 exe | /cds_stuff/benchspec/MPI2007/127.wrf2/exe
Your own private executables |
| ogo doc | /spec/Docs
There's only one Docs directory, on the original installed $SPEC tree |
| ogo config | /spec/config
The config directory is always on the original installed tree; set its protections as described in runspec.html |
| ogo 104 src | /spec/benchspec/MPI2007/104.milc/src
Run directories are under output_root, but the benchmark src directory is still in the original installed $SPEC tree. |
Note: It's not necessary to export the "GO" variable. Declaring this variable at the command-prompt is sufficient.
$ GO=/tmp/cds
$ echo $GO
/tmp/cds
ogo has also been implemented for cshrc. There is one restriction: to get the full feature set above, you must set OGO to the actual directory, not to a symbolic link.
Sorry, the ogo feature does not exist for Windows systems.
Some benchmark executables are invoked multiple times. If you'd like to know how much time was spent on each invocation, it may not be easy to tell. You could examine "speccmds.out", but that file is not easy to interpret:
$ runspec --config=mpi --size=train --iterations=1 \
--tuning=base --output_format=ps 126
$ go work 126
$ cd run_base_train_mpi.0000/
$
$ cat speccmds.out | fold -s
running commands in speccmds.cmd 1 times
runs started at 1170773904, 112144000, Tue Feb 6 06:58:24 2007
run 1 started at 1170773904, 112302000, Tue Feb 6 06:58:24 2007
child started: 0, 1170773904, 112324000, pid=26456, 'mpirun -mca btl
sm,self,tcp -np 64 -host sumocat ../run_base_train_mpi.0000/lammps_base.mpi
-in in.chain -log none'
child finished: 0, 1170773914, 530202000, sec=10, nsec=417878000, pid=26456, rc=0
child started: 0, 1170773914, 530301000, pid=26591, 'mpirun -mca btl
sm,self,tcp -np 64 -host sumocat ../run_base_train_mpi.0000/lammps_base.mpi -in in.chute -log none'
child finished: 0, 1170773927, 437473000, sec=12, nsec=907172000, pid=26591, rc=0
child started: 0, 1170773927, 437527000, pid=26723, 'mpirun -mca btl
sm,self,tcp -np 64 -host sumocat ../run_base_train_mpi.0000/lammps_base.mpi
-in in.eam -log none'
child finished: 0, 1170773938, 583633000, sec=11, nsec=146106000, pid=26723, rc=0
child started: 0, 1170773938, 583680000, pid=26855, 'mpirun -mca btl
sm,self,tcp -np 64 -host sumocat ../run_base_train_mpi.0000/lammps_base.mpi
-in in.lj -log none'
child finished: 0, 1170773949, 741938000, sec=11, nsec=158258000, pid=26855, rc=0
run 1 finished at: 1170773949, 741967000, Tue Feb 6 06:59:09 2007
run 1 elapsed time: 45, 629665000, 45.629665000
runs finished at 1170773949, 742016000, Tue Feb 6 06:59:09 2007
runs elapsed time: 45, 629872000, 45.629872000
$
Fortunately, SPEC has provided a tool to interpret the above for you:
$ printkids.pl
Seconds Command
10.42 btl sm,self,tcp -np 64 -host sumocat ../run_base_train_mpi.0000/lamm
12.91 btl sm,self,tcp -np 64 -host sumocat ../run_base_train_mpi.0000/lamm
11.15 btl sm,self,tcp -np 64 -host sumocat ../run_base_train_mpi.0000/lamm
11.16 btl sm,self,tcp -np 64 -host sumocat ../run_base_train_mpi.0000/lamm
=========
45.63 Total by adding children
45.63 Total according to speccmds.out
$
If you rearrange mount points or rename your SPEC root directory, you will probably find confusing problems like the one in the example that follows. We used to have a working SPEC tree under /mpi2007/mpi2007-kit51, but we're going to change that:
$ cd /mpi2007
$ mv mpi2007-kit51 jamesbond
$ cd jamesbond
$ . ./shrc
$ runspec -h
-bash: /mpi2007/jamesbond/bin/runspec: /mpi2007/mpi2007-kit51/bin/specperl:
bad interpreter: No such file or directory
The problem is that the first line of runspec points to a path that no longer exists:
$ head -1 bin/runspec
#!/mpi2007/mpi2007-kit51/bin/specperl
$
Fortunately, SPEC has provided a tool to relocate the tools in the new tree:
$ cd /mpi2007/jamesbond
$ . ./shrc
$ relocate
Top of SPEC benchmark tree is '/mpi2007/jamesbond'
$ runspec -h | head -3
runspec v4256 - Copyright 1999-2007 Standard Performance Evaluation Corporation
Using 'macosx-ppc' tools
Reading MANIFEST... 2565 files
Limitations: There are some limitations to this tool. Most notably, it uses perl, so if the tree has been re-arranged in such a way that perl has an identity crisis and can't find itself, you'll be in trouble. Make sure that you have sourced shrc, as in the next example; if it still fails, you may need to reinstall the MPI2007 kit.
$ cd /mpi2007
$ mv jamesbond dr_evil
$ cd dr_evil
$ bin/relocate
Your $SPEC variable is not set correctly!
Top of SPEC benchmark tree is '/mpi2007/dr_evil'
Can't locate strict.pm in @INC (@INC contains: /mpi2007/jamesbond/bin
/mpi2007/jamesbond/bin/lib .) at bin/relocate line 3.
BEGIN failed--compilation aborted at bin/relocate line 3.
$ . ./shrc
$ bin/relocate
Top of SPEC benchmark tree is '/mpi2007/dr_evil'
$ runspec -h | head -2
runspec v4256 - Copyright 1999-2007 Standard Performance Evaluation Corporation
Using 'macosx-ppc' tools
Reading MANIFEST... 2565 files
$
"rawformat" is simply another name for:
runspec --rawformat
So please see runspec.html for basic information on using this command. But a few additional points are worth mentioning:
Editing result description fields
What do you do when Marketing decides to change the name from "SuperHero 4" to "SuperHero IV" just before you publish the results, or just before you're ready to submit them to SPEC?
SPEC does allow you to edit the rawfile to make corrections. Just make sure that you only edit the fields located before the "fence":
# =============== do not edit below this point ===================
WARNING: Make a backup copy of the rawfile first!
For example:
$ cd $SPEC/result
$ cp MPIM2007.001.rsf new.rsf
edit new.rsf to make your changes
$ diff MPIM2007.001.rsf new.rsf
9c9
< spec.mpi2007.hw_model: SuperHero 4
---
> spec.mpi2007.hw_model: SuperHero IV
$
$ rawformat -o ps new.rsf
runspec v4256 - Copyright 1999-2007 Standard Performance Evaluation Corporation
Using 'macosx-ppc' tools
Reading MANIFEST... 2565 files
Loading runspec modules...............
Locating benchmarks... found 14 benchmarks in 2 benchsets.
Locating output formats: ASCII, config, CSV, flags, html, mail, PDF,
PostScript, raw, Screen, Submission Check
Formatting new.rsf
Parsing flags for 104.milc base: done
Parsing flags for 107.leslie3d base: done
Parsing flags for 113.GemsFDTD base: done
Parsing flags for 115.fds4 base: done
Parsing flags for 121.pop2 base: done
Parsing flags for 122.tachyon base: done
Parsing flags for 126.lammps base: done
Parsing flags for 127.wrf2 base: done
Parsing flags for 128.GAPgeofem base: done
Parsing flags for 129.tera.tf base: done
Parsing flags for 130.socorro base: done
Parsing flags for 132.zeusmp2 base: done
Parsing flags for 137.lu base: done
Doing flag reduction: done
format: PostScript -> new.ps
There is no log file for this run.
runspec finished at Tue May 23 15:09:25 2007; 3 total seconds elapsed
$
About continuation fields in rawfiles
Notice when editing fields that have more than one line that you need to construct the numbers with correct syntax. The syntax in the rawfile is a bit more picky than in config files (compare the following vs. the config.html section on "Field scoping and continuation". You cannot say something like this:
| WRONG: |
spec.mpi2007.sw_compiler: Intel C Compiler 9.1 for Windows XP
spec.mpi2007.sw_compiler1: Visual Studio 2005 (for libraries) |
because the tools are not prepared at this point for a mixture of numbered and un-numbered fields. If you need to have continued fields, say instead:
| RIGHT: |
spec.mpi2007.sw_compiler0: Intel C Compiler 9.1 for Windows XP
spec.mpi2007.sw_compiler1: Visual Studio 2005 (for libraries) |
This "Submission Check" always runs when I do rawformat
Because rawformat is used most often to check raw file edits for results that are to be submitted to SPEC for publication, the subcheck output format is automatically selected for all rawformat runs.
You may find that --output_format will only accept one argument at a time. So, first create the rawfile, by using --output_format raw, then use the new rawfile to create the other reports.
In order to decide whether your system got the correct answer for a benchmark, runspec writes a file called compare.cmd in the run directory, and then calls specinvoke, which calls specdiff. (The specinvoke utility is described below.)
Suppose that your results are flagged as incorrect, but you'd like to have more detail. You could ask for more than 10 lines of detail by changing the config file's default for difflines from 10 to something else, and re-running the experiment. But, you might save a lot of time and avoid the re-run of the experiment, by calling specdiff directly, with a different number of lines for its switch -l.
For example, a file chock-full of wrong answers for 107.leslie3d has been created by hand. First, let's see how specdiff would normally be run (line wraps added for readability):
$ go run namd
$ cd run_base_test_try_utilities.0000/
$ specinvoke -c 1 -f compare.cmd -n
# Use another -n on the command line to see chdir commands
# Starting run for copy #0
specperl /mpi2007/dr_evil/bin/specdiff -m -l 10 --abstol 1e-05 \
/mpi2007/dr_evil/benchspec/MPI2007/107.leslie3d/data/mref/output/tml.dat tml.dat > tml.dat.cmp
The above says that normally we test for correct answers by comparing the file $SPEC/benchspec/MPI2007/107.leslie3d/data/test/output/tml.dat versus the file tml.dat in the current directory. Normally, 10 lines of detail are printed. Let's change that to 15 lines, and compare versus our intentionally wrong file instead:
$ specdiff -m -l 15 --abstol 1e-05 \
/mpi2007/dr_evil/benchspec/MPI2007/107.leslie3d/data/mref/output/tml.dat tml_wrong.dat
0019: 0.23072 1.04730
0.23072 1.04732
^
0021: 0.25502 1.04779
0.25502 1.04781
^
0022: 0.26717 1.04804
0.26717 1.04806
^
0025: 0.30362 1.04878
0.30362 1.04880
^
0026: 0.31578 1.04903
0.31578 1.04905
^
0027: 0.32794 1.04928
0.32794 1.04930
^
0028: 0.34010 1.04953
0.34010 1.04955
^
0031: 0.37656 1.05027
0.37656 1.05029
^
0032: 0.38872 1.05052
0.38872 1.05054
^
0033: 0.40088 1.05077
0.40088 1.05079
^
0034: 0.41304 1.05102
0.41304 1.05104
^
0035: 0.42520 1.05127
0.42520 1.05129
^
0036: 0.43736 1.05151
0.43736 1.05153
^
0037: 0.44952 1.05176
0.44951 1.05178
^
0038: 0.46167 1.05201
0.46167 1.05203
^
specdiff run completed
$
Here are the switches supported by specdiff:
$ specdiff -h
Usage: /Users/cloyce/SPEC/mpi2007/bin/specdiff <options> file1 [file2]
-l,--lines # of lines of differences to print (-1 for all)
-q,--quiet don't print lines just set return code
-a,--abstol absolute tolerance (for floating point compares)
-r,--reltol relative tolerance (for floating point compares)
-o,--obiwan allow off-by-one errors
-O,--OBIWAN *don't* allow off-by-one errors
-s,--skiptol # of differences to ignore
--skipabstol # of abstol differences to ignore
--skipreltol # of reltol differences to ignore
--skipobiwan # of off-by-one differences to ignore
-t,--calctol calculate required tolerances
-H,--histogram show a histogram of differences
-f,--floatcompare Do comparison for floating-point values even if no
tolerances are specified. (Normally this is
automatic.)
-m,--mis write file2.mis with miscompares
-c,--cw collapse whitespace (probably doesn't do what you
think it does)
-C,--CW *don't* collapse whitespace
-b,--binary Do byte-wise comparison for binary outputs
-i,--ignorecase Ignore case for string comparisons
-d,--datadir Compare against file(s) in data directory
--os Set the operating system type (this is unnecessary)
-v,--verbose Set the level of noisiness for the output
-h,--help Print this message
$
In order to run a benchmark, runspec writes a series of commands in the run directory, in the file speccmds.cmd. These commands are then interpreted by specinvoke.
When something goes wrong, it is often useful to try the commands by hand. Ask specinvoke what it did, by using the "-n" switch:
$ go run lu
/mpi2007/dr_evil/benchspec/MPI2007/137.lu/run
$ cd run_base_mref_mpi.0000
$ ls
compare.cmd lu.input lu_base.mpi.err speccmds.stdout
compare.err lu.output speccmds.cmd
compare.out lu.output.cmp speccmds.err
compare.stdout lu_base.mpi speccmds.out
$ specinvoke -n
# Use another -n on the command line to see chdir commands
# Starting run for copy #0
mpirun -mca btl sm,self,tcp -np 64 -host sumocat \
../run_base_mref_mpi.0000/lu_base.mpi \
> lu.output 2>> lu_base.mpi.err
$
In the above example, we see how lu is invoked (line wraps added for readability). If you wanted to try out your own variations on the above command by hand, you could do so.
Here are the switches supported by specinvoke:
$ specinvoke -h
Usage: specinvoke [options] [instructionfile]
-i # iterations
-c # concurrent processes (overrides -u in command file)
-E return non-zero exit code if child does
-s shell shell to invoke
-f file instruction file
-o file output file
-e file error file
-d dir change to dir first
-n[n] print a 'dry_run' of commands
-A return 0 error code
-S msecs sleep between spawning copies (in milliseconds)
-r don't do I/O redirection ($command already has it)
-N open null device when no input file is specified
-Z use zero-length file when no input file is specified
-C (default) close stdin when no input file is specified
-h this message
To build a benchmark, runspec uses specmake, which is simply gnu make under a unique name to avoid possible conflicts with other versions of make on your system.
If a benchmark build fails, one of the most massively useful things you can do is to ask specmake what it did, by using the "-n" switch, which means "dry run".
For example, let's create a config file that is unlikely to work:
$ go config
$ cat > tmp.cfg
ext = silly
FC = f93
$
and try it out:
$ runspec --config=tmp --tuning=base --action=build leslie3d
runspec v4256 - Copyright 1999-2007 Standard Performance Evaluation Corporation
Using 'macosx-ppc' tools
Reading MANIFEST... 2565 files
Loading runspec modules...............
Locating benchmarks...found 31 benchmarks in 13 benchsets.
Locating output formats: ASCII, config, CSV, flags, html, mail, PDF,
PostScript, raw, Screen, Submission Check
Reading config file '/mpi2007/dr_evil/config/tmp.cfg'
Benchmarks selected: 107.leslie3d
Compiling Binaries
Building 107.leslie3d ref base silly default: (build_base_silly.0000)
Error with make 'specmake build': check file
'/mpi2007/dr_evil/benchspec/MPI2007/107.leslie3d/run/build_base_silly.0000/make.err'
Error with make!
*** Error building 107.leslie3d
If you wish to ignore this error, please use '-I' or ignore errors.
The log for this run is in /mpi2007/dr_evil/result/MPI2007.009.log
runspec finished at Thu May 25 10:01:54 2007; 8 total seconds elapsed
$
Sure enough, it failed. Let's find the appropriate run directory where our silly build was attempted:
$ go work leslie3d
/mpi2007/dr_evil/benchspec/MPI2007/107.leslie3d/run
$ ls
build_base_silly.0000 list
$ cd build_base_silly.0000
And ask specmake just exactly what it generated:
$ specmake -n
f93 -c -o modules.o modules.f
f93 -c -o flowio.o flowio.f
f93 tml.o -o leslie3d
f93 modules.o flowio.o fluxi.o fluxj.o fluxk.o \
grid.o main.o parallel.o props.o setbc.o setiv.o tmstep.o \
update.o alloc.o -lmpi -o leslie3d
OK, let's enter the command by hand:
$ f93 -c -o modules.o modules.f
-bash: f93: command not found
$
A great convenience of the MPI2007 suite is the fact that you can now proceed to play with all the files in this run directory, to prototype different changes and try out ideas, without fear that you will break anything. As long as you confine your edits to (in this example) $SPEC/benchspec/MPI2007/107.leslie3d/run/build_base_silly.0000, you are in a private sandbox and will not harm the original sources.
Here are the switches supported by specmake:
$ specmake -h
Usage: specmake [options] [target] ...
Options:
-b, -m Ignored for compatibility.
-B, --always-make Unconditionally make all targets.
-C DIRECTORY, --directory=DIRECTORY
Change to DIRECTORY before doing anything.
-d Print lots of debugging information.
--debug[=FLAGS] Print various types of debugging information.
-e, --environment-overrides
Environment variables override makefiles.
-f FILE, --file=FILE, --makefile=FILE
Read FILE as a makefile.
-h, --help Print this message and exit.
-i, --ignore-errors Ignore errors from commands.
-I DIRECTORY, --include-dir=DIRECTORY
Search DIRECTORY for included makefiles.
-j [N], --jobs[=N] Allow N jobs at once; infinite jobs with no arg.
-k, --keep-going Keep going when some targets can't be made.
-l [N], --load-average[=N], --max-load[=N]
Don't start multiple jobs unless load is below N.
-n, --just-print, --dry-run, --recon
Don't actually run any commands; just print them.
-o FILE, --old-file=FILE, --assume-old=FILE
Consider FILE to be very old and don't remake it.
-p, --print-data-base Print make's internal database.
-q, --question Run no commands; exit status says if up to date.
-r, --no-builtin-rules Disable the built-in implicit rules.
-R, --no-builtin-variables Disable the built-in variable settings.
-s, --silent, --quiet Don't echo commands.
-S, --no-keep-going, --stop
Turns off -k.
-t, --touch Touch targets instead of remaking them.
-v, --version Print the version number of make and exit.
-w, --print-directory Print the current directory.
--no-print-directory Turn off -w, even if it was turned on implicitly.
-W FILE, --what-if=FILE, --new-file=FILE, --assume-new=FILE
Consider FILE to be infinitely new.
--warn-undefined-variables Warn when an undefined variable is referenced.
This program built for powerpc-apple-darwin6.8
Report bugs to <bug-make@gnu.org>
If you have a rawfile, you can get information about the binary toolset and the versions of all of the system-independent tools files. To do so, use the toolsver utility.
For example, starting with a freshly-installed SPEC MPI2007 directory tree:
% runspec --config=try_utilities --size=test --iterations=1 \
--tuning=base --output_format=ps > /dev/null
% cd $SPEC/result
% ls
MPIM2007.001.test.ps MPIM2007.001.test.rsf MPI2007.001.log MPI2007.lock
Note that we have just the single result file. Now, attempt to extract the version information:
% toolsver MPIM2007.001.test.rsf
MPIM2007.001.test.rsf:
Binary toolset used was 'solaris-sparc'
Versions of Perl-based tools:
430 runspec
430 CPUFlagsParser.pm
445 PSPDF.pm
445 asc.pl
445 benchmark.pm
445 benchset.pm
430 cfgfile.pl
430 compare.pl
445 config.pl
445 csv.pl
445 flags.pl
435 flagutils.pl
300 format.pm
445 html.pl
430 listfile.pm
445 locate.pl
430 log.pl
430 mail.pl
300 monitor.pl
430 os.pl
492 parse.pl
430 pdf.pl
445 ps.pl
445 raw.pl
300 screen.pl
445 subcheck.pl
492 util.pl
506 vars.pl
You can invoke toolsver as:
specperl %spec%\bin\toolsver rawfile
Copyright (C) 1999-2007 Standard Performance Evaluation Corporation
All Rights Reserved