Table of Contents
|A. Who Needs runspec?|
|B. About Config Files|
|C. About Defaults|
|D. About Disk Usage and Support for Multiple Users|
|II. Before Using runspec|
Have a config file
Clean the environment
Set your path: Unix
Set your path: NT
Check your disk space
|III. Using runspec|
|A. Simplest usage|
|1. Reportable run|
|2. Running selected benchmarks|
|3. Output files|
|D. Commonly used options|
|E. Less commonly used options|
|IV. Quick reference|
Where can you find such a config file? There are 2 primary sources:
Look in directory $SPEC/config (Unix) or %SPEC%\config (NT). You may find that there is a already a config file there with a name that indicates that it is appropriate for your system. You may even find that default.cfg already contains settings that would be a good starting place for your system.
Look at the SPEC web site (http://www.spec.org/cpu2000/) for a CPU2000 result submission that used your system, or a system similar to yours. You can download the config file from that submission.
Note: links to SPEC CPU2000 documents on this web page
assume that you are reading the page from a directory that
also contains the other SPEC CPU2000 documents. If by
some chance you are reading this web page from a location
where the links do not work, try accessing the referenced
documents at one of the following locations:
$SPEC/docs/*.cfg (Unix) or %SPEC%\docs.nt\*.cfg (NT).
The order of precedence for settings is:
|Highest precedence:||runspec command|
|Lowest:||the tools as shipped by SPEC|
Therefore, when this document tells you that something is the
default, bear in mind that your config file may have changed that
setting. If in doubt, check the config file.
D. About Disk Usage and Support for Multiple Users
The structure of the CPU2000 directory tree is:
$SPEC or %SPEC% - the root directory benchspec CFP2000 - floating point benchmarks CINT2000 - integer benchmarks bin - tools to run and report on the suite config - config files result - log files and reports tools - sources for the CPU2000 toolsWithin each of the individual benchmarks, the structure is:
nnn.benchmark - root for this benchmark Spec - SPEC metadata about the benchmark data all - data used by all runs (if needed by the benchmark) ref - the real data set, required for all result reporting test - data for a simple test that an executable is functional train - data for feedback-directed optimization exe - compiled versions of the benchmark run - all builds and runs take place here src - the sources for the benchmarkWhen you find yourself wondering "Where did all my disk space go?", the answer is "The run directories." All build and run activity takes place in numbered run directories - e.g.
$SPEC/benchspec/CINT2000/164.gzip/run/00000001To get your disk space back, see the documentation of the various cleaning options, below; or issue a command such as the following (on Unix systems; NT users can select the files with Explorer):
rm -Rf $SPEC/benchspec/C*/*/runBecause all the build and run activity takes place in separately created directories, the tools are able to provide limited support for multiple users of the same directory tree:
The directory tree must be writable by each of the users, which means that the users have to trust each other not to modify or delete each others' files.
Directories such as config, result, and exe are not segregated by user, so you can only have one version of (for example) config/phil.cfg or benchspec/CINT2000/164.gzip/exe/gzip_base.les
Support for multiple users has received only limited testing during the development of CPU2000, and may have bugs. If you encounter a problem, please report it to SPEC.
Successfully install CPU2000.
Please see install_guide_unix.html
Find a config file.
Find a config file.
To check whether the variable is already defined, type
echo $SPEC (Unix) or echo %SPEC% (NT)On Unix systems, the desired output is nothing at all; on NT systems, the desired output is %SPEC%.
Set your path appropriately for your system type (Unix or NT/Windows 2000)a. If you are using a Unix system:
Change your current directory to the top-level SPEC directory and
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! !!!!!!!!!!!! START A BOURNE-COMPATIBLE SHELL !!!!!!!!!!!!! ! (if you are not already in one) ! ! ! ! That is, you need to use a command interpreter that ! ! supports the syntax used by shrc, which tries to be ! ! highly portable by avoiding proprietary or uncommon ! ! shell features. Many shells -- such as ash, bash, ! ! ksh, and zsh -- provide some degree of compatibility ! ! with the Bourne shell. The C shell (csh) definitely ! ! is NOT compatible with Bourne shell syntax. If you ! ! are in doubt, or if you see unexpected error messages ! ! from shrc, you can usually start a Bourne shell by ! ! typing 'sh'. ! !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
Then, source the file shrc. If you prefer some other shell, you can probably spawn it after sourcing shrc; when you do so, please be sure that your new shell does not clear the path variables set up by shrc! For example:
% cd /usr/rahul/cpu2000 % sh $ . ./shrc <- that's dot-space-dot-slash-shrc $ csh %b. If you are using a Windows/NT system:
Start a command window, then type:
notepad shrc.batand make the appropriate edits for your compiler paths.
Caution: you may find that the lines are not correctly formatted (the text appears to be all run together) when you edit this file. If so, see the section "Using Text Files on NT" in the NT installation guide.
Then set the path using your edited shrc.bat, for example:
For example, suppose that Wilfried has given Ryan a config file with some new integer optimizations for a Unix system. Ryan could say:
% cd /usr/ryan/cpu2000 % sh $ . ./shrc $ runspec --config wilfried_new.cfg --reportable intAs a another example, suppose that Reinhold has given Kaivalya an NT config file with changes from 12 August, and Kaivalya wants to run the floating point suite. He might say something like this:
F:\kaivalya\cpu2000> shrc F:\kaivalya\cpu2000> runspec --config reinhold_aug12a --reportable fp
% runspec --config arrian_dec25j.cfg --noreportable 252.eon 181.mcf
The format of the result files depends on what was selected in your config file, but will typically include at least .asc for ASCII text, and will always include .raw, for raw (unformatted) run data. More information about result formats can be found below, under --output_format. Note that you can always re-generate the output, using the --rawformat option, also documented below.
runspec [options] [list of benchmarks to run]Options are described in the following sections. There, you you will notice that many options have both long and short names. The long names are invoked using two dashes, and the short names use only a single dash. For long names that take a parameter, you can optionally use an equals sign. Long names can also be abbreviated, provided that you still enter enough letters for uniqueness. For example, the following commands all do the same thing:
runspec --config=dianne_july25a --debug=99 fp runspec --config dianne_july25a --debug 99 fp runspec --conf dianne_july25a --deb 99 fp runspec -c dianne_july25a -v 99 fpThe list of benchmarks to run can be either an entire suite ("int" or "fp") or one or more individual benchmarks. Individual benchmarks can be named, numbered, or both; and they can be abbreviated, as long as you enter enough characters for uniqueness. For example, each of the following commands does the same thing:
runspec -c jason_july09d --noreportable 187.facerec 168.wupwise runspec -c jason_july09d --noreportable 187 168 runspec -c jason_july09d --noreportable facerec wupwise runspec -c jason_july09d --noreportable face wupIt is also possible to exclude a benchmark, using a hat (^, also known as carat, typically found as shift-6). For example, suppose your system lacks a C++ compiler, and you therefore cannot run the benchmark 252.eon. You could run all of the integer benchmarks except eon by entering a command such as this one:
runspec --noreportable -c kathy_sep14c int ^eonNote that if hat has significance to your shell, you may need to protect it from interpretation by the shell, for example by putting it in single quotes. On NT, you will need to use both a hat and two quotes for each benchmark you want to exclude, like this:
E:\cpu2000> runspec --noreportable -c debbie_apr26a int "^gzip" "^eon"
If you want to force a different action, then you can enter one of the following runspec options:
|--action build||Compile the benchmarks. More information about compiling may be found in config.html, including information about additional files that are output during a build.|
|--action run||Run the benchmarks but do not bother to verify that they got the correct answers. This option is not normally useful, but can be selected if, for example, you are generating a performance trace and wish to avoid tracing some of the tools overhead.|
|--action setup||Setup the run directories. Copy executables and data to work directories.|
|Build (if needed), run, and generate reports|
|In addition, the following cleanup actions are available (in order by level of vigor):|
|--action clean||Empty all run and build directories for the specified benchmark set. For example, if user Jeff enters the command: runspec --action clean --config may12a fp the tools would remove run directories for the floating point benchmarks which are tagged as belonging to Jeff and which contain benchmarks generated by config file may12a.cfg.|
|--action trash||Same as clean, but do it for all users of this SPEC directory tree.|
|--action realclean||A synonym for --action trash|
|--action clobber||Clean + remove all executables of the current type|
|--action nuke||Remove everybody's run and build directories and all executables|
On the other hand, if you prefer to do all cleaning by hand (perhaps after reviewing the results of each run), you can ask the tools to never touch a used run directory. Do this by setting the environment variable: SPEC_CPU2000_NO_RUNDIR_NUKE
Alternative cleaning method:
If you prefer, you can clean disk space by entering commands such as the following (on Unix systems):
rm -Rf $SPEC/benchspec/C*/*/run rm -Rf $SPEC/benchspec/C*/*/exe
NT users can achieve a similar effect using Windows NT Explorer.
Notice that the above commands not only empty the contents of
the run and exe directories; they also delete the directories
themselves. That's fine; the tools will re-create the run and
exe directories if they are needed again later on.
D. Commonly used options
Most users of runspec will want to become familiar with the following
runspec --config golden --iterations 1 eonas the SPEC tools will inform you that you cannot change the number of iterations on a reportable run. But either of the following commands will override the config file and just run eon once:
runspec --config golden --iterations 1 --loose eon runspec --config golden --iterations 1 --noreportable eon
runspec --config ken_may12a --rate --users 4 intIf you have also entered --rawformat, then the effect of --rate is to format the rawfile for rate even if it was originally a speed run. That is, it is valid to report a single user run as both speed and rate, but only for the case where you first run it as a speed run, and then reformat for rate.
runspec --rawformat --output_format html,ps \ result/CINT2000.324.rawYou can achieve the same effect by using rawformat:
rawformat --output_format html,ps \ result/CINT2000.324.rawFor more information about rawformat, please see utility.html
For a better way of troubleshooting a problematic compile, see the information about specmake in utility.html
|-a||Same as --action|
|--action action||Do: build clean clobber nuke run setup trash or validate|
|-C||Same as --make_no_clobber|
|-c||Same as --config|
|--config file||Set config file for runspec to use|
|-D||Same as --rebuild|
|-d||Same as --deletework|
|--debug||Same as --verbose|
|--deletework||Force work directories to be rebuilt|
|-e||Same as --extension|
|--ext||Same as --extension|
|--extension ext||Set the extension|
|-f||Same as --unbuffer|
|-h||Same as --help|
|--help||Print usage message|
|-I||Same as --ignore_errors|
|-i||Same as --size|
|--ignore_errors||Continue with benchmark runs even if some fail|
|--ignoreerror||Same as --ignore_errors|
|--input||Same as --size|
|--iterations N||Run each benchmark N times|
|-l||Same as --loose|
|--loose||Do not produce a reportable result|
|--noloose||Same as --reportable|
|-m||Same as --machine|
|--mach||Same as --machine|
|--machine name||Set the machine type|
|--make_no_clobber||Do not delete existing object files before building.|
|--max_active_compares||Same as --maxcompares|
|--maxcompares N||Set the number of concurrent compares to N|
|-n||Same as --iterations|
|--noreportable||Same as --loose|
|-o||Same as --output_format|
|--output_format format||Generate report, one or more of: asc html pdf ps config|
|-R||Same as --rawformat|
|-r||Same as --rate|
|--rate||Do a throughput (rate) run|
|--rawformat||Format raw file|
|--rebuild||Force a rebuild of binaries|
|--reportable||Produce a reportable result|
|-s||Same as --reportable|
|--[no]setprocgroup||[Don't] try to create all processes in one group.|
|--size||Select data set, one of: test train ref|
|--strict||Same as --reportable|
|--nostrict||Same as --loose|
|-T||Same as --tune|
|--[no]table||Do [not] include a detailed table of results|
|--tune||Set the tuning levels to one of: base peak all|
|--tuning||Same as --tune|
|-u||Same as --users|
|--users||Set the number of users for a rate run|
|-U||Same as --username|
|--unbuffer||Turn output buffering off|
|--username||Name of user to tag as owner for run directories|
|-v||Same as --verbose|
|--verbose||Set verbosity level for messages to N|
|-V||Same as --version|
|--version||Output lots of version information|
|-?||Same as --help|