SPEC CPU2017 Result File Fields

$Id: result-fields.html 5779 2017-06-02 17:06:21Z CloyceS $ Latest: www.spec.org/cpu2017/Docs/

Contents: This document provides a glossary that briefly defines terms that are used in SPEC CPU2017 reports. Typically you arrive somewhere in the middle of the document by following a link from a report; rarely would someone sit down to read this top to bottom.
If you are that rare someone: Welcome! Here is a table of contents to tell you what lies ahead.

Top Matter

Report Titles

Performance Metrics

Energy Statistics

Tester and Date Info

Benchmark-by-benchmark result details

Results Table

Descriptions

Tester-provided notes

Flags

Hardware description

Software description

Power and Temperature information

Other information

Report Titles

SPEC CPU2017 Floating Point Rate Result Report for measurements that use a suite of 13 floating-point intensive benchmarks.
Higher scores = more throughput.
The tester chooses how many copies to run.
[Suites and Benchmarks] [SPECspeed and SPECrate]
SPEC CPU2017 Floating Point Speed Result Report for measurements that use a suite of 10 floating-point intensive benchmarks.
Higher scores = shorter times.
One copy of one program is run at a time.
[Suites and Benchmarks] [SPECspeed and SPECrate]
SPEC CPU2017 Integer Rate Result Report for measurements that use a suite of 10 integer intensive benchmarks.
Higher scores = more throughput.
The tester chooses how many copies to run.
[Suites and Benchmarks] [SPECspeed and SPECrate]
SPEC CPU2017 Integer Speed Result Report for measurements that use a suite of 10 integer intensive benchmarks.
Higher scores = shorter times.
One copy of one program is run at a time.
[Suites and Benchmarks] [SPECspeed and SPECrate]

Performance Metrics

Metric Depends on Overall ratio
for suite of
Compile
method
SPECspeed2017_int_base Time required,
running 1 task at a time.
Higher score=better performance.
10 integer
benchmarks
Less aggressive
SPECspeed2017_int_peak More aggressive
SPECspeed2017_fp_base 10 floating point
benchmarks
Less aggressive
SPECspeed2017_fp_peak More aggressive
SPECrate2017_int_base Throughput: work per unit of time;
tester picks how much work is attempted.
Higher score=better performance.
10 integer
benchmarks
Less aggressive
SPECrate2017_int_peak More aggressive
SPECrate2017_fp_base 13 floating point
benchmarks
Less aggressive
SPECrate2017_fp_peak More aggressive
  SPECspeed and SPECrate Suites and Benchmarks Base and Peak

Energy Statistics

exp_SPECspeed2017_int_energy_base
^^^Why?
Overall energy ratio running 1 integer program at a time, base tuning.
Higher scores = more computing per unit of energy.
exp_SPECspeed2017_int_energy_peak
^^^Why?
Overall energy ratio running 1 integer program at a time, peak tuning.
Higher scores = more computing per unit of energy.
exp_SPECspeed2017_fp_energy_base
^^^Why?
Overall energy ratio running 1 floating point program at a time, base tuning.
Higher scores = more computing per unit of energy.
exp_SPECspeed2017_fp_energy_peak
^^^Why?
Overall energy ratio running 1 floating point program at a time, peak tuning.
Higher scores = more computing per unit of energy.
exp_SPECrate2017_int_energy_base
^^^Why?
Overall energy ratio running N integer programs (tester chooses N), base tuning.
Higher scores = more computing per unit of energy.
exp_SPECrate2017_int_energy_peak
^^^Why?
Overall energy ratio running N integer programs (tester chooses N), peak tuning.
Higher scores = more computing per unit of energy.
exp_SPECrate2017_fp_energy_base
^^^Why?
Overall energy ratio running N floating point programs (tester chooses N), base tuning.
Higher scores = more computing per unit of energy.
exp_SPECrate2017_fp_energy_peak
^^^Why?
Overall energy ratio running N floating point programs (tester chooses N), peak tuning.
Higher scores = more computing per unit of energy.
Why exp_? For the initial release of SPEC CPU2017, the energy statistics are considered experimental, not official metrics, non-comparable, and subject to change.

Tester and Date Info

CPU2017 license # The SPEC CPU license number of the organization or individual that ran the test.
Hardware Availability The date when all the hardware necessary to run the result is generally available. For example, if the CPU is available in Aug-2025 but the memory is not available until Jan-2026, then the hardware availability date is Jan-2026 (unless some other component pushes it out farther).
Software Availability The date when all the software necessary to run the result is generally available. For example, if the operating system is available in Aug-2025 but the compiler is not available until Jan-2026, then the software availability date is Jan-2026 (unless some other component pushes it out farther).
Test Date The date when the test is run. This value is obtained from the system under test.
Test Sponsor The name of the organization or individual that sponsored the test. Generally, this is the name of the license holder.
Tested by The name of the organization or individual that ran the test. If there are installations in multiple geographic locations, sometimes that will also be listed in this field.

Results Table

Result table In addition to the graph, the results of the individual benchmark runs are also presented in table form.
Benchmark The name of the benchmark.
Copies For SPECrate runs, this column indicates the number of benchmark copies that were run simultaneously.
Threads For SPECspeed runs, this column indicates the number of OpenMP threads that the benchmark was allowed to use simultaneously.
Seconds For SPECspeed runs, this is the amount of time in seconds that the benchmark took to run.
For SPECrate runs, it is the amount of time between the start of the first copy and the end of the last copy.
Ratio Ratio of benchmark run time on the reference platform divided by time on this system.
Thus higher == better. When comparing systems, the system with the higher ratio does more computing per unit of time.
Energy kJoules Amount of energy consumed (in kiloJoules) during the execution of the benchmark, computed as watts * seconds / 1000.
Maximum Power Maximum power consumed (in watts) during the execution of the benchmark.
Average Power Average power consumed (in watts) during the execution of the benchmark.
Energy Ratio Ratio of energy consumed on the reference platform divided by energy consumed on this system.
Thus higher == better. When comparing systems, the system with the higher Energy Ratio does more computing per unit of energy.

Tester-provided notes

Notes/Tuning Information Tester's free-form notes.
Compiler Invocation Notes Tester's notes about how the compilers were invoked (example: special paths, setup scripts, and so forth.)
Submit Notes Tester's notes about how the config file submit option was used to assign processes to processors.
Portability Notes Tester's notes about portability options and flags used to build the benchmarks.
Base Tuning Notes Tester's notes about base optimization options and flags used to build the benchmarks.
Peak Tuning Notes Tester's notes about peak optimization options and flags used to build the benchmarks.
Operating System Notes Tester's notes about changes to the default operating system state and other OS tuning.
Platform Notes Tester's notes about changes to the default hardware state and other non-OS tuning.
Component Notes Tester's notes about components needed to build a particular system (for User-Built systems).
General Notes Tester's notes about anything not covered in the other notes sections.
Compiler Version Notes This section is automatically generated.
It contains output from CC_VERSION_OPTION (and FC_VERSION_OPTION and CXX_VERSION_OPTION).

Flags

Compilation Flags Used This section is generated automatically. It lists the compiler flags that were used, and links to descriptions.
Benchmarks Using <language> The compiler flags are reported according to the languages used by the benchmarks.
For base, the rules require consistency by language.
For a list of which benchmarks use which languages, see the table of Benchmarks in the documentation index.
Compiler Invocation How the compilers are invoked.
Portability Flags Flags that are claimed to be necessary in order to solve platform differences, under the portability rule.
Generally required to be performance-neutral.
Optimization Flags Flags that improve (or are intended to improve) performance.
Other Flags Compile flags that are classified as neither portability nor optimization.
Unknown Flags Flags that are not described.
Results with unknown flags are marked "invalid" and must not be published.
If you have a result with this problem, you might be able to fix it, by editing your flags file and reloading it with rawformat.
Forbidden Flags This section of the reports lists compilation flags used that are designated as "forbidden".
Results using forbidden flags are marked "invalid" and must not be published.
Errors This section is automatically inserted when there are errors present that prevent the result from being a valid reportable result.

Hardware description

See the run rules section on Hardware Configuration disclosure.

CPU Name A manufacturer-determined processor formal name.
Maximum CPU MHz The maximum clock frequency of the CPU, expressed in megahertz.
Nominal CPU MHz The nominal (advertised) clock frequency of the CPU, expressed in megahertz.
CPU(s) enabled The number of CPUs that were enabled and active during the benchmark run. More information about CPU counting is in the run rules.
CPU(s) orderable The number of CPUs that can be ordered in a system of the type being tested.
L1 Cache Description (size and organization) of the CPU's primary cache. This cache is also referred to as "L1 cache".
L2 Cache Description (size and organization) of the CPU's secondary cache. This cache is also referred to as "L2 cache".
L3 Cache Description (size and organization) of the CPU's tertiary, or "Level 3" cache.
Other Cache Description (size and organization) of any other levels of cache memory.
Memory Description of the system main memory configuration.
Options that affect performance, such as arrangement of memory modules, interleaving, latency, etc, are documented here.
Storage Subsystem A description of the disk subsystem (size, type, and RAID level if any) of the storage used to hold the benchmark tree during the run.
Other Hardware Any additional equipment added to improve performance.

Software description

See the run rules section on Software Configuration disclosure.

Operating System The operating system name and version. If there are patches applied that affect performance, they must be disclosed in the notes.
Compiler The names and versions of all compilers, preprocessors, and performance libraries used to generate the result.
Parallel This field is automatically set to "Yes" if compiler flags are used that are marked with the parallel attribute, indicating that they cause either automatic or explicit parallelism.
System Firmware The customer-accessible name and version of the firmware used on the system under test.
File System The type of the filesystem used to contain the run directories.
System State The state (sometimes called "run level") of the system while the benchmarks were being run. Generally, this is "single user", "multi-user", "default", etc.
Base Pointers Indicates whether all the benchmarks in base used 32-bit pointers, 64-bit pointers, or a mixture.
For example, if the C and C++ benchmarks used 32-bit pointers, and the Fortran benchmarks used 64-bit pointers, then "32/64-bit" would be reported here.
Peak Pointers Indicates whether all the benchmarks in peak used 32-bit pointers, 64-bit pointers, or a mixture.
Other Software Any performance-relevant non-compiler software used, including third-party libraries, accelerators, etc.

Power and Temperature information

Power Supply The number and rating of the power supplies used in this system for this run.
Power Supply Details Additional details about the power supply, such as a part number or other identifier.
Maximum Power (W) Maximum power (in Watts) that was measured during the entire benchmark suite run.
Idle Power (W) This is a 60 second measurement of idle power (in Watts) on the machine, is made after the benchmark has been run and the system was given time 10 seconds to rest.
Minimum Temperature (C) Lowest temperature measured (in C) that was registered during the entire benchmark suite run.
Power Analyzer Name used to connect the PTDaemon to the power analyzer. If more that one power analyzer was used, there will be multiple descriptions presented.
Hardware Vendor Name of the company that provides the power analyzer or temperature meter.
Model The model of the power analyzer or temperature meter.
Serial Number Serial number of the power analyzer being used.
Input Connection A description of the interface used to connect the power analyzer or temperature meter to the PTDaemon host system, e.g. RS-232 (serial port), USC, GPIB, etc.
Metrology Institute Name of the accreditation organization of the institute that did the calibration of the meter (e.g. NIST, PTB, AIST, NML, CNAS, etc.).
A list of national metrology institutes for many countries is maintained by the United States National Institute of Standards. If the main site is unavailable, the content may be viewable on the Internet Archive's Wayback Machine.
Calibration By Organization that performed the power analyzer calibration.
Calibration Label A number or character string which uniquely identifies this meter calibration event.
May appear on the calibration certificate or on a sticker applied to the power analyzer. The format of this number is specified by the metrology institute.
Calibration Date The date (DD-MMM-YYYY) the calibration certificate was issued, from the calibration label or the calibration certificate.
PTDaemon Version Version of the Power and Temperature Daemon (automatically filled out).
Setup Description A brief description of how the power analyzer or temperature meter was used.
May include which power supply was connected to this power analyzer, or how far away this temperature meter was from the air intake of the system.
Current Ranges Used A list of current (amperage) ranges used to configure the power analyzer while running the benchmarks.
Voltage Range Used Voltage range used to configure the power analyzer while running the benchmarks.
Temperature Meter The name used to connect the PTDaemon to the temperature meter. If more that one temperature meter was used, there will be multiple descriptions presented.

Other information

Median results For a reportable CPU2017 run, two or three iterations of each benchmark are run, and either the median of the three runs, or the slower of the two, is selected to be part of the overall metric. In output formats that support it, the selected results are underlined in bold.
Run order When you read a results table, results are listed in the order that they were run, in column-major order. In other words, if you're interested in reading results in the same order that they were produced, start in the upper-left corner and read down the first column, then read the middle column, and so forth. If both base and peak tuning are used, all base runs are completed before starting peak.
[details]

SPEC CPU®2017 Result File Fields: Copyright © 2017 Standard Performance Evaluation Corporation (SPEC)