SPEC CPU®2017 Changes in v1.1.5

$Id: changes-in-v1.1.5.html 6399 2019-12-11 17:04:44Z JohnHenning $ Latest: www.spec.org/cpu2017/Docs/

This document briefly describes changes in SPEC CPU®2017 v1.1.5, a product of the SPEC® non-profit corporation (about SPEC).

All users are strongly encouraged to update to v1.1.5,
to improve compatibility, reliability, accuracy, and clarity (as described below).
Updates are easy to do.

Contents:

Changes to the sysinfo utility

Updates for GCC

Updates to Run and Reporting Rules

Benchmark updates

500.perlbench_r / 600.perlbench_s

510.parest_r

Other minor updates

Changes to the sysinfo utility

The sysinfo utility has several fixes and enhancements. The changes include:

In addition, sysinfo output is now captured in the primary log file, which is useful if a run fails without producing any other reports.

All users of CPU 2017 are encouraged to update their copy of SPEC CPU 2017 to obtain these sysinfo changes, which may improve both accuracy and clarity of reports.

Updates for GCC

GCC version 10 requires workarounds for several benchmarks. These workarounds do not qualify as PORTABILITY (runrules.html#portability) and therefore must be applied consistently in base (runrules.html#BaseFlags). The example GCC config files have been updated to demonstrate how to apply the workarounds while complying with the rules.

Benchmarks with updated GCC notes include:

SPECrateSPECspeed

All who use GCC with SPEC CPU 2017 are encouraged to update their copy of SPEC CPU to obtain the new example config files, which are more compatible with contemporary GCC.

Updates to Run and Reporting Rules

Rule 2.2.4 regarding size changes was clarified, adding references to related rules.

Benchmark updates

500.perlbench_r / 600.perlbench_s

Removed op/time_loop from the (non-timed) test workload, which was discovered to be non-portable when a compiler user reported that undefined behavior on a conversion caused the test to always fail, independent of optimization level. The test was for a case that does not arise in the (timed) reference workload.

510.parest_r

Added portability option -DSPEC_PAREST_STD_FLUSH_WORKAROUND which works around an issue in module message_log.cc, which takes the address of std::flush and uses it in a comparison. If a compiler does not generate consistent results for the address, then the comparison fails and 510.parest_r does not generate any output.

It is also conceivable that a C++2003 compiler might forbid taking the address of std::flush, because it was not explicitly deemed acceptable to do so prior to C++20.

In either of those cases, this flag may be used. It substitutes an alternate flush function in the MessageLog namespace thereby allowing the comparison to succeed.

Presubmission tools

Results that are published at SPEC are reviewed by a series of perl scripts prior to publication. A subset of those tools is included with SPEC CPU 2017 v1.1.5 in directory $SPEC/bin/presubmit (Unix) or %SPEC%\bin\presubmit (Windows). If you use the presubmit tools, your review of results may be quicker or smoother. Please see the README.txt file in that directory.

Other minor updates

Your copy of SPEC CPU 2017 will be more reliable with the above fixes. You should update.

SPEC CPU®2017 Changes in v1.1.5: Copyright © 2020 Standard Performance Evaluation Corporation (SPEC®)