Skip navigation

Standard Performance Evaluation Corporation

Facebook logo LinkedIn logo Twitter logo Google+ logo

The Standard Performance Evaluation Corporation (SPEC) is a non-profit corporation formed to establish, maintain and endorse standardized benchmarks and tools to evaluate performance and energy efficiency for the newest generation of computing systems. SPEC develops benchmark suites and also reviews and publishes submitted results from our member organizations and other benchmark licensees.

What's New:

05/30/2018: SPEC's Research Group has issued a call for nominations for the Kaivalya Dixit Distinguished Dissertation Award recognizing outstanding doctoral dissertations in the field of computer benchmarking, performance evaluation, and experimental system analysis. The award is open to dissertations that have been defended between October 2017 and September 2018.

05/23/2018: The SPEC Graphics Performance Characterization Group (SPECgpc) has released an all-new version of its SPECviewperf graphics performance benchmark. SPECviewperf 13 features new volume visualization viewsets for energy and medical applications, a redesigned GUI, improved scoring and reporting methods, and updated viewsets to support more recent versions of the applications on which they are based.

03/01/2018: As of March 1, 2018, SPEC offers HPG benchmarks (ACCEL, MPI2007, OMP2012) free of charge to qualified non-profit organizations worldwide. Qualified organizations can request a license and software download.

02/21/2018: The Kaivalya Dixit Distinguished Dissertation Award selection committee has chosen this year to select a winner and a runner up based on the high quality of both submissions. Aleksandar Milenkoski of Julius-Maximilians-Universität (JMU) Würzburg as the winner of the 2017 SPEC Kaivalya Distinguished Dissertation Award and the committee decided to publicly recognize Benjamin Heintz, University of Minnesota Twin Cities. The awards will be presented at the ICPE 2018 conference in Berlin, Germany.

01/07/2018: SPEC relies on contributors outside of its membership to help develop far-reaching benchmark suites such as SPEC CPU2017. Here's the story of one of those contributors.

12/20/2017: The SPEC Storage committee has released an update to the storage benchmark, SPEC SFS 2014 SP2. This update includes a new workload for the electronic design automation (EDA) market along with other enhancements to the benchmark suite.

12/20/2017: SPEC's Solution File Server committee has changed their name to the Storage committee, chartered to create, distribute, and maintain a standardized benchmark for measuring the performance of storage solutions.

12/19/2017: The SPEC Power committee has released an update to the SPEC SERT tool to V2.0.1, which corrects an issue with the calculation of the final metric. A patch to update from 2.0.0 to 2.0.1 is available for existing licensees.

12/13/2017: SPECapc released an updated version of the Maya 2017 benchmark that improves the way multi-sample anti-aliasing (MSAA) is measured. Results from V1.0.6 of SPECapc for Maya 2017 are not comparable to those from previous versions of the benchmark. Free downloads of the benchmark are available to everyone but sellers of computers and related products.

10/17/2017: The research and industrial/experience paper submission deadline for the ICPE 2018 conference to be held April 9-13, 2018 in Berlin, has been extended to October 25, 2017. Please see the important dates for abstract and paper submission and notification schedules.

10/12/2017: The SPECjbb2015 benchmark has been updated to version 1.01. This update adds support for Java SE 9. Licensees of version 1.0 are entitled to a complimentary V1.01 upgrade. SPEC will be retiring SPECjbb2015 V1.0 in January 2018, after which time all result submissions must be made with V1.01.

09/27/2017: SPEC's Application Performance Characterization (SPECapc) project group has released all-new performance evaluation software for systems running Autodesk Maya 2017 3D animation software. SPECapc for Maya 2017 software includes nearly 80 individual tests that reflect the processes used to model, animate and render scenes within the application. The new benchmark was developed by SPECapc in cooperation with Autodesk, which supplied seven new graphics models and scenes.

08/14/2017: With the release of SPEC CPU2017, SPEC will be retiring SPEC CPU2006 in January 2018. Full details on the retirement schedule can be found here.

07/11/2017: Michael Hines, a senior engineer at DigitalOcean, is not your typical user of a SPEC CPU benchmark suite. He's not a high-end server user, a hardware system vendor, a researcher or an independent software vendor (ISV). Then why does he find the SPEC CPU2017 benchmark suite critically important to his work? Find out here.

06/27/2017: SPEC's High-Performance Group (SPEC/HPG) has released a new version of its SPEC ACCEL software that adds a suite of OpenMP applications for measuring the performance of systems using hardware accelerator devices and supporting software. Other major updates in SPEC ACCEL 1.2 include the ability to add and change directives within the OpenMP and OpenACC suites to expose more parallelism for peak performance testing, and revised applications in the OpenACC suite to allow for successful GNU compiling.

06/20/2017: SPEC announces the release of SPEC CPU2017, an all-new version of the benchmark suite for evaluating compute-intensive performance across a wide range of hardware systems. CPU2017 contains 43 individual benchmarks organized into four suites. Among the new features are the option to use OpenMP for several of the SPECspeed 2017 benchmarks and the ability to optionally measure and report power statistics. The suite is available now, with licensees of earlier versions of the CPU suites eligible for a discounted upgrade price.

06/20/2017: Due to a known issue in Microsoft Windows Server 2016 memory management, SERT will not be supported on that operating system until a fix is made available. 

06/20/2017: SPEC cannot provide support on SPECpower_ssj2008 for test failures using Microsoft Windows Server 2016 due to an issue with large pages and memory management in the operating system.  Please contact Microsoft technical support for assistance.  Completed runs on Windows Server 2016 will be processed for review as usual.

06/08/2017: The 9th ACM/SPEC International Conference on Performance Engineering (ICPE 2018) has issued a call for contributions. Topics of interest include: performance modeling of software; performance and software development processes/paradigms; performance measurement, monitoring and analysis; benchmarking; run-time performance management; power and performance, energy efficiency; performance modeling and evaluation in different environments and application domains; and other topics related to performance of software and systems. ICPE2018 will take place April 9-13, 2018 in Berlin, Germany.

06/05/2017: SPECpower has released a new version of PTDaemon, the software that allows power measurement devices to be incorporated into performance evaluation software. PTDaemon 1.8.1 includes new support for Windows USBTMC using Keysight drivers and the Hioki PW3335 (single channel) power measurement device. Users of SPECpower_ssj2008 and the SERT suite are encouraged to download and apply the update.

05/23/2017: SPEC's Research Group has issued a call for nominations for the Kaivalya Dixit Distinguished Dissertation Award recognizing outstanding doctoral dissertations in the field of computer benchmarking, performance evaluation, and experimental system analysis. The award is open to dissertations that have been defended between October 2016 and September 2017.

05/22/2017: In an exclusive interview, Klaus-Dieter Lange, SPECpower chair, discusses the origin of SERT, the challenges of creating an international standard, the prospect of continued improvements in server efficiency, and his own personal growth in leading a diverse group of technologists.

05/22/2017: SPEC's High-Performance Group (SPEC/HPG) is offering rewards of up to $5,000 and a free benchmark license for application code and datasets accepted under its new SPEC MPI Accelerator Benchmark Search Program. Applications that make it through the entire search program will be incorporated into a new benchmark currently called SPEC MPI ACCEL. The new benchmark will combine components of the current SPEC MPI and SPEC ACCEL benchmark suites.

04/18/2017: Version 2.0 of the Chauffeur Workload Development Kit (WDK) has been released. This major release is based on the SERT 2.0 infrastructure and includes significant enhancements to the hardware detection, customization options of generating HTML reports, and developer documentation. It has now reduced memory requirements for the Director when signing results files and reduced the size of the result output for large systems or clusters. There is also an updated list of supported operating systems including Ubuntu (14.04 LTS and 16.04 LTS) as well as current versions of Windows Server, RHEL, SLES, AIX, and Solaris.

04/04/2017: A patch has been released for SPECvirt_sc2013 V1.1 which addresses an issue encountered when running the webserver workload with TLS encryptions which would cause any result to be marked as invalid. Any result submissions using TLSv1, TLSv1.1, or TLSv1.2 encryptions for the webserver workload must use this patch.

04/04/2017: Scott Beamer III from the University of California, Berkeley is the winner of the 2016 SPEC Kaivalya Dixit Distinguished Dissertation Award with his dissertation on the characterization and treatment of fundamental performance bottlenecks in graph processing and the contribution of the Graph Algorithm Platform (GAP) benchmark suite.

03/28/2017: SPEC announces the release of the Server Efficiency Rating Tool (SERT) 2.0, which adds a single-value metric, reduces runtime, improves automation and testing, and broadens device and platform support. Designed to be simple to configure and use via a comprehensive graphical user interface, the SERT suite uses a set of synthetic worklets to test discrete system components such as processors, memory and storage, providing detailed power consumption data at different load levels.

03/22/2017: Walter Bays steps down as SPEC president after presiding over 12 years of growth; he is succeeded by David Reiner, a SPEC 12-year veteran.


[Older items can be found in the archive.]