Skip navigation

Standard Performance Evaluation Corporation

 
 

SPECvirt®sc2013

The SPEC virt® benchmark suites are used to measure performance of virtualized platforms. These benchmark suites are targeted for use by hardware vendors, virtualization software vendors, application software vendors, datacenter managers, and academic researchers.

The SPEC virt® sc2013 benchmark addresses performance evaluation of datacenter servers used in virtualized server consolidation (sc). 

The SPEC virt® sc2013 benchmark is available for purchase via the SPEC Order Form.

SPEC virt® sc2013 benchmark measures the end-to-end performance of all system components including the hardware, virtualization platform, and the virtualized guest operating system and application software.   The benchmark supports hardware virtualization, operating system virtualization, and hardware.

The SPEC virt® sc2013 benchmark utilizes several SPEC workloads representing applications that are common targets of virtualization and server consolidation. We modified each of these standard workloads to match a typical server consolidation scenario of CPU resource requirements, memory, disk I/O, and network utilization for each workload. These workloads are modified versions of SPECweb®2005, SPECjAppServer®2004, SPECmail®2008, and SPEC INT® 2006 benchmarks. The client-side SPEC virt® sc2013 harness controls the workloads. Scaling is achieved by running additional sets of virtual machines, called "tiles", until overall throughput reaches a peak. All VMs must continue to meet required quality of service (QoS) criteria.

The benchmark also includes options for measuring power consumption and power/performance relationships. Based on SPEC Power® methodology, power measurement requires the use one of the accepted power analyzers. The benchmarker has the option of running with power monitoring enabled and can submit results to any of three categories:

  • Performance only (SPEC virt® sc2013)
  • Performance/power for the System Under Test including storage (SPEC virt® sc2013_PPW)
  • Performance/power for the Server-only (SPEC virt® sc2013_ServerPPW)

Changes from SPECvirt_sc®2010:

SPEC virt® sc2013 benchmark shares the general benchmark architecture from SPECvirt_sc®2010 benchmark, including the benchmark harness and most of the application workloads and VM types. However, since the SPECvirt_sc®2010 benchmark was developed, many companies are virtualizing significantly heavier workloads than those in the first generation benchmark. In the SPEC virt® sc2013 benchmark, we are providing heavier and more bursty workloads that require virtual machines with more memory and vCPUs and further stress the system's ability to meet the benchmark's quality of service requirements. The list below summarizes some of the key changes:

  • The J2EE Application workload based on SPECjAppServer®2004 has been modified to further increase the burstiness of the workload so that the injection rate averages 100 IR (was 20 IR).
  • The Database Server VM is now shared by up to four Appserver VMs (fewer if number of tiles is not a multiple of four) to ensure that there is a VM requiring multiple vCPUs (SMP).
  • The Web Server workload based on the Support workload in SPECweb®2005 has been modified to require SSL (HTTPS) between the clients and the web server, and the load level has been increased 2500 Simultaneous Sessions (was 500).
  • The Mail Server workload based on SPECmail®2008 has been modified to present a bursty workload by increasing the variability of the think times in the benchmark.
  • The Idle Server in the prior version has been replaced by the Batch Server which periodically runs a set of jobs that utilize the 401.bzip2 benchmark and harness from SPEC INT® 2006. This allows the jobs to be parallelized based on the number of vCPUs in the batch server VM. Like the idle server, this workload is not scored but must meet QoS requirements for completing each set of batch jobs within set limits as well as being responsive to periodic client polling.

Results

Submitted Results
Includes all of the results submitted to SPEC from the SPEC member companies and other licensees of the benchmark.

Search SPEC virt® sc2013 results in SPEC's online result database.

Information

Press Releases
Press release material, documents, and announcements:

SPEC virt®sc2013 Benchmark Documentation

SPEC virt®sc2013 Benchmark Technical Support