Standard Performance Evaluation Corporation
SPECvirt_sc2013 is SPEC's updated benchmark addressing performance evaluation of datacenter servers used in virtualized server consolidation. SPECvirt_sc2013 measures the end-to-end performance of all system components including the hardware, virtualization platform, and the virtualized guest operating system and application software. The benchmark supports hardware virtualization, operating system virtualization, and hardware.
The benchmark utilizes several SPEC workloads representing applications that are common targets of virtualization and server consolidation. We modified each of these standard workloads to match a typical server consolidation scenario of CPU resource requirements, memory, disk I/O, and network utilization for each workload. These workloads are modified versions of SPECweb2005, SPECjAppServer2004, SPECmail2008, and SPEC CPU2006. The client-side SPECvirt_sc2013 harness controls the workloads. Scaling is achieved by running additional sets of virtual machines, called "tiles", until overall throughput reaches a peak. All VMs must continue to meet required quality of service (QoS) criteria.
The benchmark also includes options for measuring power consumption and power/performance relationships. Based on SPECpower methodology, power measurement requires the use one of the accepted power analyzers. The benchmarker has the option of running with power monitoring enabled and can submit results to any of three categories:
The benchmark is targeted for use by hardware vendors, virtualization software vendors, application software vendors, datacenter managers, and academic researchers.
Changes from SPECvirt_sc2010
SPECvirt_sc2013 shares the general benchmark architecture from SPECvirt_sc2010, including the benchmark harness and most of the application workloads and VM types. However, since SPECvirt_sc2010 was developed, many companies are virtualizing significantly heavier workloads than those in the first generation benchmark. In SPECvirt_sc2013, we are providing heavier and more bursty workloads that require virtual machines with more memory and vCPUs and further stress the system's ability to meet the benchmark's quality of service requirements. The list below summarizes some of the key changes made for SPECvirt_sc2013:
SPECvirt_sc2013 Benchmark Documentation