What SPECviewperf® Does and Doesn't Do
Nearly all benchmarks are designed for a specific purpose. Quite
often, however, users broaden that purpose beyond what the benchmark
is designed to do, or in some cases they assume the benchmark
can't do something that it actually can do. The SPECviewperf®
benchmark is no different: it has been both overextended and underappreciated,
sometimes reducing its overall value to the user. Here is a closer
look at SPECviewperf: what it is and what it can and can't do.
SPECviewperf measures the 3D rendering performance of systems
running under OpenGL. The SPECopcSM project group has
worked with independent software vendors (ISVs) to obtain tests,
data sets and weights that constitute what is called a viewset.
Each viewset represents the graphics rendering portion of an actual
application. The ISVs that develop SPECopc viewsets have provided
percentage weights for each test for which a performance number
is reported. ISVs have defined these percentages to indicate the
relative importance of a test within the overall application.
SPECviewperf offers the following characteristics:
- It provides a single-source code for apples-to-apples comparison
and performance tuning across different hardware platforms.
- It runs on multiple operating systems.
- It runs across different processors.
- It runs on multiple windowing environments.
- It encompasses a wide variety of OpenGL features and rendering
- It is easily accessible through the SPECopc project subcommittee,
ftp and through OpenGL sample disk distribution.
Several factors make SPECviewperf unique from other benchmarks:
- It uses datasets that are designed for and used by real applications.
- It uses rendering parameters and models selected by independent
software vendors (ISVs) and graphics users.
- It produces numbers based on frames per second, a measurement
with which users can readily identify.
- It provides one number for each rendering path using one
What SPECviewperf Measures
SPECviewperf measures performance for the following entities:
- 3D primitives, including points, lines, line_strip, line_loop,
triangles, triangle_strip, triangle_fan, quads and polygons;
- attributes per vertex, per primitive and per frame;
- texture mapping;
- alpha blending;
- anti-aliasing; and
- depth buffering.
The Five-Step Program
SPECviewperf is not a single-number benchmark. In order to use
it to its fullest advantage, ISVs and users need to relate the
benchmark to their actual applications. Here are the five steps
recommended for using SPECviewperf effectively:
- Identify software code paths that are important to the application.
- Identify the primitives used within the application.
- Select datasets that are most appropriate to the application.
The datasets should reflect the level of geometry and rasterization
found in the application.
- Identify attributes and the level at which they are applied
(per vertex, per primitive or per frame).
- Assign a weight to each path based on the percentage of time
in each path and the importance of the path to the application.
What SPECviewperf Can't Do
Although SPECviewperf is a good tool for measuring OpenGL performance
as it relates to applications, like all benchmarks it has limitations.
Most important of these is that it cannot be used to compare performance
across different application programming interfaces (APIs). Also,
it does not run itself; users must participate in the benchmarking
process. When testing and reporting results, SPECviewperf does
not account for the following key factors:
- effects caused by switching primitives,
- input effects on the event loop,
- user interface rendering and management,
- complex motion of multiple models,
- effects of CPU load on the graphics subsystem,
- color index visual performance, and
- multi-context, multi-window effects.
Development of SPECviewperf within the SPECopc group is an ongoing
process, with future enhancements designed to address key graphics
applications issues not covered by the current benchmark.