Standard Performance Evaluation Corporation
SPEC Frequently Asked Questions
SPEC, the Standard Performance Evaluation Corporation, is a non-profit corporation formed to "establish, maintain and endorse a standardized set of relevant benchmarks that can be applied to the newest generation of high-performance computers" (quoted from SPEC's bylaws). The founders of this organization believe that the user community will benefit greatly from an objective series of applications-oriented tests, which can serve as common reference points and be considered during the evaluation process. While no one benchmark can fully characterize overall system performance, the results of a variety of realistic benchmarks can give valuable insight into expected real performance.
Legally, SPEC is a non-profit corporation registered in California.
SPEC basically performs two functions:
For more in depth inquiry, we have more information online about SPEC, including its background, the philosophy, and organizational structure.
For SPEC contact information, see the SPEC Contact Info Page.
SPEC is comprised of four groups:
Membership in SPEC is open to any interested company or entity that is willing to commit to SPEC's standards and is able to provide for the support of the organization.
Membership is group-based. One can be a member of one or more groups depending upon one's interests. Each group (GWPG, HPG, OSG, RG) is run independently; each member is responsible for paying the full dues and other responsibilities for each group they join -- there are no "bulk discounts".
SPEC does make an effort to accommodate the participation of academic institutions and other non-profit organizations with reduced (or waived) dues and other arrangements. SPEC has "associates", which pay little or no dues, but still get full access to the benchmark development process.
The dues go towards offsetting SPEC's operating costs. Typically these dues are a few thousand dollars per year with a similar initiation fee to cover starting costs; however, the exact values vary from group to group (and potentially year to year) depending upon the specific costs.
Membership allows for full access to existing benchmarks (this included company-wide licenses for current benchmark products) and full participation in ongoing development of new benchmarks. Additionally, members get unlimited result publication through SPEC (as membership dues already cover much of the cost). The benefits for associates are the same as for members except for having voting rights. The intention for associates is that they can act in an advisory capacity to SPEC, getting first-hand experience in an area that is widely neglected in academia but nonetheless very important in the "real world", and providing technical input to SPEC's tasks.
All members and associates of each group are welcome at all meetings for that group. Meetings are usually held four to six times a year, with subcommittee teleconferences held on an as-needed basis (often weekly or bi-weekly during benchmark development). Final decisions are usually made by a "steering committee" elected by the membership of the group, with benchmark standards needing ratification by the groups full membership.
For further information about membership in any of SPEC's groups, contact SPEC at firstname.lastname@example.org.
SPEC's main product is the benchmark suites. The code in the suites are developed by SPEC from code donated by various sources. SPEC works on portability and creates tools and meaningful workloads for the codes chosen as benchmarks. Therefore, the SPEC benchmarks are not the same as public domain programs that might exist under a similar name or the same name, and their execution times will, in general, be different.
The SPEC benchmark sources are generally available; though, they are licensed software products. Each SPEC group is charging separately for its benchmark suites. The income from the benchmark source sales is intended to support the administrative costs of the corporation. Buyers of the benchmarks receive a shrink-wrapped site license with their first order of any of SPEC's products. SPEC uses CD-ROMs as the distribution media for its benchmarks.
SPEC, the Standard Performance Evaluation Corporation, produces performance evaluation software [benchmarks] for modern computer systems. The various types software used in the SPEC benchmarks do not require nor use dates in any computations of a business critical or performance sensitive manner.
SPEC does not anticipate any issues with its products due to Year 2000 date effects. Should specific problems be found, SPEC will address them on a case by case basis.
The software products that SPEC licenses are not warranted to be Y2K compliant. SPEC makes no warrantees, express or implied, related to the sale, use or performance of its products, which are supplied on an "as is" basis. SPEC has no plans to implement any Y2K compliance programs for any of its products.
SPEC will often retire a benchmark when there is a more recent replacement or sometimes when the original benchmark has become too out-of-date to continue to support. "Retired" means that SPEC no longer provides technical support or accepts result submissions for posting on the SPEC web site. Recognizing that there may be historical interest in these older benchmarks, in many cases it may still be possible to obtain copies of such a benchmark by contacting the SPEC office, but the buyer should be aware that additional support will not be provided. Notice of planned retirements will be posted on the SPEC web site, and such news is usually reported months in advance of the retirement date.
We do not offer trial versions of the SPEC benchmarks. If you purchase software and are dissatisfied for any reason, you may return the software and license for a full refund of the software purchase price within thirty days. Contact the SPEC office for an RMA number.
SPEC benchmark suites are provided with a site license which allows the licensee to install and run the software on as many systems within their organization as they wish. You do not need to purchase multiple licenses. A sample license agreement (PDF) is available for review prior to purchase.
Educational discounts are available only when the software is purchased by and licensed to an educational institution; the end user must be a faculty member or a full-time employee of the institution, not a student.
Non-profit pricing is available to recognized tax-exempt organizations and government agencies. Individuals do not qualify for the discounted pricing.
Unfortunately, SPEC does not make recommendations on computer purchases. SPEC will provide what data it can (all available on the Web site http://www.spec.org/) but can not offer opinions on it.
Without knowing a lot about your exact workload, it would be difficult to provide any useful specific advice; and providing general advice can get us all in legal trouble.
SPEC 1170 does not have anything to do with SPEC, the Standard Performance Evaluation Corporation. It's a different spec.
SPEC 1170 is an older name for what is now called UNIX 98, which is an X/Open brand for compliance with a "Single UNIX Specification" -- basically a standardized UNIX. More information about UNIX 98 can be obtained from: http://www.unix-systems.org.
Most questions can be answered by browsing the appropriate areas: