Standard Performance Evaluation Corporation (SPEC)
SPECpower_ssj2008 Submission Checklist
7001 Heritage Village Plaza, Suite 225 Gainesville, VA 20155, USA

SPECpower Committee

Table of Contents

1.	Before you run the benchmark:	3

2.	Before you submit results for acceptance:	3

3.	At the time of submission:	4

4.	Before publication (during the review period):	5

5.	After publication:	5

SVN Revision:	722

SVN Date:	2013/04/11 11:07:17

The SPECpower committee welcomes your inquiries and disclosure submissions. You may find this checklist helpful as you prepare your system to run the benchmark, and the benchmark test result for submission to the SPEC acceptance process. Please address any questions or requests for assistance to info@spec.org.

Before you run the benchmark:

Read, understand and abide by the current Run and Reporting Rules. Some issues that have been seen in submissions include:

Power analyzers should not be placed in voltage or current auto-ranging modes.

Insure the power analyzer calibration date and standard meet benchmark requirements.

The SUT should meet benchmark and SPEC general availability requirements.

Review previously accepted results.

Understanding how accepted results are documented will provide insight into the specific requirements of acceptance.

While reviewing accepted results give precedence to more recently accepted disclosure reports over older disclosures, but be aware that accepted disclosures may themselves contain errors.

Review the online Disclosure Glossary for documentation requirements.

The Disclosure Glossary specifies the meaning of each disclosure field.

Review the list of accepted measurement devices.

The latest information and use restrictions for all SPEC-accepted measurement devices are maintained here. Not all modes of every measurement device are accepted for compliant benchmark disclosures.

Verify that you are using devices and device modes accepted by SPEC for benchmark disclosure purposes.

Review third-party end user license agreements.

If you used third-party hardware and software, confirm your end-user license agreement grants you the right to publish performance and power data using these items, or obtain the prior consent of the third-party for disclosure.

The submitter is responsible for the entire content of the results disclosure including but not limited to accuracy and license compliance.

If necessary prepare a publication fee payment to SPEC.

Submitters who are not members of SPEC are required to pay a fee before publication of their disclosure. Results will not be published until the payment is processed.

Before you submit results for acceptance:

Read and understand the benchmark submission process and submission calendar.

A vendor site may not publicly disclose a benchmark result that has not been accepted by SPEC unless it has previously had at least one result accepted by SPEC. Plan your submission so that the review schedule and acceptance of the result meets your needs for public disclosure.

The review and acceptance process requires a minimum of two weeks.

If you are required to pay a fee for publication, the required payment must be received by the SPEC office before your result will be published.

Do not plan on submitting your result at the last minute, especially if this if your first submission. At the submission deadline (12:01am PT) the SPEC office is closed for the evening and SPEC staff are not available to help if your submission is rejected by the automatic tool.

If you intend to submit more than ten benchmark results in a single submission cycle you are required to provide two weeks' notice to the benchmark subcommittee that will be reviewing your results to ensure that sufficient resources are available to perform the reviews.

Review your result disclosure.

Verify that it complies with all Run and Reporting Rules.

If the result disclosure is marked invalid by the benchmark it will be rejected by the automatic submission tool. Correct the problem and re-run your test before submitting.

The run must be fully documented and technically accurate to the standard that an experienced user would be able to reproduce the result.

Check the spelling of all items. Publication of a disclosure on the SPEC website reflects on both the submitter and on SPEC.

Documentation corrections may be made to the benchmark result raw file but are limited to the portion of the file above the line marked "The remaining properties may not be edited." If any corrections are needed below this line then the props file(s) must be modified and the benchmark run must be repeated.

Retain all required documentation:

All benchmark-generated files for the submitted run should be retained at least until the submission review is concluded. Any file generated by a benchmark component may be requested during the results review process.

The Run and Reporting Rules require that a list of active OS services on the SUT be generated during the benchmark run and retained at least until the submission review is concluded.

[Optional/recommended] Maintain the availability of the SUT and test fixture during the review period.

Maintaining the availability of the SUT may assist you to defend your submission if technical objections are raised against it.

Some submitters have been asked to re-run their tests due to rules violations. In such cases expedited reviews have occasionally been granted.

At the time of submission:

Follow the documented benchmark submission process.

Verify that you receive a valid email receipt from the automated results acceptance process.

If the email receipt indicates non-acceptance of your submission into the review process, you may be able to correct the problem following the instructions in the email. Address requests for assistance with the automated submission process to info@spec.org.

SPEC OSG members may submit collateral documentation (e.g. links to product websites, on-line documentation) directly to the committee email alias at the time of submission. Other benchmark licensees may be requested to submit collateral material during the review period.

Before publication (during the review period):

Monitor your email for communications from committee members reviewing your result.

The automated results submission acceptor records and marks your submission with the email account from which you submitted it. This is the account you should monitor.

You may be requested to answer questions or provide documentation with regard to the SUT or how the result was produced. Any SPEC OSG member may review a result and request information.

[Optional/recommended] SPEC OSG members may monitor the power committee email alias for rules or process issues that your submission brings to light, but which may be resolved within the committee without the input of the submitter.

[Optional/recommended] Be available to attend the committee meetings during the period when your disclosure is under review.

Questions may be raised about a result up to and including the committee meeting at which the review concludes. Timely answers to any questions help insure publication of a disclosure at the earliest possible time.

After publication:

Comply with SPEC Fair Use Rules and the benchmark-specific Addendum to OSG Fair Use Policy when making public statements regarding benchmark results.

Benchmark licensees who violate Fair Use Rules may be penalized. Penalties vary with the nature of the violation.

[Optional/recommended] Retain all documentation required for the submission.

Results may be re-reviewed and found to be not in compliance with the Run and Reporting Rules (in force at the time of acceptance) even after acceptance by SPEC. Retaining this documentation may help you support your claim of compliance should your result come under re-review.

[Optional/recommended] Retain all props files and run scripts so that you may provide documentation of your run or in the worst case reproduce it if required.