SPECjbb

SPECjbb® 2015 benchmark

SPEC.org Submission Requirements	Vendors may publish compliant results independently, provided that run does not produce any warning or invalid messages and all run and reporting rules are followed. Any result which has warnings can only be used once accepted by the OSGjava subcommittee.
SPEC Metrics	SPECjbb®2015-<category> max-jOPS and SPECjbb®2015-<category> critical-jOPS where <category>: [Composite / MultiJVM / Distributed]
Required Metrics	Both metrics SPECjbb®2015-<category> max-jOPS and SPECjbb®2015-<category> critical-jOPS must be stated in proximity.
Conditionally Required Metrics
Use of Estimates	Not allowed
Disallowed Comparisons	In addition to the requirements that results not be compared to other benchmarks: Any comparison across categories SPECjbb®2015-Composite, SPECjbb®2015-MultiJVM and SPECjbb®2015-Distributed is prohibited

RETIRED	The SPECjbb 2013 benchmark was retired on December 9, 2014. All public use of results for this benchmark must plainly disclose that the benchmark has been retired, as described above. No further submissions will be accepted for publication at www.spec.org. SPEC is no longer reviewing results for this benchmark. Independent publication of new results is not allowed (because a defect impacts the comparability of results).
SPEC.org Submission Requirements	Vendors may publish compliant results independently, provided that run does not produce any warning or invalid messages and all run and reporting rules are followed. Any result which has warnings can only be used once accepted by the OSGjava subcommittee.
SPEC Metrics	SPECjbb®2013-<category> max-jOPS and SPECjbb®2013-<category> critical-jOPS where <category> : [Composite / MultiJVM / Distributed]
Required Metrics	Both metrics SPECjbb®2013-<category> max-jOPS and SPECjbb®2013-<category> critical-jOPS must be stated in proximity.
Conditionally Required Metrics
Use of Estimates	Not allowed
Disallowed Comparisons	In addition to the requirements that results not be compared to other benchmarks: Any comparison across categories SPECjbb®2013-Composite, SPECjbb®2013-MultiJVM and SPECjbb®2013-Distributed is prohibited

RETIRED	The SPECjbb 2005 benchmark was retired on October 1, 2013 in favor of its successor, the SPECjbb 2013 benchmark. All public use of results for this benchmark must plainly disclose that the benchmark has been retired, as described above. No further submissions will be accepted for publication at www.spec.org. SPEC is no longer reviewing results for this benchmark. Rule-compliant results may be published independently, provided that the fact of retirement is plainly disclosed
SPEC.org Submission Requirements	Vendors may publish compliant results independently, provided that the first use of input.expected_peak_warehouse property by the vendor be reviewed by the subcommittee to determine compliance with run rules section 2.3. Future publications by the vendor using input.expected_peak_warehouse do not require review unless the technical reason for setting the flag differs from what was previously accepted by the subcommittee.
SPEC Metrics	SPECjbb®2005 bops, SPECjbb®2005 bops/JVM
Required Metrics	The number of jvms used in the benchmark must be stated Both throughput metrics (SPECjbb®2005 bops and SPECjbb®2005 bops/JVM) must be stated.
Conditionally Required Metrics
Use of Estimates	Not allowed
Disallowed Comparisons	No additional requirements beyond the requirements that results not be compared to other benchmarks.