Disclosure of tuning options for SPEC Java benchmark results

Fujitsu Ltd.

Mar 20, 2002

Because the same software (HotSpot 4.0 Java Virtual Maschine, Solaris Operating System) was used as in a previous SPECjbb2000 result submission from Sun Microsystems, the following text of this disclosure is identical to that of http://www.spec.org/osg/jbb2000/flags/Sun-20020307.html.

JVM Command Line Options




Use server (as opposed to client) JVM


disables background compilation


Specify the size of stack for each new Java thread.


Use Solaris Intimate Shared Memory (ISM)


This option instructs the JVM to push memory use to the limit

For more information on large heaps with ISM, see below. Also see "Java HotSpotTM Performance Engine README"

/etc/system operating system tuning parameters


The frequency of file system sync operations.


Number of clock ticks before a process is deemed to have lost all affinity for the last CPU it ran on. After this interval expires, any CPU is considered a candidate for scheduling a thread. This parameter is relevant only for threads in the timesharing class. Real-time threads are scheduled on the first available CPU.


Maximum size of system V shared memory segment that can be created.


System wide limit on number of shared memory segments that can be created.


Limit on the number of shared memory segments that any one process can create.


The number of seconds between fsflush invocations for checking dirty memory.

Environment Variable tuning parameters


Use alternate thread library.

The standard Solaris threads implementation is built upon a two-level threading model in which user-level threads are multiplexed over possibly fewer lightweight processes, or LWPs. An LWP is the fundamental unit of execution that is dispatched to a processor by the operating system. The Solaris 8 Operating Environment provides an alternate threads implementation of a one-level model in which user-level threads are associated one-to-one with LWPs. This implementation is simpler than the standard implementation and may be beneficial to some multithreaded applications. It provides exactly the same interfaces, both for POSIX threads and Solaris threads, as the standard implementation.

The following information was copied from http://java.sun.com/docs/hotspot/ism.html on March 20, 2001.

Big Heaps and Intimate Shared Memory (ISM)


The performance of some applications is limited by the amount of memory that the JVM can address.  In particular, multithreaded programs that allocate heavily often bottleneck in garbage collection, and the total time spent in garbage collection can generally be reduced by using a bigger heap.

This page describes options useful for improving performance on systems with large physical memory.  These options are available starting with J2SE 1.3.1, and are only intended for large server systems.  Large heaps should be avoided if there is insufficient physical memory to avoid paging.Because the options described have specific system requirements for correct operation and may require privileged access to system configuration parameters, they are not recommended for casual use.

These options may disappear in a future release, like all other options that begin with -X .  For example, future JVMs supporting a full 64 bit address space will likely have different sizing needs.

Using a Large Heap

Heaps larger than 2GB are available starting with J2SE 1.3.1.  These are options that can be useed to tune the use of memory by the JVM.  The values given should not be taken as suggestions; best values for your application will vary and should be set by measurement.

-Xms3g -Xmx3g

This example sets the minimum and maximum total heap size to 3GB.  The default maximum size is 64MB, but for many server applications it makes sense to be much larger.  The g suffix may be replaced by m to measure in megabytes.
Ordinarily the JVM consumes as little memory as possible, starting with a heap of the minimum size specified by the -Xms flag, and adaptively growing as required up to the maximum specified by -Xmx.  Setting these to the same value avoids being conservative, and will often improve startup time.


This option instructs the JVM to push memory use to the limit: the overall heap is 3900MB, the new generation is 3072MB, the allocation area of each thread is 256KB, and the collection policy defers as long as possible before collecting.
This option should be used with caution.  Because it makes the JVM greedy with memory resources, it is not appropriate for many programs and makes it easier to run out of memory.  For example, by using most of the 4GB address space for heap, some programs will run out of space for stack memory.  In such cases -Xss may sometimes help by reducing stack requirements.


A large heap places harsher demands on the hardware memory system.  Intimate Shared Memory (ISM) is a Solaris facility that can help by:

  1. Enabling larger pages - 4MB instead of the default 8KB

  2. Locking pages in memory - no paging to disk

  3. Allowing shared virtual to physical address data structures

The 1.3.1 JVM exploits the first two.  By reducing the overhead of virtual to physical address translation, a significant performance boost can often be obtained (typically 5-10%) .  However, not all applications improve with ISM (it is possible to get worse), so it is important to monitor performance and resource use.


Using ISM

To use ISM, add the following to the /etc/system file:

set shmsys:shminfo_shmmax=0xffffffff
set shmsys:shminfo_shmseg=32

If your system is 2.6, you may also need to add

set enable_grp_ism=1

After those modifications, reboot and run the VM with the flag


Problems and Dangers

ISM locks memory and makes it unavailable to other processes or the OS.  Since the number of locked pages is based on the total heap specified, one could potential hold all of the system's memory.  This option is designed for systems dedicated to specific tasks, such as a web server; ISM should not be enabled on systems with no control over the applications that arbitrary users can run.

ISM does not guarantee that the entire heap uses 4MB pages.  At boot time most of physical memory is contiguous, but due to fragmentation it is possible that 4MB blocks of physical memory are no longer available.  ISM will use as many 4MB pages as possible and then silently use 8KB pages for the rest, reducing performance.  A performance loss may be observed over time as more applications use and return fragmented pages to the OS; the only way to guarantee that the entire region is using 4MB pages is to first reboot the system.

ISM uses the system calls shmget to allocate a region of memory, and shmat with the SHM_SHARE_MMU flag to attach to this region.  Under abnormal termination it is possible to have the shared memory segment stay resident in the system. Although normal exits and common signals prompt the VM to remove the shared segment, a SIGKILL (kill -9) of a process is not catchable so no cleanup will occur. This leads to a portion of physical memory permanently unavailable to any applications.  To remove such a discarded segment manually, use the command ipcs to determine the shared memory ID, and ipcrm -m to remove it.


Copyright © 2000, 2001 Sun Microsystems, Inc. All Rights Reserved.