Disclosure of tuning options for SPEC Java benchmark results

Fujitsu Siemens Computers

Nov 13, 2002


For more information on command line options see http://java.sun.com/docs/hotspot 

JVM Command Line Options

Option

Description

-server

Use server (as opposed to client) JVM

-Xbatch

disables background compilation

-Xss128k

Specify the size of stack for each new Java thread.

-XX:+UseISM

Use Solaris Intimate Shared Memory (ISM)

-XX:+AggressiveHeap

This option instructs the JVM to push memory use to the limit

-XX:NewSize

Default size of new generation (in bytes)

-XX:MaxNewSize Maximum size of new generation (in bytes)

-XX:OverrideDefaultLibthread

Originally this flag was required when using the alternate threads library (see below). It is now obsolete and has no measureable performance influence.


For more information on large heaps with ISM, see below. Also see "Java HotSpotTM Performance Engine README"

/etc/system operating system tuning parameters

autoup

The frequency of file system sync operations.

rechoose_interval

Number of clock ticks before a process is deemed to have lost all affinity for the last CPU it ran on. After this interval expires, any CPU is considered a candidate for scheduling a thread. This parameter is relevant only for threads in the timesharing class. Real-time threads are scheduled on the first available CPU.

shmsys:shminfo_shmmax

Maximum size of system V shared memory segment that can be created.

shmsys:shminfo_shmmni

System wide limit on number of shared memory segments that can be created.

shmsys:shminfo_shmseg

Limit on the number of shared memory segments that any one process can create.

tune_t_fsflushr

The number of seconds between fsflush invocations for checking dirty memory.



Environment Variable tuning parameters

LD_LIBRARY_PATH=/usr/lib/lwp:/usr/lib

Use alternate thread library.


The standard Solaris threads implementation is built upon a two-level threading model in which user-level threads are multiplexed over possibly fewer lightweight processes, or LWPs. An LWP is the fundamental unit of execution that is dispatched to a processor by the operating system. The Solaris 8 Operating Environment provides an alternate threads implementation of a one-level model in which user-level threads are associated one-to-one with LWPs. This implementation is simpler than the standard implementation and may be beneficial to some multithreaded applications. It provides exactly the same interfaces, both for POSIX threads and Solaris threads, as the standard implementation.



The following information was copied from http://java.sun.com/docs/hotspot/ism.htmlon March 20, 2001.




Big Heaps and Intimate Shared Memory (ISM)


Introduction

The performance of some applications is limited by the amount of memory that the JVM can address.  In particular, multithreaded programs that allocate heavily often bottleneck in garbage collection, and the total time spent in garbage collection can generally be reduced by using a bigger heap.

This page describes options useful for improving performance on systems with large physical memory.  These options are available starting with J2SE 1.3.1, and are only intended for large server systems.  Large heaps should be avoided if there is insufficient physical memory to avoid paging.Because the options described have specific system requirements for correct operation and may require privileged access to system configuration parameters, they are not recommended for casual use.

These options may disappear in a future release, like all other options that begin with -X .  For example, future JVMs supporting a full 64 bit address space will likely have different sizing needs.
 

Using a Large Heap

Heaps larger than 2GB are available starting with J2SE 1.3.1.  These are options that can be useed to tune the use of memory by the JVM.  The values given should not be taken as suggestions; best values for your application will vary and should be set by measurement.

-Xms3g -Xmx3g

This example sets the minimum and maximum total heap size to 3GB.  The default maximum size is 64MB, but for many server applications it makes sense to be much larger.  The g suffix may be replaced by m to measure in megabytes.
Ordinarily the JVM consumes as little memory as possible, starting with a heap of the minimum size specified by the -Xms flag, and adaptively growing as required up to the maximum specified by -Xmx.  Setting these to the same value avoids being conservative, and will often improve startup time.

-XX:+AggressiveHeap

This option instructs the JVM to push memory use to the limit: the overall heap is 3900MB, the new generation is 3072MB, the allocation area of each thread is 256KB, and the collection policy defers as long as possible before collecting.
This option should be used with caution.  Because it makes the JVM greedy with memory resources, it is not appropriate for many programs and makes it easier to run out of memory.  For example, by using most of the 4GB address space for heap, some programs will run out of space for stack memory.  In such cases -Xss may sometimes help by reducing stack requirements.
 

ISM

A large heap places harsher demands on the hardware memory system.  Intimate Shared Memory (ISM) is a Solaris facility that can help by:

  1. Enabling larger pages - 4MB instead of the default 8KB

  2. Locking pages in memory - no paging to disk

  3. Allowing shared virtual to physical address data structures

The 1.3.1 JVM exploits the first two.  By reducing the overhead of virtual to physical address translation, a significant performance boost can often be obtained (typically 5-10%) .  However, not all applications improve with ISM (it is possible to get worse), so it is important to monitor performance and resource use.

Requirements

Using ISM

To use ISM, add the following to the /etc/system file:

set shmsys:shminfo_shmmax=0xffffffff
set shmsys:shminfo_shmseg=32

If your system is 2.6, you may also need to add

set enable_grp_ism=1

After those modifications, reboot and run the VM with the flag

    -XX:+UseISM

Problems and Dangers

ISM locks memory and makes it unavailable to other processes or the OS.  Since the number of locked pages is based on the total heap specified, one could potential hold all of the system's memory.  This option is designed for systems dedicated to specific tasks, such as a web server; ISM should not be enabled on systems with no control over the applications that arbitrary users can run.

ISM does not guarantee that the entire heap uses 4MB pages.  At boot time most of physical memory is contiguous, but due to fragmentation it is possible that 4MB blocks of physical memory are no longer available.  ISM will use as many 4MB pages as possible and then silently use 8KB pages for the rest, reducing performance.  A performance loss may be observed over time as more applications use and return fragmented pages to the OS; the only way to guarantee that the entire region is using 4MB pages is to first reboot the system.

ISM uses the system calls shmget to allocate a region of memory, and shmat with the SHM_SHARE_MMU flag to attach to this region.  Under abnormal termination it is possible to have the shared memory segment stay resident in the system. Although normal exits and common signals prompt the VM to remove the shared segment, a SIGKILL (kill -9) of a process is not catchable so no cleanup will occur. This leads to a portion of physical memory permanently unavailable to any applications.  To remove such a discarded segment manually, use the command ipcs to determine the shared memory ID, and ipcrm -m to remove it.

 

Copyright © 2000, 2001 Sun Microsystems, Inc. All Rights Reserved.