Oracle SPARC Series Platform File

Commands and Options Used to Submit Benchmark Runs

submit=echo 'pbind -b...' > dobmk; sh dobmk (SPEC tools, Unix shell)
When running multiple copies of benchmarks, the SPEC config file feature submit is often used to cause individual jobs to be bound to specific processors. If so, the specific command may be found in the config file; here is a guide to understanding that command:


Shell, Environment, and Other Software Settings

LD_LIBRARY_PATH=<directories>
LD_LIBRARY_PATH controls the search order for both the compile-time and run-time linkers. Usually, it can be defaulted; but testers may sometimes choose to explicitly set it (as documented in the notes in the submission), in order to ensure that the correct versions of libraries are picked up.

MADV=access_lwp and LD_PRELOAD=madv.so.1
When the madv.so.1 shared object is present in the LD_PRELOAD list, it is possible to provide advice to the system about how memory is likely to be accessed. The advice present in MADV applies to all processes and their descendants. A commonly used value is access_lwp, which means that when memory is allocated, the next process to touch it will be the primary user. Examples of other possible values include sequential, for memory that is used only once and then no longer needed and acces_many when many processes will be sharing data.

MPSSHEAP=<size>, MPSSSTACK=<size>, and LD_PRELOAD=mpss.so.1
When these variables are set, the mpss.so.1 shared object will set the preferred page size for new processes, and their descendants, to the requested sizes for the heap and stack.


Operating System Tuning Parameters

Platform settings

One or more of the following settings may have been applied to the testbed. If so, the "Platform Notes" section of the report will say so; and you can read below to find out more about what these settings mean.

autoup=<n> (Unix /etc/system)
When the file system flush daemon fsflush runs, it writes to disk all modified file buffers that are more than n seconds old.

bufhwm=<n> (Unix /etc/system)
Sets the upper limit of the file system buffer cache. The units for bufhwm are in kilobytes.

cpu_bringup_set=<n> (Unix /etc/system)
Specifies which processors are enabled at boot time. <n> represents a bitmap of the processors to be brought online.

poweradm set administrative-authority=none (Unix, superuser commands)
Solaris power management settings are determined by an 'administrative-authority'. When the authority is set to 'none', Solaris does not attempt to manage power.

psrset -c <n> (Unix, superuser commands)
Creates a new processor set and displays the new processor set ID.

psrset -e <n> (Unix, superuser commands)
Executes a command (with optional arguments) in the specified processor set. The command process and any child processes are executed only by processors in the processor set.

segmap_percent=<n> (Unix /etc/system)
This value controls the size of the segmap cache as a percent of total memory. Set this value to help keep the file system cache from consuming memory unnecessarily.

STACKSIZE=<n> (Unix environment variable)
Set the size of the stack (temporary storage area) for each slave thread of a multithreaded program.

svcadm disable webconsole (Unix, superuser commands)
Turns off the Sun Web Console, a browser-based interface that performs systems management. If it is enabled, system administrators can manage systems, devices and services from remote systems.

ts_dispatch_extended=<n> (Unix /etc/system)
Controls which dispatch table is loaded upon boot. A value of 1 loads the large system table, a value of 0 loads the regular system table.

tune_t_fsflushr=<n> (Unix /etc/system)
Controls the number of seconds between runs of the file system flush daemon, fsflush.

user_reserve_hint_pct=<n> (Unix /etc/system)
Informs the system about how much memory is expected to be used by applications (as a percentage).


Virtualization Settings

Virtualization in SPARC systems

SPARC systems provide virtualization capabilities at several levels:

At the time of release of SPEC CPU2017, Oracle submitted results to SPEC for a
1-Chip VM with SPARC M7
which used 2 of the above technologies, namely LDoms and PDoms. The example shown here is that system.

From the point of view of the VM

A SUT (System Under Test) using Oracle VM Server for SPARC is able to accurately inventory cores and virtual processors using ordinary Solaris commands.
In this example system, there are 256 cpus, 32 cores, 1 chip:

$ hostname
m7-16-002c-ld3
$ 
$ pkg info entire | grep Version
          Version: 0.5.11 (Oracle Solaris 11.3.10.5.0)
$ 
$ psrinfo -pv
The physical processor has 32 cores and 256 virtual processors (0-255)
  The core has 8 virtual processors (0-7)
  The core has 8 virtual processors (8-15)
  The core has 8 virtual processors (16-23)
  The core has 8 virtual processors (24-31)
  The core has 8 virtual processors (32-39)
  The core has 8 virtual processors (40-47)
  The core has 8 virtual processors (48-55)
  The core has 8 virtual processors (56-63)
  The core has 8 virtual processors (64-71)
  The core has 8 virtual processors (72-79)
  The core has 8 virtual processors (80-87)
  The core has 8 virtual processors (88-95)
  The core has 8 virtual processors (96-103)
  The core has 8 virtual processors (104-111)
  The core has 8 virtual processors (112-119)
  The core has 8 virtual processors (120-127)
  The core has 8 virtual processors (128-135)
  The core has 8 virtual processors (136-143)
  The core has 8 virtual processors (144-151)
  The core has 8 virtual processors (152-159)
  The core has 8 virtual processors (160-167)
  The core has 8 virtual processors (168-175)
  The core has 8 virtual processors (176-183)
  The core has 8 virtual processors (184-191)
  The core has 8 virtual processors (192-199)
  The core has 8 virtual processors (200-207)
  The core has 8 virtual processors (208-215)
  The core has 8 virtual processors (216-223)
  The core has 8 virtual processors (224-231)
  The core has 8 virtual processors (232-239)
  The core has 8 virtual processors (240-247)
  The core has 8 virtual processors (248-255)
    SPARC-M7 (chipid 3, clock 4133 MHz)
$ 

From the point of view of the Host

Oracle VM Server for SPARC manages Logical Domains using the ldm command.
From its point of view, the example SUT just above is ld3, which uses CMIOU11 (Chip/Memory/IO Unit #11).

# hostname
m7-16-002c
# pkg info entire | grep Version:
       Version: 0.5.11 (Oracle Solaris 11.3.5.1.0)
# ldm --version

Logical Domains Manager (v 3.3.0.1.4)
        Hypervisor control protocol v 1.12
        Using Hypervisor MD v 1.4

System PROM:
        Hostconfig      v. 1.6.2.b      @(#)Hostconfig 1.6.2.b 2015/12/07 09:05
        Hypervisor      v. 1.15.2.a     @(#)Hypervisor 1.15.2.a 2015/11/20 04:45 
        OpenBoot        v. 4.38.2       @(#)OpenBoot 4.38.2 2015/10/30 13:09
# ldm list
NAME             STATE      FLAGS   CONS    VCPU  MEMORY   UTIL  NORM  UPTIME
primary          active     -n-cv-  UART    256   489984M  0.1%  0.1%  7d 2h 38m
ld1              active     -n----  5001    256   489984M  0.1%  0.1%  7d 2h 38m
ld2              active     -n----  5002    256   489984M  0.0%  0.0%  7d 2h 38m
ld3              active     -n----  5003    256   489984M  0.0%  0.0%  3d 8h 45m
# ldm list-rsrc-group -l -o mem,core /SYS/CMIOU11
NAME                                    
/SYS/CMIOU11                            

CORE
    CID                                             BOUND            
    704, 705, 706, 707, 712, 713, 714, 715          ld3              
    720, 721, 722, 723, 728, 729, 730, 731          ld3              
    736, 737, 738, 739, 744, 745, 746, 747          ld3              
    752, 753, 754, 755, 760, 761, 762, 763          ld3              

MEMORY
    PA               SIZE             BOUND            
    0x580050000000   489984M          ld3              
    0x5877f0000000   64M              _sys_            
    0x5877f4000000   192M             _sys_            
# 

From the point of view of the Service Processor

Using the Service Processor, a system administrator defines Physical Domains (PDoms).
In the example system, there is one Physical Domain per DCU (Domain Configurable Unit).
CMIOU11 is part of DCU2, which is assigned to HOST2.

-> show /SP system_description

  /SP
    Properties:
        system_description = SPARC M7-16, ILOM v3.2.5.8.g, r105871

-> show /System system_fw_version

  /System
    Properties:
        system_fw_version = Sun System Firmware 9.5.2.g 2015/12/07 11:57

-> show HOST2 dcus_assigned   

  /HOST2
    Properties:
        dcus_assigned = /SYS/DCU2

-> show /SYS/DCU2 members     

  /SYS/DCU2
    Properties:
        members = /SYS/CMIOU8/CM/CMP /SYS/CMIOU9/CM/CMP /SYS/CMIOU10/CM/CMP 
                  /SYS/CMIOU11/CM/CMP

-> 

The service processor is able to show many details about the hardware
For the system discussed in this example, CMIOU11 has 16 memory DIMMs, each of which has 32 GB.

   [Output was edited to reduce white space]

-> show /SYS/CMIOU11/CM/CMP fru_name -t -format nowrap
Target                             | Property         | Value                    
-----------------------------------+------------------+--------------------------
/SYS/CMIOU11/CM/CMP                | fru_name         | Oracle SPARC M7
/SYS/CMIOU11/CM/CMP/BOB00/CH0/DIMM | fru_name         | 32768MB DDR4 SDRAM DIMM
/SYS/CMIOU11/CM/CMP/BOB00/CH1/DIMM | fru_name         | 32768MB DDR4 SDRAM DIMM
/SYS/CMIOU11/CM/CMP/BOB01/CH0/DIMM | fru_name         | 32768MB DDR4 SDRAM DIMM
/SYS/CMIOU11/CM/CMP/BOB01/CH1/DIMM | fru_name         | 32768MB DDR4 SDRAM DIMM
/SYS/CMIOU11/CM/CMP/BOB10/CH0/DIMM | fru_name         | 32768MB DDR4 SDRAM DIMM
/SYS/CMIOU11/CM/CMP/BOB10/CH1/DIMM | fru_name         | 32768MB DDR4 SDRAM DIMM
/SYS/CMIOU11/CM/CMP/BOB11/CH0/DIMM | fru_name         | 32768MB DDR4 SDRAM DIMM
/SYS/CMIOU11/CM/CMP/BOB11/CH1/DIMM | fru_name         | 32768MB DDR4 SDRAM DIMM
/SYS/CMIOU11/CM/CMP/BOB20/CH0/DIMM | fru_name         | 32768MB DDR4 SDRAM DIMM
/SYS/CMIOU11/CM/CMP/BOB20/CH1/DIMM | fru_name         | 32768MB DDR4 SDRAM DIMM
/SYS/CMIOU11/CM/CMP/BOB21/CH0/DIMM | fru_name         | 32768MB DDR4 SDRAM DIMM
/SYS/CMIOU11/CM/CMP/BOB21/CH1/DIMM | fru_name         | 32768MB DDR4 SDRAM DIMM
/SYS/CMIOU11/CM/CMP/BOB30/CH0/DIMM | fru_name         | 32768MB DDR4 SDRAM DIMM
/SYS/CMIOU11/CM/CMP/BOB30/CH1/DIMM | fru_name         | 32768MB DDR4 SDRAM DIMM
/SYS/CMIOU11/CM/CMP/BOB31/CH0/DIMM | fru_name         | 32768MB DDR4 SDRAM DIMM
/SYS/CMIOU11/CM/CMP/BOB31/CH1/DIMM | fru_name         | 32768MB DDR4 SDRAM DIMM

-> show /SYS/CMIOU11/CM/CMP fru_manufacturer -t -format nowrap
Target                             | Property         | Value                    
-----------------------------------+------------------+--------------------------
/SYS/CMIOU11/CM/CMP                | fru_manufacturer | Oracle Corporation
/SYS/CMIOU11/CM/CMP/BOB00/CH0/DIMM | fru_manufacturer | Samsung
/SYS/CMIOU11/CM/CMP/BOB00/CH1/DIMM | fru_manufacturer | Samsung
/SYS/CMIOU11/CM/CMP/BOB01/CH0/DIMM | fru_manufacturer | Samsung
/SYS/CMIOU11/CM/CMP/BOB01/CH1/DIMM | fru_manufacturer | Samsung
/SYS/CMIOU11/CM/CMP/BOB10/CH0/DIMM | fru_manufacturer | Samsung
/SYS/CMIOU11/CM/CMP/BOB10/CH1/DIMM | fru_manufacturer | Samsung
/SYS/CMIOU11/CM/CMP/BOB11/CH0/DIMM | fru_manufacturer | Samsung
/SYS/CMIOU11/CM/CMP/BOB11/CH1/DIMM | fru_manufacturer | Samsung
/SYS/CMIOU11/CM/CMP/BOB20/CH0/DIMM | fru_manufacturer | Samsung
/SYS/CMIOU11/CM/CMP/BOB20/CH1/DIMM | fru_manufacturer | Samsung
/SYS/CMIOU11/CM/CMP/BOB21/CH0/DIMM | fru_manufacturer | Samsung
/SYS/CMIOU11/CM/CMP/BOB21/CH1/DIMM | fru_manufacturer | Samsung
/SYS/CMIOU11/CM/CMP/BOB30/CH0/DIMM | fru_manufacturer | Samsung
/SYS/CMIOU11/CM/CMP/BOB30/CH1/DIMM | fru_manufacturer | Samsung
/SYS/CMIOU11/CM/CMP/BOB31/CH0/DIMM | fru_manufacturer | Samsung
/SYS/CMIOU11/CM/CMP/BOB31/CH1/DIMM | fru_manufacturer | Samsung

-> show /SYS/CMIOU11/CM/CMP fru_part_number -t -format nowrap 
Target                             | Property         | Value                    
-----------------------------------+------------------+--------------------------
/SYS/CMIOU11/CM/CMP/BOB00/CH0/DIMM | fru_part_number  | 07075400,M393A4K40BB1-CRC
/SYS/CMIOU11/CM/CMP/BOB00/CH1/DIMM | fru_part_number  | 07075400,M393A4K40BB1-CRC
/SYS/CMIOU11/CM/CMP/BOB01/CH0/DIMM | fru_part_number  | 07075400,M393A4K40BB1-CRC
/SYS/CMIOU11/CM/CMP/BOB01/CH1/DIMM | fru_part_number  | 07075400,M393A4K40BB1-CRC
/SYS/CMIOU11/CM/CMP/BOB10/CH0/DIMM | fru_part_number  | 07075400,M393A4K40BB1-CRC
/SYS/CMIOU11/CM/CMP/BOB10/CH1/DIMM | fru_part_number  | 07075400,M393A4K40BB1-CRC
/SYS/CMIOU11/CM/CMP/BOB11/CH0/DIMM | fru_part_number  | 07075400,M393A4K40BB1-CRC
/SYS/CMIOU11/CM/CMP/BOB11/CH1/DIMM | fru_part_number  | 07075400,M393A4K40BB1-CRC
/SYS/CMIOU11/CM/CMP/BOB20/CH0/DIMM | fru_part_number  | 07075400,M393A4K40BB1-CRC
/SYS/CMIOU11/CM/CMP/BOB20/CH1/DIMM | fru_part_number  | 07075400,M393A4K40BB1-CRC
/SYS/CMIOU11/CM/CMP/BOB21/CH0/DIMM | fru_part_number  | 07075400,M393A4K40BB1-CRC
/SYS/CMIOU11/CM/CMP/BOB21/CH1/DIMM | fru_part_number  | 07075400,M393A4K40BB1-CRC
/SYS/CMIOU11/CM/CMP/BOB30/CH0/DIMM | fru_part_number  | 07075400,M393A4K40BB1-CRC
/SYS/CMIOU11/CM/CMP/BOB30/CH1/DIMM | fru_part_number  | 07075400,M393A4K40BB1-CRC
/SYS/CMIOU11/CM/CMP/BOB31/CH0/DIMM | fru_part_number  | 07075400,M393A4K40BB1-CRC
/SYS/CMIOU11/CM/CMP/BOB31/CH1/DIMM | fru_part_number  | 07075400,M393A4K40BB1-CRC

-> 

For more information

See: www.oracle.com/technetwork/server-storage/sun-sparc-enterprise/documentation/sparc-t7-m7-server-architecture-2702877.pdf


Firmware / BIOS / Microcode Settings

Power management policy (Integrated Lights Out Manager, ILOM)

The power management settings available from the ILOM menus, and their meanings, are: