SPEC® MPIM2007 Result

Copyright 2006-2010 Standard Performance Evaluation Corporation

SGI

SGI ICE X (Intel Xeon E5-2690 v2, 3.0 GHz)

MPI2007 license: 4 Test date: Dec-2013
Test sponsor: SGI Hardware Availability: Sep-2013
Tested by: SGI Software Availability: Nov-2013
Benchmark results graph

Results Table

Benchmark Base Peak
Ranks Seconds Ratio Seconds Ratio Seconds Ratio Ranks Seconds Ratio Seconds Ratio Seconds Ratio
Results appear in the order in which they were run. Bold underlined text indicates a median measurement.
104.milc 800 11.4 138   11.4 138   11.3 138   800 11.4 138   11.4 138   11.3 138  
107.leslie3d 800 34.2 153   34.2 153   34.3 152   800 34.2 153   34.2 153   34.3 152  
113.GemsFDTD 800 215   29.4 216   29.2 215   29.3 160 156   40.3 156   40.3 157   40.2
115.fds4 800 12.5 156   11.3 173   12.5 156   640 11.6 167   11.5 170   11.8 166  
121.pop2 800 105   39.3 105   39.4 105   39.3 800 105   39.3 105   39.4 105   39.3
122.tachyon 800 23.2 120   23.6 119   23.3 120   800 23.2 120   23.6 119   23.3 120  
126.lammps 800 102   28.6 102   28.6 102   28.6 160 102   28.7 101   28.8 101   28.8
127.wrf2 800 35.2 222   35.4 220   35.2 222   800 35.2 222   35.4 220   35.2 222  
128.GAPgeofem 800 10.5 197   11.0 187   11.1 186   800 10.5 197   11.0 187   11.1 186  
129.tera_tf 800 23.8 116   23.4 118   23.7 117   800 23.8 116   23.4 118   23.7 117  
130.socorro 800 41.1 93.0 41.1 93.0 41.3 92.4 800 41.1 93.0 41.1 93.0 41.3 92.4
132.zeusmp2 800 25.1 123   25.0 124   25.1 123   800 25.1 123   25.0 124   25.1 123  
137.lu 800 25.6 144   25.5 144   25.7 143   800 25.6 144   25.5 144   25.7 143  
Hardware Summary
Type of System: Homogeneous
Compute Node: SGI ICE X IP-113 Compute Node
Interconnect: InfiniBand (MPI and I/O)
File Server Node: SGI Modular InfiniteStorage Server
Total Compute Nodes: 40
Total Chips: 80
Total Cores: 800
Total Threads: 1600
Total Memory: 2560 GB
Base Ranks Run: 800
Minimum Peak Ranks: 160
Maximum Peak Ranks: 800
Software Summary
C Compiler: Intel C++ Composer XE 2011 for Linux,
Version 14.0.0.080 Build 20130728
C++ Compiler: Intel C++ Composer XE 2011 for Linux,
Version 14.0.0.080 Build 20130728
Fortran Compiler: Intel Fortran Composer XE 2011 for Linux,
Version 14.0.0.080 Build 20130728
Base Pointers: 64-bit
Peak Pointers: 64-bit
MPI Library: SGI MPT 2.09
Other MPI Info: OFED 1.5.2
Pre-processors: None
Other Software: None

Node Description: SGI ICE X IP-113 Compute Node

Hardware
Number of nodes: 40
Uses of the node: compute
Vendor: SGI
Model: SGI ICE X IP-113 (Intel Xeon E5-2690 v2, 3.0 GHz)
CPU Name: Intel Xeon E5-2690 v2
CPU(s) orderable: 1-2 chips
Chips enabled: 2
Cores enabled: 20
Cores per chip: 10
Threads per core: 2
CPU Characteristics: Ten Core, 3.0 GHz, 8.0 GT/s QPI
Intel Turbo Boost Technology up to 3.60 GHz
Hyper-Threading Technology enabled
CPU MHz: 3000
Primary Cache: 32 KB I + 32 KB D on chip per core
Secondary Cache: 256 KB I+D on chip per core
L3 Cache: 25 MB I+D on chip per chip
Other Cache: None
Memory: 64 GB (8 x 8 GB 2Rx4 PC3-14900R-13, ECC)
Disk Subsystem: None
Other Hardware: None
Adapter: Mellanox MT27500 with ConnectX-3 ASIC
(PCIe x8 Gen3 8 GT/s)
Number of Adapters: 2
Slot Type: PCIe x8 Gen3
Data Rate: InfiniBand 4x FDR
Ports Used: 2
Interconnect Type: InfiniBand
Software
Adapter: Mellanox MT27500 with ConnectX-3 ASIC
(PCIe x8 Gen3 8 GT/s)
Adapter Driver: OFED-1.5.2
Adapter Firmware: 2.11.312
Operating System: SUSE Linux Enterprise Server 11 SP2,
Kernel 3.0.80-0.7-default
Local File System: NFSv3
Shared File System: NFSv3 IPoIB
System State: Multi-user, run level 3
Other Software: SGI Tempo Compute Node 2.7.3,
Build 708rp14.sles11sp2-1305311204

Node Description: SGI Modular InfiniteStorage Server

Hardware
Number of nodes: 1
Uses of the node: fileserver
Vendor: SGI
Model: SGI Modular InfiniteStorage Server
CPU Name: Intel Xeon E5-2670
CPU(s) orderable: 1-2 chips
Chips enabled: 2
Cores enabled: 16
Cores per chip: 8
Threads per core: 2
CPU Characteristics: Intel Turbo Boost Technology up to 3.33 GHz
Hyper-Threading Technology enabled
CPU MHz: 2600
Primary Cache: 32 KB I + 32 KB D on chip per core
Secondary Cache: 256 KB I+D on chip per chip
L3 Cache: 20 MB I+D on chip per chip
Other Cache: None
Memory: 128 GB (8 * 16 GB 2Rx4 PC3-12800R-11, ECC)
Disk Subsystem: 64.8 TB RAID 6
72 x 900 GB SAS (Western Digital, 10K RPM)
Other Hardware: None
Adapter: Mellanox MT27500 with ConnectX-3 ASIC
(PCIe x8 Gen3 8 GT/s)
Number of Adapters: 2
Slot Type: PCIe x8 Gen3
Data Rate: InfiniBand 4x FDR
Ports Used: 2
Interconnect Type: InfiniBand
Software
Adapter: Mellanox MT27500 with ConnectX-3 ASIC
(PCIe x8 Gen3 8 GT/s)
Adapter Driver: OFED-1.5.0
Adapter Firmware: 2.11.312
Operating System: SUSE Linux Enterprise Server 11 SP3
Kernel
Local File System: xfs
Shared File System: --
System State: Multi-user, run level 3
Other Software: SGI Foundation Software 2.9,
Build 700r3.sles11-1004061553

Interconnect Description: InfiniBand (MPI and I/O)

Hardware
Vendor: Mellanox Technologies and SGI
Model: None
Switch Model: SGI FDR Integrated IB Switch Blade 2SW9x27 with
Mellanox SwitchX device 51000
Number of Switches: 12
Number of Ports: 36
Data Rate: InfiniBand 4x FDR
Firmware: 07130007_LLR and 08130007_LLR
Topology: Enhanced Hypercube
Primary Use: MPI and I/O traffic

Submit Notes

The config file option 'submit' was used.

General Notes

130.socorro (base): "nullify_ptrs" src.alt was used.

 Software environment:
   export MPI_REQUEST_MAX=65536
   export MPI_TYPE_MAX=32768
   export MPI_BUFS_THRESHOLD=1
   export MPI_IB_RAILS=2
   export MPI_CONNECTIONS_THRESHOLD=0
   ulimit -s unlimited

 BIOS settings:
   AMI BIOS version 3.0
   Hyper-Threading Technology enabled (default)
   Intel Turbo Boost Technology enabled (default)
   Intel Turbo Boost Technology activated in the OS via
     /etc/init.d/acpid start
     /etc/init.d/powersaved start
     powersave -f

 Job Placement:
   Each MPI job was assigned to a topologically compact set
   of nodes, i.e. the minimal needed number of switches was
   used for each job: 2 switches for up to 180 ranks,
   4 switches for up to 320 ranks, 8 switches for 640 ranks,
   10 switches for 800 ranks, 16 switches for 1280 ranks,
   22 switches for 1920 ranks, and 30 switches for 2560 ranks.

 Additional notes regarding interconnect:
   The Infiniband network consists of two independent planes,
   with half the switches in the system allocated to each plane.
   I/O traffic is restricted to one plane, while MPI traffic can
   use both planes.

 Peak run:
   In the peak run, some benchmarks used different number of ranks
   from base. It is the only difference between base and peak.

Compiler Invocation

C benchmarks:

 icc 

C++ benchmarks:

126.lammps:  icpc 

Fortran benchmarks:

 ifort 

Benchmarks using both Fortran and C:

 icc   ifort 

Portability Flags

121.pop2:  -DSPEC_MPI_CASE_FLAG 
127.wrf2:  -DSPEC_MPI_CASE_FLAG   -DSPEC_MPI_LINUX 
130.socorro:  -assume nostd_intent_in 

Base Optimization Flags

C benchmarks:

 -O3   -xAVX   -no-prec-div 

C++ benchmarks:

126.lammps:  -O3   -xAVX   -no-prec-div   -ansi-alias 

Fortran benchmarks:

 -O3   -xAVX   -no-prec-div 

Benchmarks using both Fortran and C:

 -O3   -xAVX   -no-prec-div 

Peak Optimization Flags

C benchmarks:

104.milc:  basepeak = yes 
122.tachyon:  basepeak = yes 

C++ benchmarks:

126.lammps:  -O3   -xAVX   -no-prec-div   -ansi-alias 

Fortran benchmarks:

107.leslie3d:  basepeak = yes 
113.GemsFDTD:  -O3   -xAVX   -no-prec-div 
129.tera_tf:  basepeak = yes 
137.lu:  basepeak = yes 

Benchmarks using both Fortran and C:

115.fds4:  -O3   -xAVX   -no-prec-div 
121.pop2:  basepeak = yes 
127.wrf2:  basepeak = yes 
128.GAPgeofem:  basepeak = yes 
130.socorro:  basepeak = yes 
132.zeusmp2:  basepeak = yes 

Other Flags

C benchmarks:

 -lmpi 

C++ benchmarks:

126.lammps:  -lmpi 

Fortran benchmarks:

 -lmpi 

Benchmarks using both Fortran and C:

 -lmpi 

The flags file that was used to format this result can be browsed at
http://www.spec.org/mpi2007/flags/SGI_x86_64_Intel14_flags.html.

You can also download the XML flags source by saving the following link:
http://www.spec.org/mpi2007/flags/SGI_x86_64_Intel14_flags.xml.