SPEC Seal of Reviewal SPECjAppServer®2004 Result
Copyright © 2004 Standard Performance Evaluation Corporation
Oracle WebLogic Server Standard Edition Release 10.3.3 on Cisco UCS B230 M1 Blade Servers
11,283.80 SPECjAppServer2004 JOPS@Standard
Submitter:
Cisco Systems, Inc.
SPEC license # 9019 Test date:
Aug-2010
Software Products
Oracle WebLogic Server Standard Edition Release 10.3.3
Oracle JRockit(R) 6.0 JDK (R28.0.1) (Linux x86 64bit)
Java HotSpot(TM) 64-Bit Server VM on Linux, version 1.6.0_21
Oracle JDBC Driver 11.2.0.1.0 (Thin)
Oracle Database Enterprise Edition Release 11.2.0.1.0
Software Configurations
J2EE Application Server
Emulator SW Config
Database SW Config
Driver SW Config Primary and Satellites
Hardware Systems
J2EE Application Server
Database Server HW
Primary Driver, Satellite Drivers & Emulator HW

System Configuration Diagram

Benchmark
Modifications
Configuration
Bill of Materials
Other Info
General Notes
Full Disclosure Archive

SUT Configuration
J2EE Server Nodes: 2   DB Server Nodes: 1
J2EE Server CPUs: 32 cores, 4 chips   DB Server CPUs: 32 cores, 4 chips
J2EE Instances: 16   DB Instances: 1
Other SUT Components:
   

Benchmark Settings
Dealer Injection Rate: 6,750   Ramp Up Time: 1,200 seconds
# of Dealer Agents: 30   Ramp Down Time: 300 seconds
# of Manufacturing Agents: 24   Steady State Time: 3,600 seconds
DB Load Injection Rate: 7,000   Trigger Time: 1,200 seconds

Detailed Results
Dealer Transaction Mix Count Tx. Mix
Purchase 6,097,245 24.99%
Manage 6,097,420 24.99%
Browse 12,202,982 50.02%
Total # Dealer Transactions 24,397,647
Total # Manufacturing Transactions 16,224,042
Response Times Average Maximum 90th% Reqd
Purchase 0.37 15.98 0.90 2.00
Manage 0.27 17.53 0.60 2.00
Browse 0.47 12.67 1.10 2.00
Manufacturing 2.32 11.32 4.00 5.00
Reproducibility Run (SPECjAppServer2004 JOPS) 11,284.19

Oracle WebLogic Server Standard Edition Release 10.3.3
Vendor: Oracle Corporation Availability: Apr-2010
CTS Version: 5.0 Date Passed CTS: Feb-2007
OS Name: Oracle Enterprise Linux 5 Update 5 x86_64
EJB Protocol: RMI/T3

Oracle JRockit(R) 6.0 JDK (R28.0.1) (Linux x86 64bit)
Vendor: Oracle Corporation Availability: Jul-2010
OS Name: Oracle Enterprise Linux 5 Update 5 x86_64

Java HotSpot(TM) 64-Bit Server VM on Linux, version 1.6.0_21
Vendor: Oracle Corporation Availability: Jul-2010
OS Name: Red Hat Enterprise Linux 5 Update 5 x86_64

Oracle JDBC Driver 11.2.0.1.0 (Thin)
Vendor: Oracle Corporation Availability: Oct-2009
OS Name: Oracle Enterprise Linux 5 Update 5 x86_64

Oracle Database Enterprise Edition Release 11.2.0.1.0
Vendor: Oracle Corporation Availability: Oct-2009
OS Name: Oracle Enterprise Linux 5 Update 5 x86_64

J2EE Application Server
J2EE Product: Oracle WebLogic Server Standard Edition Release 10.3.3
JVM Product: Oracle JRockit(R) 6.0 JDK (R28.0.1) (Linux x86 64bit)
JDBC Product: Oracle JDBC Driver 11.2.0.1.0 (Thin)
Hardware: J2EE Application Server
# of Instances: 16
Interfaces:
Web Interface for the Dealer Domain
Web Interface for the Supplier Domain
EJB Interface for the Dealer Domain
EJB Interface for the Manufacturing Domain
EJB Interface for the Supplier Domain
Notes / Tuning Information
-Xms6400m -Xmx6400m -Xns3360m -XXaggressive -Xgc:genpar -Xlargepages
-Xverbose:gcpause,opt,gcreport -showversion -showversion
-Doracle.jdbc.defaultRowPrefetch=200
-Dweblogic.MuxerClass=weblogic.socket.NIOSocketMuxer -Dweblogic.SocketReaders=1
-Dweblogic.management.discover=false
-Dweblogic.diagnostics.debug.DebugLogger.DISABLED=true
-Djavax.xml.parsers.DocumentBuilderFactory=com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderFactoryImpl

Emulator SW Config
JVM Product: Java HotSpot(TM) 64-Bit Server VM on Linux, version 1.6.0_21
Web Product: Oracle WebLogic Server Standard Edition Release 10.3.3
Hardware: Primary Driver, Satellite Drivers & Emulator HW
# of Instances: 1
Notes / Tuning Information
-Xmx22g -Xms22g -Xmn7g -Xss272k -XX:PermSize=95m -XX:MaxPermSize=512M
-XX:+UseLargePages -XX:+AggressiveOpts -XX:+DisableExplicitGC
-verbosegc -Xloggc:emugc.log -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:+PrintCommandLineFlags
-Dsun.net.inetaddr.ttl=1 -Dnetworkaddress.cache.ttl=1 -Dweblogic.SocketReaders=1
-Dweblogic.management.discover=false
-Djavax.xml.parsers.DocumentBuilderFactory=com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderFactoryImpl

Database SW Config
DB Product: Oracle Database Enterprise Edition Release 11.2.0.1.0
Hardware: Database Server HW
# of Instances: 1
Notes / Tuning Information
db_name                 = spec
compatible              = 11.2.0.1.0
control_files           = ?/dbs/cntrlspec
aq_tm_processes         = 0
db_4k_cache_size        = 12000M
db_8k_cache_size        = 20000M
db_block_checking       = FALSE
db_block_checksum       = FALSE
db_block_size           = 2048
db_cache_advice         = OFF
db_cache_size           = 45000M
db_file_multiblock_read_count = 128
db_files                = 256
db_keep_cache_size      = 10000M
db_writer_processes     = 8
dml_locks               = 2200
filesystemio_options    = DirectIO
log_buffer              = 67108864
log_checkpoint_interval = 0
log_checkpoints_to_alert= TRUE
open_cursors            = 3000
parallel_max_servers    = 100
pga_aggregate_target    = 0
processes               = 3000
query_rewrite_enabled   = FALSE
replication_dependency_tracking = FALSE
session_cached_cursors  = 3000
sessions                = 3000
shared_pool_size        = 6000M
statistics_level        = BASIC
timed_statistics        = FALSE
trace_enabled           = FALSE
transactions            = 15000
transactions_per_rollback_segment = 1
undo_management         = AUTO
undo_retention          = 300
undo_tablespace         = undo_ts

Driver SW Config Primary and Satellites
JVM Product: Java HotSpot(TM) 64-Bit Server VM on Linux, version 1.6.0_21
Hardware: Primary Driver, Satellite Drivers & Emulator HW
# of Instances: 4
Notes / Tuning Information
-server -Xms2048M -Xmx2048M -XX:+DisableExplicitGC -XX:+UseParallelGC
-Dsun.net.inetaddr.ttl=0 -Dnetworkaddress.cache.ttl=0

J2EE Application Server (SUT hardware)
Hardware Vendor: Cisco OS Vendor: Oracle Corporation
Model Name: Cisco UCS B230 M1 Blade Server OS Name: Oracle Enterprise Linux 5 Update 5 x86_64
Processor: Eight Core Intel(R) Xeon(R) X7560 Filesystem: ext3
MHz: 2262 Disks: 2x 64GB 3Gb/s SATA SSD
# of CPUs: 16 cores, 2 chips, 8 cores/chip, 2 threads/core (Hyper-Threading) Network Interface: Cisco UCS CNA M72KR-Q QLogic Converged Network Adapter with 2 x 10 Gigabit Ethernet and 2 x 4Gbps Fiber Channel
Memory (MB): 131072 Other Hardware: See notes
L1 Cache: 32KB(I)+32KB(D) on chip per core # of Systems: 2
L2 Cache: 256KB on chip per core H/W Available: Oct-2010
Other Cache: 24MB L3 on chip per chip OS Available: Apr-2010
Notes / Tuning Information
Added to /etc/sysctl.conf:
fs.file-max = 6815744
kernel.sem = 250 32000 100 142
kernel.shmall = 1073741824
kernel.shmmax = 4398046511104
net.core.netdev_max_backlog = 400000
net.core.optmem_max = 30000000
net.core.rmem_default = 30000000
net.core.rmem_max = 30000000
net.core.somaxconn = 40000
net.core.wmem_default = 30000000
net.core.wmem_max = 30000000
net.ipv4.ip_local_port_range = 1024 65000
net.ipv4.tcp_fin_timeout = 10
net.ipv4.tcp_max_syn_backlog = 30000
net.ipv4.tcp_max_tw_buckets = 2000000
net.ipv4.tcp_mem = 30000000 30000000 30000000
net.ipv4.tcp_rmem = 30000000 30000000 30000000
net.ipv4.tcp_timestamps = 0
net.ipv4.tcp_wmem = 30000000 30000000 30000000
vm.zone_reclaim_mode=1

For each of the NICs:
set arp_announce = 2
set arp_ignore = 1
set txqueuelen 240000
enable tso and gso
bind interrupt to a core corresponding to the server instance

Server instances were started using numactl, binding 4 instances per chip.
Stop iptables.
Stop irqbalance.
Configure 32000 hugepages.

Other hardware:
EMC CLARiiON CX4-240
VAULT PACK 450GB 15k 4G Drives
EMC 450 GB 15K 4G FC HDD

System was configured with 32 drives in 2 x (8+8) RAID1/0
for jms and server logs.
2 drives configured as spares.

Database Server HW (SUT hardware)
Hardware Vendor: Cisco OS Vendor: Oracle Corporation
Model Name: Cisco UCS B440 M1 Blade Server OS Name: Oracle Enterprise Linux 5 Update 5 x86_64
Processor: Eight Core Intel(R) Xeon(R) X7560 Filesystem: ext3
MHz: 2262 Disks: 4x 300GB SAS 10K RPM
# of CPUs: 32 cores, 4 chips, 8 cores/chip, 2 threads/core (Hyper-Threading) Network Interface: Cisco UCS M71KR-Q QLogic Converged Network Adapter with 2 x 10 Gigabit Ethernet and 2 x 4Gbps Fiber Channel
Memory (MB): 262144 Other Hardware: See notes
L1 Cache: 32KB(I)+32KB(D) on chip per core # of Systems: 1
L2 Cache: 256KB on chip per core H/W Available: Jun-2010
Other Cache: 24MB L3 on chip per chip OS Available: Apr-2010
Notes / Tuning Information
Added to /etc/sysctl.conf:
fs.aio-max-nr= 3145728
fs.file-max = 6815744
kernel.msgmni = 2878
kernel.sem = 250 32000 100 142
kernel.shmmax = 4398046511104
net.core.netdev_max_backlog = 400000
net.core.rmem_default = 30000000
net.core.rmem_max = 30000000
net.core.somaxconn = 15000
net.core.wmem_default = 30000000
net.core.wmem_max = 30000000
net.ipv4.ip_local_port_range = 1024 65000
net.ipv4.tcp_max_syn_backlog = 12000
net.ipv4.tcp_mem = 30000000 30000000 30000000
net.ipv4.tcp_rmem = 30000000 30000000 30000000
net.ipv4.tcp_timestamps = 0
net.ipv4.tcp_wmem = 30000000 30000000 30000000

For each of the NICs used:
set arp_announce = 2
set arp_ignore = 1
set txqueuelen 240000
enable tso and gso

Stop iptables.
Stop irqbalance.
Configure 55000 hugepages.

Lgwr nice priority was set to -20
named nice priority was set to -10
8 Oracle listener processes were started and bound each to 8 Hardware Threads.

Other hardware:
EMC CLARiiON CX-480
VAULT PACK 450GB 15k 4G Drives
EMC 450 GB 15K 4G FC HDD

40 drives in 4 x (5+5) RAID1/0 for data
20 drives in 2 x (5+5) RAID1/0 for logs
2 drives configured as spares

Primary Driver, Satellite Drivers & Emulator HW (non-SUT hardware)
Hardware Vendor: Cisco OS Vendor: Oracle Corporation
Model Name: Cisco UCS B200 M2 Blade Server OS Name: Oracle Enterprise Linux 5 Update 5 x86_64
Processor: Six Core Intel(R) Xeon(R) X5680 Filesystem: ext3
MHz: 3333 Disks: 2x 300GB SAS 10K RPM
# of CPUs: 12 cores, 2 chips, 6 cores/chip, 2 threads/core (Hyper-Threading) Network Interface: Cisco UCS M71KR-Q QLogic Converged Network Adapter with 2 x 10 Gigabit Ethernet
Memory (MB): 98304 Other Hardware: none
L1 Cache: 32KB(I)+32KB(D) on chip per core # of Systems: 4
L2 Cache: 256KB on chip per core H/W Available: Apr-2010
Other Cache: 12MB(I+D) L3 on chip per chip OS Available: Apr-2010
Notes / Tuning Information
Added to /etc/sysctl.conf:
fs.file-max = 6815744
kernel.sem = 250 32000 100 142
net.core.netdev_max_backlog = 400000
net.core.optmem_max = 30000000
net.core.rmem_default = 30000000
net.core.rmem_max = 30000000
net.core.somaxconn = 40000
net.core.wmem_default = 30000000
net.core.wmem_max = 30000000
net.ipv4.ip_local_port_range = 1024 65000
net.ipv4.tcp_fin_timeout = 10
net.ipv4.tcp_max_syn_backlog = 30000
net.ipv4.tcp_max_tw_buckets = 2000000
net.ipv4.tcp_mem = 30000000 30000000 30000000
net.ipv4.tcp_rmem = 30000000 30000000 30000000
net.ipv4.tcp_timestamps = 0
net.ipv4.tcp_wmem = 30000000 30000000 30000000

For each of the NICs used:
set arp_announce = 2
set arp_ignore = 1
set txqueuelen 240000
enable tso and gso

Stop iptables.

Benchmark Modifications
Schema Modifications:
  Tablespace sizes were increased to support the Injection rate. Table and index
  initrans were modified. The C_CUSTOMERINVENTORY, M_WORKORDER and O_ORDERLINE tables were
  horizontally partitioned. C_CUSTOMER, M_PARTS, M_BOM and M_INVENTORY were hash clustered.
  Table locks were disabled during the run for all the tables using ALTER TABLE DISABLE TABLE LOCK.
  Scripts to create the DB are included in the FDA.
Load Program Modifications:
  The load program was not modified.

Benchmark Configuration Information
Isolation Requirement Info:
  The following Beans were deployed in READ_COMMITTED mode as per the benchmark
  requirement:
  Mfg      - AssemblyEnt, BomEnt, ComponentEnt, LargeOrderEnt, PartEnt
  Orders   - ItemEnt
  Supplier - SupplierCompEnt, SupplierEnt
  
  All other beans were deployed using pessimistic locking mode such that all
  selects including finders are issued with a FOR UPDATE clause in select
  statement:
  Corp     - CustomerEnt, CustomerInventoryEnt
  Mfg      - InventoryEnt, WorkOrderEnt
  Orders   - OrderEnt, OrderLineEnt
  Supplier - POEnt, POLineEnt, SComponentEnt
  Util     - SequenceEnt
  
  The ItemEnt bean was cached for 20 minute intervals by setting
  the following in bea-orders.xml deployment descriptor:
  concurrency-strategy to ReadOnly
  read-timeout-seconds to 1200
Durability Requirement Info:
  To ensure database durability, RAID 10 was used for the data files and redo logs.
  
  The application server used RAID 1 to provide durability for
  server logs and JMS persistence filestores.
  
  The RAID controller on the database and the application server machines were
  configured with Write-Back cache policy supported with battery backup unit on
  the controller.
Storage Requirement Info:
  An 85 min run at Injection Rate of 6700 increased storage by 14.35 GB.
  Extrapolating for 24 hrs we need 243 GB. The system is configured with
  over 5TB of durable storage.
Argument Passing Semantics:
  Oracle WebLogic Server Standard Edition uses pass-by-value as required by
  the EJB specification.

Bill of Materials
Supplier  Description                                                SKU #              Qty
--------  --------------------------------------------------------   ----------------   ---
Application Server
Cisco    Cisco UCS B230 M1 Blade Server                              N20-B6730-1          2
Cisco    Eight Core Intel(R) Xeon(R) Processor X7560                 A01-X0200            4
         24MB L3 cache, 2.26GHz, 130W, Turbo
Cisco    Cisco 8GB DDR3-1333MHz RDIMM/PC3-10600/2x4GB Kit            A02-M308GB3-2       32
Cisco    Cisco UCS CNA M72KR-Q QLogic Converged Network Adapter      N20-AQ0102  2
Intel    Intel 64 gb SSD                                             N20-D064SSD          2
EMC      EMC CLARiiON CX4-240                                        CX4-240C             1
EMC      VAULT PACK 450GB 15k 4G Drives                              V-CX4-45015K         1
EMC      EMC 450 GB 15K 4G FC HDD                                    CX-4G15-450         34
         32 drives in 2 x (8+8) RAID1/0 for jms logs
         2 drives configured as spares


Database Server
Cisco    Cisco UCS B440 M1 Blade Server                                                   1
Cisco    Eight Core Intel(R) Xeon(R) Processor X7560                 A01-X0200            4
         24MB L3 cache, 2.26GHz, 130W, Turbo
Cisco    Cisco 16GB DDR3-1333MHz RDIMM/PC3-10600/2x8GB Kit           A02-M316GB3-2       32
Cisco    Cisco UCS CNA M72KR-Q QLogic Converged Network Adapter      N20-AQ0102           1
Cisco    300GB 6Gb SAS 10K RPM SFF HDD                               A03-D300GA2          4
EMC      EMC CLARiiON CX-480                                         CX4-480C             1
EMC      VAULT PACK 450GB 15k 4G Drives                              V-CX4-45015K         1
EMC      EMC 450 GB 15K 4G FC HDD                                    CX-4G15-450         62
         40 drives in 4 x (5+5) RAID1/0 for data
         20 drives in 2 x (5+5) RAID1/0 for logs
         2 drives configured as spares

 

Blade Server Enclosure

Cisco     UCS 5108 Blade Server Chassis                             N20-C6508             1
Cisco     UCS 2104XP Fabric Extender/4 external 10Gb ports          N20-I6584             2
Cisco                  UCS 6120XP 20-port Fabric Interconnect       N10-S6100             2
Cisco                  4-port 10 GE/4-port 4Gb FC/Expansion module  N10-E0440             2
Cisco     Cisco MDS 9134 Multilayer Fiber Switch                    DS-C9134AP-K9         1

Oracle    Oracle Enterprise Linux Basic Support for 3 years                               3
Oracle    Oracle Database 11g Enterprise Edition, Per Processor
          Unlimited Users for 3 years                                                    16*
Oracle    Partitioning, Per Processor, Unlimited Users for 3 years                       16*
Oracle    Oracle Premium Support for 3 years                                              2
Oracle    Oracle WebLogic Server Standard Edition Release 10.3,
          Per Processor for 3 years                                                      16*
Oracle    Oracle Premium Support for 3 years                                              2



(* 16 = 0.50 * 32)  Explanation:  For the purposes of counting
the number of processors which require licensing, an Intel multicore chip with "n"
cores shall be determined by multiplying "n" cores by a factor of 0.50).



Other Benchmark Information
  The xerces implementation version 2.8.1 from xml.apache.org was used as it comes
  packaged with Oracle Weblogic Server Standard Edition.
  
  Requests were directed to the Appserver instances using DNS round robin, with
  the database node acting as the DNS server.

General Notes
On the database, incremental checkpointing ensured that no buffer will remain
dirty (in the cache) for more than the allowed time.

The only errors in the driver error logs are the application errors generated by
this benchmark.


For questions about this result, please contact the submitter: Cisco Systems, Inc.
For other inquiries, please contact webmaster@spec.org

Copyright © 2004 Standard Performance Evaluation Corporation

First published at SPEC.org on 20-Sep-2010