SPEC SFS(R)2014_swbuild Result Oracle Corporation : Oracle ZFS Storage ZS3-2 SPEC SFS2014_swbuild = 240 Builds (Overall Response Time = 1.71 msec) =============================================================================== Performance =========== Business Average Metric Latency Builds Builds (Builds) (msec) Ops/Sec MB/Sec ------------ ------------ ------------ ------------ 24 0.6 12001 142 48 0.6 24003 284 72 0.8 36004 426 96 0.8 48005 568 120 0.8 60007 709 144 1.0 72008 851 168 1.3 84010 993 192 2.0 96011 1136 216 5.1 108011 1278 240 8.7 116020 1373 =============================================================================== Product and Test Information ============================ +---------------------------------------------------------------+ | Oracle ZFS Storage ZS3-2 | +---------------------------------------------------------------+ Tested by Oracle Corporation Hardware Available August 2016 Software Available August 2016 Date Tested August 2016 License Number 6 Licensee Locations Broomfield, CO USA The Oracle ZFS Storage ZS3-2 is a mid-range high-performance storage system that offers enterprise-class NAS and SAN capabilities with industry-leading Oracle Database integration, in a cost-effective high-availability configuration. The Oracle ZFS Storage ZS3-2 offers simplified set up and management combined with industry-leading storage analytics and a performance-optimized platform that uses specialized Read and Write Flash caching devices. The Oracle ZFS Storage ZS3-2 can scale to 512 GB Memory, 32 CPU cores, and 1.5 PB capacity, with up to 12.8 TB of Flash Cache in a high-availability configuration. Oracle ZFS Storage Appliances deliver additional economic value bundled data services such as file- and block-level protocols including connectivity over InfiniBand, Compression, Deduplication, Thin provisioning, DTrace Analytics, Virus Scan, Snapshots, Triple Mirror, Triple Parity RAID, Phone-home, NDMP, Clustering, etc. Solution Under Test Bill of Materials ===================================== Item No Qty Type Vendor Model/Name Description ---- ---- ---------- ---------- ---------- ----------------------------------- 1 2 Storage Oracle Oracle ZFS Oracle ZFS Storage ZS3-2: Controller Storage ZS controllers part #7103829 3-2:contro includes 1 - SAS2 PCIE 16 port HBA ller 2 32 Controller Oracle Memory 16 GB DDR3-1600 registered DIMM Memory DIMM (for factory installation) part #7102984 3 6 10 Oracle Sun PCI-E Sun Dual 10GbE SFP+ PCIe 2.0 Low Gigabit Dual 10GbE Profile adapter Ethernet Fiber incorporating Intel 82599 10 Adapter Gigabit Ethernet controller and supporting pluggable SFP+ Transceivers. ROHS-5. ATO option (2 installed in Client Oracle X5-2) part #7051223 4 12 Short Wave Oracle 10Gbps Dual rate transceiver: SFP+ SR. Pluggable Short Wave Support 1 Gb/sec and 10 Transceive Pluggable Gb/sec dual rate (for factory r Transceive installation both ZS3-2 r (SFP+) storage server and X5-2 client) part #2129A 5 6 Storage Oracle Oracle Oracle Storage Drive Enclosure Drive Storage DE2-24P: base chassis (for Enclosure Drive factory installation) part #7103910 Enclosure Note: 4 of the DE2-24P DE2-24P enclosures are populated with 24 disk drives and 2 of the DE2-24p enclosures have 20 disks drives and 4 log devices 6 136 Disk Oracle Disk 300 GB 10000 rpm 2.5 inch SAS-2 HDD Drives Drives (for factory 300GB 10K installation) part #7103911 RPM 2.5 inch SAS-2 HDD 7 8 SSD Drives Oracle SAS-2 73GB 2.5 inch SAS-2 SSD write flash 2.5-inch accelerator with evo bracket SSD Write (for factory installation) part Flash Acce #7048983 lerator 8 2 HBA Oracle SAS-2 PCIE SAS-2 back end HBA part #7103790 6Gbs (for factory installation) 16-port HBA 9 12 Cables Oracle SAS-2 SAS-2 back end cables part #7104928 Cables 10 1 Switch Arista Arista 10Gb Ethernet Optical Switch *Note: 7124SX the switch currently in 10Gb not available for order from the Switch manufacturer but is available from other vendors. Factory support for the Arista switch will continue thru 2017. 11 1 Client Oracle Oracle with factory installed memory of X5-2 128GB Configuration Diagrams ====================== 1) sfs2014-20160825-00014.config1.jpg (see SPEC SFS2014 results webpage) Component Software ================== Item Name and No Component Type Version Description ---- ------------ ------------ ------------ ----------------------------------- 1 Oracle ZFS Storage 8.6 Oracle Storage ZFS OS for storage Storage Controllers controllers 2 Oracle Client Node 11.3 Oracle Operating System on client Solaris OS node Solaris 11.3 Hardware Configuration and Tuning - Physical ============================================ +----------------------------------------------------------------------+ | Oracle ZS3-2 storage controllers | +----------------------------------------------------------------------+ Parameter Name Value Description --------------- --------------- ---------------------------------------- MTU 9000 Jumbo Frames setup +----------------------------------------------------------------------+ | Oracle X5-2 Client | +----------------------------------------------------------------------+ Parameter Name Value Description --------------- --------------- ---------------------------------------- MTU 9000 Jumbo Frames setup Hardware Configuration and Tuning Notes --------------------------------------- Oracle ZS3-2 storage controllers 10Gb ethernet ports are set up to MTU of 9000 jumbo frames. Oracle X5-2 client is 10Gb ethernet ports are set up to MTU of 9000 jumbo frames. Software Configuration and Tuning - Physical ============================================ +----------------------------------------------------------------------+ | Oracle X5-2 Client | +----------------------------------------------------------------------+ Parameter Name Value Description --------------- --------------- ---------------------------------------- vers 3 NFS mount option to set NFS mount version 3 rsize,wsize 16384 NFS mount option for data block size forcedirectio forcedirectio NFS mount option for directio to storage server max_buf 16777216 TCP max send receive buffer size send_buf 4194304 TCP send buffer size recv_buf 4194304 TCP receive buffer size +----------------------------------------------------------------------+ | Oracle ZS3-2 Controllers | +----------------------------------------------------------------------+ Parameter Name Value Description --------------- --------------- ---------------------------------------- Database record 16KB Record size for each filesystem of both size Oracle ZS3-2 Controllers Maximum # of 1000 Sets up maximum number of NFS server server threads threads used by Oracle ZS3-2 Controllers Software Configuration and Tuning Notes --------------------------------------- Tune the communications between Oracle X5-2 client and the Oracle ZS3-2 controllers over the 10Gb ethernet by optimizing amount of data transfer and minimum overhead. This includes setting the Oracle X5-2 clients mounts of the Oracle ZS3-2 files systems to use forcedirectio, read and write sizes to 16384, along with increasing the Oracle X5-2 client send and receive buffers sizes to 4194304. Service SLA Notes ----------------- None Storage and Filesystems ======================= Item Stable No Description Data Protection Storage Qty ---- ------------------------------------- ------------------ -------- ----- 1 300GB SAS 10K RPM Disk RAID-10 Yes 136 Drives 2 73GB SAS-2 SSD Write Flash None Yes 8 Accelerator Used for the ZFS Intent Log (ZIL). 3 500GB SATA 7.2K RPM Disk Mirrored no 4 Drives Oracle ZS3-2 Controllers OS disk drives 4 500GB SATA 7.2K RPM Disk Mirrored no 2 Drives Oracle X5-2 Client OS Number of Filesystems 240 Total Capacity 17.36TiB Filesystem Type ZFS Filesystem Creation Notes ------------------------- Both controllers are set up with 8 storage pools total (4 storage pools per Oracle ZS3-2 controller). Each of the controller's storage pools are configured with 16 disk drives, 1 write flash accelerator (1 log device) and 1 spare disk drive. When configuring the storage pool via the administrative html interface of each ZS3-2 storage controller, at the start you will be ask to select the number of disk drives and log devices to use per tray. Select 6 drives on 2 of the trays and 5 drives on the 1 tray that has the log devices and select 1 log device. In total the number of disk drives selected is 17 and 1 log device. The system will configure the spare into the storage pool after you select the data profile. Then the storage pools are set up to mirror the data (RAID-10) across all 16 drives (Note: When configuring storage pools on the Oracle ZS3-2 controllers this is a data profile of Mirrored). The write flash accelerator in each storage pool is used for the ZFS Intent Log (ZIL). Do these step for each storage pool until there are 4 per ZS3-2 storage controller have been configured. After the storage pools are created. Each of the storage pools are configured with 30 ZFS filesystems each. Since each controller is configured with 4 storage pools and each storage pool contains 30 ZFS filesystems, in total each controller has 120 ZFS filesystems. Both of the controllers together in total have 240 ZFS filesystems. There are 2 internal mirrored system disk drives per Oracle ZS3-2 controller and are used only for the controllers core operating system. These drives are not used for data cache or storing user data. Storage and Filesystem Notes ---------------------------- All filesystems on both Oracle ZS3-2 controllers are create with setting of the Database Record Size of 16KB. Which is a common practice for storage solutions with the Oracle ZS3-2 storage controllers. Transport Configuration - Physical ================================== Item Number of No Transport Type Ports Used Notes ---- --------------- ---------- ----------------------------------------------- 1 10 Gigabit 12 Each Oracle ZS3-2 controller has 2 Dual port 10 Ethernet Gigabit Ethernet cards. Oracle X5-2 client also has 2 Dual port 10 Gigabit Ethernet cards. Transport Configuration Notes ----------------------------- Each Oracle ZS3-2 uses 2 active 10Gb ports and 2 standby ports. Total Oracle ZS3-2 controllers use 4 ports active and 4 ports on standby. All ports active and standby are setup with the MTU size is set to 9000 on each of the 10 Gb ports. The single Oracle X5-2 client uses both ports of the 2 dual port 10 Gb ethernet cards. The 4 ports used are set to MTU of 9000. Switches - Physical =================== Total Used Item Port Port No Switch Name Switch Type Count Count Notes ---- -------------------- --------------- ------ ----- ------------------------ 1 Arista 7124 10Gb 10Gb Ethernet 24 12 All ports set up to do Switch Jumbo Frames on the Arista 7124SX 10Gb Switch Processing Elements - Physical ============================== Item No Qty Type Location Description Processing Function ---- ---- -------- -------------- ------------------------- ------------------- 1 4 CPU Oracle ZS3-2 2.1GHz Intel Xeon E5-2658 NFS, ZFS, TCP/IP, Storage Server RAID/Storage Drivers 2 2 CPU Oracle X5-2 2.6GHz Intel Xeon E5-2660 NFS Client Solaris Client OS Processing Element Notes ------------------------ Each Oracle ZFS Storage ZS3-2 controller contains 2 physical processors, each with 8 processing cores. Oracle X5-2 client contains 2 physical processors, each with 10 processing cores. Oracle X5-2 client processors SMT is set to the default settings of Solaris 11.3. Memory - Physical ================= Size in Number of Description GiB Instances Nonvolatile Total GiB ------------------------- ---------- ---------- ------------ ------------ Memory for Oracle ZFS 256 2 V 512 ZS3-2 Storage Controllers Memory for Oracle X5-2 128 1 V 128 Client Grand Total Memory Gibibytes 640 Memory Notes ------------ The Oracle ZFS Storage ZS3-2 controllers' main memory is used for the Adaptive Replacement Cache (ARC), the data cache, and operating system memory. Oracle X5-2 client memory is not used for storage or cache of the Oracle ZFS ZS3-2 storage controllers, just for the client use. Stable Storage ============== The Stable Storage requirement is guaranteed by the ZFS Intent Log (ZIL) which logs writes and other filesystem changing transactions to either a write flash accelerator or a disk drive. Writes and other filesystem changing transactions are not acknowledged until the data is written to stable storage. Since this is an active-active cluster high availability system, in the event of a controller failing or power loss, the other active controller can take over for the failed controller. Since the write flash accelerators or disk drives are located in the disk shelves and can be accessed via the 4 backend SAS channels from both controllers, the remaining active controller can complete any outstanding transactions using the ZIL. In the event of power loss to both controllers, the ZIL is used after power is restored to reinstate any writes and other filesystem changes. Solution Under Test Configuration Notes ======================================= The system under test is a Oracle ZFS Storage ZS3-2 storage controllers are setup in an active-active cluster configuration with failover capabilities. Other Solution Notes ==================== None Dataflow ======== Please reference the configuration diagram. A single client is used for the as the benchmark load generator. The client Oracle X5-2 mounted all of the Oracle ZS3-2 storage controllers filesystems via NFSv3. The Oracle ZS3-2 storage controllers filesystems are numbered 1-240. ZS3-2a has filesystems 1-120 and ZS3-2b has filesystems 121-240 all total 240 filesystems between the 2 ZS3-2 controllers. Oracle ZS3-2 storage controller active ports, 1,2,3 and 4 are assigned separate subnets. Mount the filesystem 1 of the ZS3-2a using 10Gb ethernet port 1 then the filesystem 121 is mounted of the ZS3-2b using 10Gb ethernet port 3, then filesystem 31 is mounted of the ZS3-2a using 10Gb ethernet port 2 and then filesystem 151 is mounted of the ZS3-2b using 10Gb ethernet port 4. The next set would be filesystem 61 of the ZS3-2a using 10Gb ethernet port 1 then the filesystem 181 is mounted of the ZS3-2b using 10Gb ethernet port 3, then filesystem 91 is mounted of the ZS3-2a using 10Gb ethernet port 2 and then filesystem 211 is mounted of the ZS3-2b, using 10Gb ethernet port 4, next filesystems 2, 122, 32,and 152, and so on, until all 240 filesystems are mounted. So in effect, this will round-robin mount the filesystems to spread the load across the storage pools and the Oracle ZS3-2 controllers. Other Notes =========== Oracle is a registered trademark of Oracle Corporation. Intel and Xeon are registered trademarks of the Intel Corporation in the U.S. and/or other countries. Arista is a registered trademark for Arista. Other Report Notes ================== None =============================================================================== Generated on Wed Mar 13 16:51:42 2019 by SpecReport Copyright (C) 2016-2019 Standard Performance Evaluation Corporation