SPEC CPU2017 Platform Settings for Nettrix Systems

Operating System Tuning Parameters

Used to set user limits of system-wide resources. Provides control over resources available to the shell and processes started by it. Some common ulimit commands may include:
Disabling Linux services
Certain Linux services may be disabled to minimize tasks that may consume CPU cycles.
Disabled through "service irqbalance stop". Depending on the workload involved, the irqbalance service reassigns various IRQ's to system CPUs. Though this service might help in some situations, disabling it can also help environments which need to minimize or eliminate latency to more quickly respond to events.
Performance Governors (Linux)
In-kernel CPU frequency governors are pre-configured power schemes for the CPU. The CPUfreq governors use P-states to change frequencies and lower power consumption.The dynamic governors can switch between CPU frequencies, based on CPU utilization to allow for power savings while not sacrificing performance. Other options beside a generic performance governor can be set, such as the perf-bias. On supported Intel processors, this option sets a register which allows the cpupower utility (or other software/firmware) to set a policy that controls the relative importance of performance versus energy savings to the processor. The range of valid numbers is 0-15, where 0 is maximum performance and 15 is maximum energy efficiency. The processor uses this information in model-specific ways when it must select trade-offs between performance and energy efficiency. This policy hint does not supersede Processor Performance states (P-states) or CPU Idle power states (C-states), but allows software to have influence where it would otherwise be unable to express a preference. On many Linux systems one can set the perf-bias for all CPUs through the cpupower utility with one of the following commands:
Tuning Kernel parameters
The following Linux Kernel parameters were tuned to better optimize performance of some areas of the system:
The tuned-adm tool is a commandline interface for switching between different tuning profiles available to the tuned tuning daeomn available in supported Linux distros. The default configuration file is located in /etc/tuned.conf and the supported profiles can be found in /etc/tune-profiles. Some profiles that may be available by default include: balanced, cpu-partitioning, desktop-powersave, desktop, latency-performance, network-latency, network-throughput, powersave, throughput-performance, virtual-guest, and virtual-host. To set a profile, one can issue the command "tuned-adm profile (profile_name)". Here are details about some frequently used profiles.
Transparent Huge Pages (THP)
THP is an abstraction layer that automates most aspects of creating, managing, and using huge pages. THP is designed to hide much of the complexity in using huge pages from system administrators and developers, as normal huge pages must be assigned at boot time, can be difficult to manage manually, and often require significant changes to code in order to be used effectively. Transparent Hugepages increase the memory page size from 4 kilobytes to 2 megabytes. Transparent Hugepages provide significant performance advantages on systems with highly contended resources and large memory workloads. If memory utilization is too high or memory is badly fragmented which prevents hugepages being allocated, the kernel will assign smaller 4k pages instead. Most recent Linux OS releases have THP enabled by default.
Linux Huge Page settings
If you need finer control and manually set the Huge Pages you can follow the below steps: For further information about huge pages may be found in your Linux documentation file: /usr/src/linux/Documentation/vm/hugetlbpage.txt

Firmware / BIOS / Microcode Settings

Application Performance Profile:
Application Performance Profile is designed for customers who need an easy way to optimize BIOS settings according to their application scenarios. The option could be set as "Disabled", "Computing Throughput Mode", "Computing Latency Mode", "Memory Bandwidth Mode", "Energy Efficient Mode", "Java Application Mode", and "High Reliability Mode". Default = "High Reliability Mode".
Autonomous Core C-State:
"Enabled" this option helps to boost the frequency of busy cores on which workloads are running and save the power consumption of idle cores on which no job is running. While "Disabled" this option helps to set the frequency of all cores at a same value.
C1 Enhanced Mode:
Enabling C1E (C1 enhanced) state can save power by halting CPU cores that are idle. Default is "Enable".
Turbo Mode:
Enabling turbo mode can boost the overall CPU performance when all CPU cores are not being fully utilized. A CPU core can run above its rated frequency for a short perios of time when it is in turbo mode. Default is "Enable".
Enabling Hyper-Threading let operating system addresses two virtual or logical cores for a physical presented core. Workloads can be shared between virtual or logical cores when possible. The main function of hyper-threading is to increase the number of independent instructions in the pipeline for using the processor resources more efficiently. Default is "Enabled".
Execute Disable Bit:
The execute disable bit allows memory to be marked as executable or non-executable when used with a supporting operating system. This can improve system security by configuring the processor to raise an error to the operating system when code attempts to run in non-executable memory. Default is Enable.
Hardware P-states:
This setting allows the user to select between OS and hardware-controlled P-states. Selecting Native Mode allows the OS to choose a P-state. Selecting Out of Band Mode allows the hardware to autonomously choose a P-state without OS guidance. Selecting Native Mode with No Legacy Support functions as Native Mode with no support for older hardware. Default is "Disable".
Energy Efficient Turbo:
When energy efficient turbo is enabled, the CPU's optimal turbo frequency will be tuned dynamically based on CPU utilization. Default is Enable.
MONITOR/MWAIT instructions are used to engage core C-states. Default is "Disabled".
Sub-NUMA Cluster (SNC):
SNC breaks up the last level cache (LLC) into disjoint clusters based on address range, with each cluster bound to a subset of the memory controllers in the system. SNC improves average latency to the LLC and memory. SNC is a replacement for the cluster on die (COD) feature found in previous processor families. For a multi-socketed system, all SNC clusters are mapped to unique NUMA domains. (See also IMC interleaving.) Values for this BIOS option can be: Default is "Disabled".
IMC Interleaving:
This BIOS option controls the interleaving between the Integrated Memory Controllers (IMCs). If 2 iMCs are 2-way interleaved, the channel population behind both iMCs must be identical. For iMCs in 1-way interleave, there are no requirements for matching across iMCs. Default is "Auto".
XPT Prefetch
XPT prefetch is a mechanism that enables a read request that is being sent to the last level cache to speculatively issue a copy of that read to the memory controller prefetching. This can be one of the following:
KTI Prefetch
KTI prefetch is a mechanism to get the memory read started early on a DDR bus. This can be one of the following: The default setting is "Disabled".
UPI Prefetcher
UPI prefetch is a mechanism to get the memroy read started early on DDR bus. The UPI receive path will spawn a memory read to the memory controller prefetcher. Default is Enabled.
Patrol Scrub:
Patrol Scrub is a memory RAS feature which runs a background memory scrub against all DIMMs. Can negatively impact performance. This option allows for correction of soft memory errors. Over the length of system runtime, the risk of producing multi-bit and uncorrected errors is reduced with this option. Values for this BIOS setting can be: Default is Enabled.
DCU Streamer Prefetcher:
DCU (Level 1 Data Cache) streamer prefetcher is an L1 data cache prefetcher. Lightly threaded applications and some benchmarks can benefit from having the DCU streamer prefetcher enabled. Default setting is Enable.
Stale AtoS
The in-memory directory has three states: invalid (I), snoopAll (A), and shared (S). Invalid (I) state means the data is clean and does not exist in any other socket`s cache. The snoopAll (A) state means the data may exist in another socket in exclusive or modified state. Shared (S) state means the data is clean and may be shared across one or more socket`s caches. When doing a read to memory, if the directory line is in the A state we must snoop all the other sockets because another socket may have the line in modified state. If this is the case, the snoop will return the modified data. However, it may be the case that a line is read in A state and all the snoops come back a miss. This can happen if another socket read the line earlier and then silently dropped it from its cache without modifying it. Values for this BIOS option can be: The default setting is Disabled. Stale A to S may be beneficial in a workload where there are many cross-socket reads.
LLC Dead Line Allocation
In some Intel CPU caching schemes, mid-level cache (MLC) evictions are filled into the last level cache (LLC). If a line is evicted from the MLC to the LLC, the core can flag the evicted MLC lines as "dead." This means that the lines are not likely to be read again. This option allows dead lines to be dropped and never fill the LLC if the option is disabled. Values for this BIOS option can be: The default setting is "Enabled".
Intel Virtualization Technology:
Intel Virtualization Technology allows a platform to run multiple operating systems and applications in independent partitions, so that one computer system can function as multiple virtual system. Default is disable.
Hardware Prefethcer:
When this option is enable, a dedicated hardware mechanism in the processor is supported to watch the stream of instructions or data being requested by the executing program, recognize the next few elements that the program might need based on this stream and prefetch into the processor's cache. The program with good instruction and data locality will benefit from this feature when this option is enable. Default is enable.
Trusted Execution Technology:
Enable Intel Trusted Execution Technology (Intel TXT). Default is disable.
Page Policy:
Adaptive Open Page Policy can improve performance for applications with a highly localized memory access pattern; Closed Page Policy can benifit applications that access memory more randomly. The default is "Auto".
Cooling Policy
The feature to configure "Cooling Policy" option is provided on BMC webpage. This option provides 4 choices: "Balance Mode", "Performance Mode", "Silent Mode" and "Manual Mode" and default is "Balance Mode".