CPU2017 Flag Description
Lenovo Global Technology Wentian WR5220 G3 (1.90 GHz, Intel Xeon Platinum 8490H)

Copyright © 2016 Intel Corporation. All Rights Reserved.

Implicitly Included Flags

This section contains descriptions of flags that were included implicitly by other flags, but which do not have a permanent home at SPEC.

• • -O2
  • 

  • Enable optimizations for speed. This is the generally recommended optimization level. This option also enables:
    - Inlining of intrinsics
    - Intra-file interprocedural optimizations, which include:
    - inlining
    - constant propagation
    - forward substitution
    - routine attribute propagation
    - variable address-taken analysis
    - dead static function elimination
    - removal of unreferenced variables
    - The following capabilities for performance gain:
    - constant propagation
    - copy propagation
    - dead-code elimination
    - global register allocation
    - global instruction scheduling and control speculation
    - loop unrolling
    - optimized code selection
    - partial redundancy elimination
    - strength reduction/induction variable simplification
    - variable renaming
    - exception handling optimizations
    - tail recursions
    - peephole optimizations
    - structure assignment lowering and optimizations
    - dead store elimination
    

  • Includes:
    • -O1
      • -funroll-loops
      • -fno-builtin
      • -mno-ieee-fp
      • -fomit-framepointer
      • -ffunction-sections
      • -ftz
• • -O1
  • 

  • Enable optimizations for speed and disables some optimizations that increase code size and affect speed.
    To limit code size, this option:

    • Enable global optimization; this includes data-flow analysis, code motion, strength reduction and test replacement, split-lifetime analysis, and instruction scheduling.
    • Disables intrinsic recognition and intrinsics inlining.

    The O1 option may improve performance for applications with very large code size, many branches, and execution time not dominated by code within loops.

    -O1 sets the following options:
    -funroll-loops0, -fno-builtin, -mno-ieee-fp, -fomit-framepointer, -ffunction-sections, -ftz
  • Includes:
    • -funroll-loops
    • -fno-builtin
    • -mno-ieee-fp
    • -fomit-framepointer
    • -ffunction-sections
    • -ftz
• • -funroll-loops
  • 

  • Tells the compiler the maximum number of times to unroll loops. For example -funroll-loops0 would disable unrolling of loops.

• • -fno-builtin
  • 

  • -fno-builtin disables inline expansion for all intrinsic functions.

• • -mno-ieee-fp
  • 

  • This option trades off floating-point precision for speed by removing the restriction to conform to the IEEE standard.

• • -fomit-framepointer
  • 

  • EBP is used as a general-purpose register in optimizations.

• • -ffunction-sections
  • 

  • Places each function in its own COMDAT section.

• • -ftz
  • 

  • Flushes denormal results to zero.

Commands and Options Used to Submit Benchmark Runs

submit= MYMASK=`printf '0x%x' $((1<<$SPECCOPYNUM))`; /usr/bin/taskset $MYMASK $command

When running multiple copies of benchmarks, the SPEC config file feature submit is used to cause individual jobs to be bound to specific processors. This specific submit command, using taskset, is used for Linux64 systems without numactl.
Here is a brief guide to understanding the specific command which will be found in the config file:

• /usr/bin/taskset [options] [mask] [pid | command [arg] ... ]:
  taskset is used to set or retreive the CPU affinity of a running process given its PID or to launch a new COMMAND with a given CPU affinity. The CPU affinity is represented as a bitmask, with the lowest order bit corresponding to the first logical CPU and highest order bit corresponding to the last logical CPU. When the taskset returns, it is guaranteed that the given program has been scheduled to a specific, legal CPU, as defined by the mask setting.
• [mask]: The bitmask (in hexadecimal) corresponding to a specific SPECCOPYNUM. The specific example above, computes this mask value in the variable $MYMASK. The value of this mask for the first copy of a rate run will be 0x00000001, for the second copy of the rate will be 0x00000002 etc. Thus, the first copy of the rate run will have a CPU affinity of CPU0, the second copy will have the affinity CPU1 etc.
• $command: Program to be started, in this case, the benchmark instance to be started.

submit= numactl --localalloc --physcpubind=$SPECCOPYNUM $command

When running multiple copies of benchmarks, the SPEC config file feature submit is used to cause individual jobs to be bound to specific processors. This specific submit command is used for Linux64 systems with support for numactl.
Here is a brief guide to understanding the specific command which will be found in the config file:

• syntax: numactl [--interleave=nodes] [--preferred=node] [--physcpubind=cpus] [--cpunodebind=nodes] [--membind=nodes] [--localalloc] command args ...
• numactl runs processes with a specific NUMA scheduling or memory placement policy. The policy is set for a command and inherited by all of its children.
• "--localalloc" instructs numactl to keep a process memory on the local node while "-m" specifies which node(s) to place a process memory.
• "--physcpubind" specifies which core(s) to bind the process. In this case, copy 0 is bound to processor 0 etc.
• For full details on using numactl, please refer to your Linux documentation, 'man numactl'

Shell, Environment, and Other Software Settings

numactl --interleave=all "runspec command"

Launching a process with numactl --interleave=all sets the memory interleave policy so that memory will be allocated using round robin on nodes. When memory cannot be allocated on the current interleave target fall back to other nodes.

KMP_STACKSIZE

Specify stack size to be allocated for each thread.

KMP_AFFINITY

Syntax: KMP_AFFINITY=[<modifier>,...]<type>[,<permute>][,<offset>]
The value for the environment variable KMP_AFFINITY affects how the threads from an auto-parallelized program are scheduled across processors.
It applies to binaries built with -qopenmp and -parallel (Linux and Mac OS X) or /Qopenmp and /Qparallel (Windows).
modifier:
    granularity=fine Causes each OpenMP thread to be bound to a single thread context.
type:
    compact Specifying compact assigns the OpenMP thread <n>+1 to a free thread context as close as possible to the thread context where the <n> OpenMP thread was placed.
    scatter Specifying scatter distributes the threads as evenly as possible across the entire system.
permute: The permute specifier is an integer value controls which levels are most significant when sorting the machine topology map. A value for permute forces the mappings to make the specified number of most significant levels of the sort the least significant, and it inverts the order of significance.
offset: The offset specifier indicates the starting position for thread assignment.

Please see the Thread Affinity Interface article in the Intel Composer XE Documentation for more details.

Example: KMP_AFFINITY=granularity=fine,scatter
Specifying granularity=fine selects the finest granularity level and causes each OpenMP or auto-par thread to be bound to a single thread context.
This ensures that there is only one thread per core on cores supporting HyperThreading Technology
Specifying scatter distributes the threads as evenly as possible across the entire system.
Hence a combination of these two options, will spread the threads evenly across sockets, with one thread per physical core.

Example: KMP_AFFINITY=compact,1,0
Specifying compact will assign the n+1 thread to a free thread context as close as possible to thread n.
A default granularity=core is implied if no granularity is explicitly specified.
Specifying 1,0 sets permute and offset values of the thread assignment.
With a permute value of 1, thread n+1 is assigned to a consecutive core. With an offset of 0, the process's first thread 0 will be assigned to thread 0.
The same behavior is exhibited in a multisocket system.

OMP_NUM_THREADS

Sets the maximum number of threads to use for OpenMP* parallel regions if no other value is specified in the application. This environment variable applies to both -qopenmp and -parallel (Linux and Mac OS X) or /Qopenmp and /Qparallel (Windows). Example syntax on a Linux system with 8 cores: export OMP_NUM_THREADS=8

Set stack size to unlimited

The command "ulimit -s unlimited" is used to set the stack size limit to unlimited.

Free the file system page cache

The command "echo 3> /proc/sys/vm/drop_caches" is used to free up the filesystem page cache as well as reclaimable slab objects like dentries and inodes.

Red Hat Specific features

Transparent Huge Pages

On RedHat EL 6 and later, Transparent Hugepages increase the memory page size from 4 kilobytes to 2 megabytes. Transparent Hugepages provide significant performance advantages on systems with highly contended resources and large memory workloads. If memory utilization is too high or memory is badly fragmented which prevents hugepages being allocated, the kernel will assign smaller 4k pages instead.
Hugepages are used by default unless the /sys/kernel/mm/redhat_transparent_hugepage/enabled field is changed from its RedHat EL6 default of 'always'.

Operating System Tuning Parameters

sched_cfs_bandwidth_slice_us

This OS setting controls the amount of run-time(bandwidth) transferred to a run queue from the task's control group bandwidth pool. Small values allow the global bandwidth to be shared in a fine-grained manner among tasks, larger values reduce transfer overhead. The default value is 5000 (ns).

sched_latency_ns

This OS setting configures targeted preemption latency for CPU bound tasks. The default value is 24000000 (ns).

sched_migration_cost_ns

Amount of time after the last execution that a task is considered to be "cache hot" in migration decisions. A "hot" task is less likely to be migrated to another CPU, so increasing this variable reduces task migrations. The default value is 500000 (ns).

sched_min_granularity_ns

This OS setting controls the minimal preemption granularity for CPU bound tasks. As the number of runnable tasks increases, CFS(Complete Fair Scheduler), the scheduler of the Linux kernel, decreases the timeslices of tasks. If the number of runnable tasks exceeds sched_latency_ns/sched_min_granularity_ns, the timeslice becomes number_of_running_tasks * sched_min_granularity_ns. The default value is 8000000 (ns).

sched_wakeup_granularity_ns

This OS setting controls the wake-up preemption granularity. Increasing this variable reduces wake-up preemption, reducing disturbance of compute bound tasks. Lowering it improves wake-up latency and throughput for latency critical tasks, particularly when a short duty cycle load component must compete with CPU bound components. The default value is 10000000 (ns).

numa_balancing

This OS setting controls automatic NUMA balancing on memory mapping and process placement. NUMA balancing incurs overhead for no benefit on workloads that are already bound to NUMA nodes. Possible settings:

• 0: disables this feature
• 1: enables the feature (this is the default)

For more information see the numa_balancing entry in the Linux sysctl documentation.

Transparent Hugepages (THP)

THP is an abstraction layer that automates most aspects of creating, managing, and using huge pages. It is designed to hide much of the complexity in using huge pages from system administrators and developers. Huge pages increase the memory page size from 4 kilobytes to 2 megabytes. This provides significant performance advantages on systems with highly contended resources and large memory workloads. If memory utilization is too high or memory is badly fragmented which prevents hugepages being allocated, the kernel will assign smaller 4k pages instead. Most recent Linux OS releases have THP enabled by default.
THP usage is controlled by the sysfs setting /sys/kernel/mm/transparent_hugepage/enabled. Possible values:

• never: entirely disable THP usage.
• madvise: enable THP usage only inside regions marked MADV_HUGEPAGE using madvise(3).
• always: enable THP usage system-wide. This is the default.

THP creation is controlled by the sysfs setting /sys/kernel/mm/transparent_hugepage/defrag. Possible values:

• never: if no THP are available to satisfy a request, do not attempt to make any.
• defer: an allocation requesting THP when none are available get normal pages while requesting THP creation in the background.
• defer+madvise: acts like "always", but only for allocations in regions marked MADV_HUGEPAGE using madvise(3); for all other regions it's like "defer".
• madvise: acts like "always", but only for allocations in regions marked MADV_HUGEPAGE using madvise(3). This is the default.
• always: an allocation requesting THP when none are available will stall until some are made.

An application that "always" requests THP often can benefit from waiting for an allocation until those huge pages can be assembled.
For more information see the Linux transparent hugepage documentation.

cpupower

The OS 'cpupower' utility is used to change CPU power governors settings. Available settings are:

• Performance: Run the CPU at the maximum frequency.
• powersave: Run the CPU at the minimum frequency.
• ondemand: Scales the frequency dynamically according to current load.

tuned-adm

The tuned-adm tool is a commandline interface for switching between different tuning profiles available to the tuned tuning daemon available in supported Linux distros. The default configuration file is located in /etc/tuned.conf and the supported profiles can be found in /etc/tune-profiles. Some profiles that may be available by default include: default, desktop-powersave, server-powersave, laptop-ac-powersave, laptop-battery-powersave, spindown-disk, throughput-performance, latency-performance, enterprise-storage. To set a profile, one can issue the command "tuned-adm profile (profile_name)". Here are details about relevant profiles:

• throughput-performance: Server profile for typical throughput tuning. This profile disables tuned and ktune power saving features, enables sysctl settings that may improve disk and network IO throughput performance, switches to the deadline scheduler, and sets the CPU governor to performance.
• latency-performance: Server profile for typical latency tuning. This profile disables tuned and ktune power saving features, enables the deadline IO scheduler, and sets the CPU governor to performance.
• enterprise-storage: Server profile to high disk throughput tuning. This profile disables tuned and ktune power saving features, enables the deadline IO scheduler, enables hugepages and disables disk barriers, increases disk readahead values, and sets the CPU governor to performance

dirty_background_ratio

Set through "echo 40 > /proc/sys/vm/dirty_background_ratio". This setting can help Linux disk caching and performance by setting the percentage of system memory that can be filled with dirty pages.

dirty_ratio

Set through "echo 8 > /proc/sys/vm/dirty_ratio". This setting is the absolute maximum amount of system memory that can be filled with dirty pages before everything must get committed to disk.

ksm/sleep_millisecs

Set through "echo 200 > /sys/kernel/mm/ksm/sleep_millisecs". This setting controls how many milliseconds the ksmd (KSM daemon) should sleep before the next scan.

swappiness

The swappiness value can range from 1 to 100. A value of 100 will cause the kernel to swap out inactive processes frequently in favor of file system performance, resulting in large disk cache sizes. A value of 1 tells the kernel to only swap processes to disk if absolutely necessary. This can be set through a command like "echo 1 > /proc/sys/vm/swappiness"

Zone Reclaim Mode

Zone reclaim allows the reclaiming of pages from a zone if the number of free pages falls below a watermark even if other zones still have enough pages available. Reclaiming a page can be more beneficial than taking the performance penalties that are associated with allocating a page on a remote zone, especially for NUMA machines. To tell the kernel to free local node memory rather than grabbing free memory from remote nodes, use a command like "echo 1 > /proc/sys/vm/zone_reclaim_mode"

Free the file system page cache

The command "echo 3> /proc/sys/vm/drop_caches" is used to free pagecache, dentries and inodes.

Firmware / BIOS / Microcode Settings

Power Mode (Default is Efficiency -Favor Performance)

The average customer doesn't know the best way to set each individual power/performance feature for their specific environment. Because of this, a menu option is provided that can help a customer optimize the system for things such as minimum power usage/acoustic levels, maximum efficiency, Energy Star optimization, or maximum performance.

• "Saving" mode strives to minimize the absolute power consumption of the system while it is operating. The tradeoff is that performance may be reduced in this mode depending on the application that is running.
• "Efficiency -Favor Power" mode maximizes the performance/watt efficiency with a bias towards power savings. It provides the best features for reducing power and increasing performance in applications where maximum bus speeds are not critical. "Efficiency -Favor Power" mode maintains backwards compatibility with systems that included the preset operating modes before Energy Star for servers was released.
• "Efficiency -Favor Performance" mode optimizes the performance/watt efficiency with a bias towards performance. It is the favored mode for Energy Star. Note that this mode is slightly different than "Efficiency -Favor Power" mode. In "Efficiency - Favor Performance" mode, no bus speeds are derated as they are in "Efficiency -Favor Power" mode. "Efficiency -Favor Performance" mode is the default mode.
• "Performance" mode will maximize the absolute performance of the system without regard for power. In this mode, power consumption is a don't care. Things like fan speed and heat output of the system may increase in addition to power consumption. Efficiency of the system may go down in this mode, but the absolute performance may increase depending on the workload that is running.
• A fifth setting, [Custom], may be selected after any of the other 4 presets allowing low-level settings, which otherwise are preset and unchangeable, to be individually modified

C1 Enhanced Mode:

Enabling C1E (C1 enhanced) state can save power by halting CPU cores that are idle. Default is Enabled.

Sub-NUMA Cluster (SNC):

Sub_NUMA Cluster (SNC) partitions the cores and last level cache (LLC) into clusters with each cluster bound to a set of memory controllers in the system, dividing each CPU package into 2 or 4 NUMA nodes. This can improve average latency to the last level cache and memory. Default is Disabled.

• Disabled:the processor socket is treated as one cluster. No partitioning occurs.
• Enable SNC2 (2-clusters):each socket is divided into two groups, each with its own cores, LLC, and memory controller. System address is divided into two regions, one for each SNC. Required memory configuration to be left/right symmetric.
• Enable SNC4 (4-clusters):each socket is divided into four groups, each with its own cores, LLC, and memory controller. System address is divided into four regions, one for each SNC. Required memory configuration to be fully symmetric (left/right and top/bottom).

Patrol Scrub:

Patrol scrub works in the background to proactively check DIMMs for memory errors. There are two options in this mode: Disabled, Enabled. When "Disabled" is selected, memory latency will be lower but with risk of lower memory reliability. When "Enabled" is selected, memory reliability will be improved. Default is Enabled.

DCU Streamer Prefetcher:

DCU (Level 1 Data Cache) streamer prefetcher is an L1 data cache prefetcher. When the DCU streamer prefetcher detects multiple loads from the same cache line done within a time limit, the DCU streamer prefetcher assumes the next line will be required. The next line is prefetched in to the L1 data cache from memory or L2. Lightly threaded applications and some benchmarks can benefit from having the DCU streamer prefetcher enabled. Default setting is Enabled.

Intel Virtualization Technology:

Intel Virtualization Technology allows a platform to run multiple operating systems and applications in independent partitions, so that one computer system can function as multiple virtual system. Default setting is Enabled.

Package C State:

The concept of package C-States relates to the state of all the shared resources on the socket, as a function of the individual C-states that the cores enter. Invoking package C-states impacts the state of the mesh and the LLC, the core's power planes and the uncore PLL, and indirectly the state of the integrated peripherals. If set Package C State to special states, when system idle, the core will run at the limited package Cx state. This option allows 6 options: C0/C1 state, C2 state, C6(non Retention) state, C6(Retention) state, No Limit, Auto. Default setting is Auto.

• C0/C1 state:If the "C0/C1 state" is selected in the BIOS, the processor chip always remains active. It can improve the performance of latency sensitive workloads.
• C2 state:Stop the CPU's internal master clock through the hardware, memory path is open.
• C6(non Retention) state:All cores have saved their architectural state and have had their core voltages reduced to zero volts.
• C6(Retention) state:This state provides more power saving than C6 non-retention mode.
• No Limit:C state can downgrade to C10 when system is in idle.
• Auto:C state automatically adjusts based on system load.

LLC Prefetch:

LLC prefetcher is an additional prefetch mechanism on the top of the existing prefetchers that prefetch data into the core DCU amd MLC. Enabling LLC Prefetch gives the core prefetcher the ability to prefetch data directly into the LLC without necessarily filling into the MLC. Default is Disabled.

Flag description origin markings:

	Indicates that the flag description came from the user flags file.
	Indicates that the flag description came from the suite-wide flags file.
	Indicates that the flag description came from a per-benchmark flags file.

For questions about the meanings of these flags, please contact the tester.
For other inquiries, please contact info@spec.org
Copyright 2017-2024 Standard Performance Evaluation Corporation
Tested with SPEC CPU2017 v1.1.9.
Report generated on 2024-01-29 17:44:49 by SPEC CPU2017 flags formatter v5178.