CPU2017 Flag Description
Cisco Systems Cisco UCS C125 (AMD EPYC 7552, 2.20 GHz)
Compilers: AMD Optimizing C/C++ Compiler Suite
Base Compiler Invocation
-
- clang
![[user]](https://www.spec.org/auto/cpu2017/images/user.png)
- CC, LD
-
clang is a C compiler which encompasses preprocessing, parsing, optimization, code generation, assembly, and linking.
Depending on which high-level mode setting is passed, Clang will stop before doing a full link.
-
- clang++
- CXX, LD
-
clang++ C++ compiler which encompasses preprocessing, parsing, optimization, code generation, assembly, and linking.
Depending on which high-level mode setting is passed, Clang will stop before doing a full link.
-
- flang
- FC, LD
-
flang is a Fortran compiler which encompasses parsing, optimization, code generation, assembly, and linking. Depending on
which high-level mode setting is passed, Flang will stop before doing a full link.
-
- flang
- FC, LD
-
flang is a Fortran compiler which encompasses parsing, optimization, code generation, assembly, and linking. Depending on
which high-level mode setting is passed, Flang will stop before doing a full link.
-
- clang
- CC
-
clang is a C compiler which encompasses preprocessing, parsing, optimization, code generation, assembly, and linking.
Depending on which high-level mode setting is passed, Clang will stop before doing a full link.
-
- clang++
- CXX, LD
-
clang++ C++ compiler which encompasses preprocessing, parsing, optimization, code generation, assembly, and linking.
Depending on which high-level mode setting is passed, Clang will stop before doing a full link.
-
- clang
- CC
-
clang is a C compiler which encompasses preprocessing, parsing, optimization, code generation, assembly, and linking.
Depending on which high-level mode setting is passed, Clang will stop before doing a full link.
-
- clang++
- CXX, LD
-
clang++ C++ compiler which encompasses preprocessing, parsing, optimization, code generation, assembly, and linking.
Depending on which high-level mode setting is passed, Clang will stop before doing a full link.
-
- clang
- CC
-
clang is a C compiler which encompasses preprocessing, parsing, optimization, code generation, assembly, and linking.
Depending on which high-level mode setting is passed, Clang will stop before doing a full link.
-
- flang
- FC
-
flang is a Fortran compiler which encompasses parsing, optimization, code generation, assembly, and linking. Depending on
which high-level mode setting is passed, Flang will stop before doing a full link.
Base Portability Flags
-
![[suite]](https://www.spec.org/auto/cpu2017/images/suite.png)
- EXTRA_PORTABILITY
-
This option is used to indicate that the host system's integers are 32-bits
wide, and longs and pointers are 64-bits wide. Not all benchmarks
recognize this macro, but the preferred practice for data model selection
applies the flags to all benchmarks; this flag description is a placeholder
for those benchmarks that do not recognize this macro.
-
- EXTRA_PORTABILITY
-
This option is used to indicate that the host system's integers are 32-bits
wide, and longs and pointers are 64-bits wide. Not all benchmarks
recognize this macro, but the preferred practice for data model selection
applies the flags to all benchmarks; this flag description is a placeholder
for those benchmarks that do not recognize this macro.
-
- EXTRA_PORTABILITY
-
This option is used to indicate that the host system's integers are 32-bits
wide, and longs and pointers are 64-bits wide. Not all benchmarks
recognize this macro, but the preferred practice for data model selection
applies the flags to all benchmarks; this flag description is a placeholder
for those benchmarks that do not recognize this macro.
-
- EXTRA_PORTABILITY
-
This option is used to indicate that the host system's integers are 32-bits
wide, and longs and pointers are 64-bits wide. Not all benchmarks
recognize this macro, but the preferred practice for data model selection
applies the flags to all benchmarks; this flag description is a placeholder
for those benchmarks that do not recognize this macro.
-
- EXTRA_PORTABILITY
-
This option is used to indicate that the host system's integers are 32-bits
wide, and longs and pointers are 64-bits wide. Not all benchmarks
recognize this macro, but the preferred practice for data model selection
applies the flags to all benchmarks; this flag description is a placeholder
for those benchmarks that do not recognize this macro.
-
- EXTRA_PORTABILITY
-
This option is used to indicate that the host system's integers are 32-bits
wide, and longs and pointers are 64-bits wide. Not all benchmarks
recognize this macro, but the preferred practice for data model selection
applies the flags to all benchmarks; this flag description is a placeholder
for those benchmarks that do not recognize this macro.
-
-
-
- EXTRA_PORTABILITY
-
This option is used to indicate that the host system's integers are 32-bits
wide, and longs and pointers are 64-bits wide. Not all benchmarks
recognize this macro, but the preferred practice for data model selection
applies the flags to all benchmarks; this flag description is a placeholder
for those benchmarks that do not recognize this macro.
-
-
-
- EXTRA_PORTABILITY
-
This option is used to indicate that the host system's integers are 32-bits
wide, and longs and pointers are 64-bits wide. Not all benchmarks
recognize this macro, but the preferred practice for data model selection
applies the flags to all benchmarks; this flag description is a placeholder
for those benchmarks that do not recognize this macro.
-
-
- EXTRA_PORTABILITY
-
This option is used to indicate that the host system's integers are 32-bits
wide, and longs and pointers are 64-bits wide. Not all benchmarks
recognize this macro, but the preferred practice for data model selection
applies the flags to all benchmarks; this flag description is a placeholder
for those benchmarks that do not recognize this macro.
-
- EXTRA_PORTABILITY
-
This option is used to indicate that the host system's integers are 32-bits
wide, and longs and pointers are 64-bits wide. Not all benchmarks
recognize this macro, but the preferred practice for data model selection
applies the flags to all benchmarks; this flag description is a placeholder
for those benchmarks that do not recognize this macro.
-
- EXTRA_PORTABILITY
-
This option is used to indicate that the host system's integers are 32-bits
wide, and longs and pointers are 64-bits wide. Not all benchmarks
recognize this macro, but the preferred practice for data model selection
applies the flags to all benchmarks; this flag description is a placeholder
for those benchmarks that do not recognize this macro.
-
- EXTRA_PORTABILITY
-
This option is used to indicate that the host system's integers are 32-bits
wide, and longs and pointers are 64-bits wide. Not all benchmarks
recognize this macro, but the preferred practice for data model selection
applies the flags to all benchmarks; this flag description is a placeholder
for those benchmarks that do not recognize this macro.
-
- EXTRA_PORTABILITY
-
This option is used to indicate that the host system's integers are 32-bits
wide, and longs and pointers are 64-bits wide. Not all benchmarks
recognize this macro, but the preferred practice for data model selection
applies the flags to all benchmarks; this flag description is a placeholder
for those benchmarks that do not recognize this macro.
Base Optimization Flags
-
-
-
-
-
-
- -O3
- COPTIMIZE
-
Like -O2, except that it enables optimizations that take longer to perform or that may generate larger code (in
an attempt to make the program run faster).
If multiple "O" options are used, with or without level numbers, the last such option is the one that is effective.
- Includes:
-
- -ffast-math
- COPTIMIZE
-
Enables a range of optimizations that provide faster, though sometimes less precise, mathematical operations that may
not conform to the IEEE-754 specifications. When this option is specified, the __STDC_IEC_559__ macro is
ignored even if set by the system headers.
-
- -march=znver2
- COPTIMIZE
-
Specify that Clang should generate code for a specific processor family member and later. For example, if you specify
-march=znver1, the compiler is allowed to generate instructions that are valid on AMD Zen processors, but
which may not exist on earlier products.
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
- -O3
- CXXOPTIMIZE
-
Like -O2, except that it enables optimizations that take longer to perform or that may generate larger code (in
an attempt to make the program run faster).
If multiple "O" options are used, with or without level numbers, the last such option is the one that is effective.
- Includes:
-
- -ffast-math
- CXXOPTIMIZE
-
Enables a range of optimizations that provide faster, though sometimes less precise, mathematical operations that may
not conform to the IEEE-754 specifications. When this option is specified, the __STDC_IEC_559__ macro is
ignored even if set by the system headers.
-
- -march=znver2
- CXXOPTIMIZE
-
Specify that Clang should generate code for a specific processor family member and later. For example, if you specify
-march=znver1, the compiler is allowed to generate instructions that are valid on AMD Zen processors, but
which may not exist on earlier products.
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
- -O3
- FOPTIMIZE
-
Like -O2, except that it enables optimizations that take longer to perform or that may generate larger code (in
an attempt to make the program run faster).
If multiple "O" options are used, with or without level numbers, the last such option is the one that is effective.
- Includes:
-
- -march=znver2
- FOPTIMIZE
-
Specify that Clang should generate code for a specific processor family member and later. For example, if you specify
-march=znver1, the compiler is allowed to generate instructions that are valid on AMD Zen processors, but
which may not exist on earlier products.
-
-
- -Mrecursive
- FOPTIMIZE
-
Allocate local variables on the stack, thus allowing recursion.
SAVEd, data-initialized, or namelist members are always allocated
statically, regardless of the setting of this switch.
-
-
-
-
-
-
-
-
-
-
-
-
-
-
- -O3
- COPTIMIZE, FOPTIMIZE
-
Like -O2, except that it enables optimizations that take longer to perform or that may generate larger code (in
an attempt to make the program run faster).
If multiple "O" options are used, with or without level numbers, the last such option is the one that is effective.
- Includes:
-
- -ffast-math
- COPTIMIZE
-
Enables a range of optimizations that provide faster, though sometimes less precise, mathematical operations that may
not conform to the IEEE-754 specifications. When this option is specified, the __STDC_IEC_559__ macro is
ignored even if set by the system headers.
-
- -march=znver2
- COPTIMIZE, FOPTIMIZE
-
Specify that Clang should generate code for a specific processor family member and later. For example, if you specify
-march=znver1, the compiler is allowed to generate instructions that are valid on AMD Zen processors, but
which may not exist on earlier products.
-
-
-
-
-
-
-
-
-
-
-
-
- -Mrecursive
- FOPTIMIZE
-
Allocate local variables on the stack, thus allowing recursion.
SAVEd, data-initialized, or namelist members are always allocated
statically, regardless of the setting of this switch.
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
- -O3
- COPTIMIZE, CXXOPTIMIZE
-
Like -O2, except that it enables optimizations that take longer to perform or that may generate larger code (in
an attempt to make the program run faster).
If multiple "O" options are used, with or without level numbers, the last such option is the one that is effective.
- Includes:
-
- -ffast-math
- COPTIMIZE, CXXOPTIMIZE
-
Enables a range of optimizations that provide faster, though sometimes less precise, mathematical operations that may
not conform to the IEEE-754 specifications. When this option is specified, the __STDC_IEC_559__ macro is
ignored even if set by the system headers.
-
- -march=znver2
- COPTIMIZE, CXXOPTIMIZE
-
Specify that Clang should generate code for a specific processor family member and later. For example, if you specify
-march=znver1, the compiler is allowed to generate instructions that are valid on AMD Zen processors, but
which may not exist on earlier products.
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
- -O3
- COPTIMIZE, CXXOPTIMIZE, FOPTIMIZE
-
Like -O2, except that it enables optimizations that take longer to perform or that may generate larger code (in
an attempt to make the program run faster).
If multiple "O" options are used, with or without level numbers, the last such option is the one that is effective.
- Includes:
-
- -ffast-math
- COPTIMIZE, CXXOPTIMIZE
-
Enables a range of optimizations that provide faster, though sometimes less precise, mathematical operations that may
not conform to the IEEE-754 specifications. When this option is specified, the __STDC_IEC_559__ macro is
ignored even if set by the system headers.
-
- -march=znver2
- COPTIMIZE, CXXOPTIMIZE, FOPTIMIZE
-
Specify that Clang should generate code for a specific processor family member and later. For example, if you specify
-march=znver1, the compiler is allowed to generate instructions that are valid on AMD Zen processors, but
which may not exist on earlier products.
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
- -Mrecursive
- FOPTIMIZE
-
Allocate local variables on the stack, thus allowing recursion.
SAVEd, data-initialized, or namelist members are always allocated
statically, regardless of the setting of this switch.
-
-
-
-
-
-
-
Implicitly Included Flags
This section contains descriptions of flags that were included implicitly
by other flags, but which do not have a permanent home at SPEC.
-
- -O2
-
Moderate level of optimization which enables most optimizations. This is the default when no "-O" option is
specified, or if no value is specified (i.e. "-O").
If multiple "O" options are used, with or without level numbers, the last such option is the one that is effective.
- Includes:
-
- -O1
-
Somewhere between -O0 and -O2.
If multiple "O" options are used, with or without level numbers, the last such option is the one that is effective.
Using numactl to bind processes and memory to cores
For multi-copy runs or single copy runs on systems with multiple sockets, it is advantageous to bind a process to a
particular core. Otherwise, the OS may arbitrarily move your process from one core to another. This can affect
performance. To help, SPEC allows the use of a "submit" command where users can specify a utility to use to bind
processes. We have found the utility 'numactl' to be the best choice.
numactl runs processes with a specific NUMA scheduling or memory placement policy. The policy is set for a
command and inherited by all of its children. The numactl flag "--physcpubind" specifies
which core(s) to bind the process. "-l" instructs numactl to keep a process's memory on the
local node while "-m" specifies which node(s) to place a process's memory. For full details on using
numactl, please refer to your Linux documentation, 'man numactl'
Note that some older versions of numactl incorrectly interpret application arguments as its own. For
example, with the command "numactl --physcpubind=0 -l a.out -m a", numactl will interpret
a.out's "-m" option as its own "-m" option. To work around this problem, we put
the command to be run in a shell script and then run the shell script using numactl. For example:
"echo 'a.out -m a' > run.sh ; numactl --physcpubind=0 bash run.sh"
Transparent Huge Pages (THP)
THP is an abstraction layer that automates most aspects of creating, managing, and using huge pages. THP is designed
to hide much of the complexity in using huge pages from system administrators and developers, as normal huge pages
must be assigned at boot time, can be difficult to manage manually, and often require significant changes to code in
order to be used effectively. Most recent Linux OS releases have THP enabled by default.
Linux Huge Page settings
If you need finer control you can manually set huge pages using the following steps:
- Create a mount point for the huge pages:
mkdir /mnt/hugepages
- The huge page file system needs to be mounted when the systems reboots. Add the following to a system boot
configuration file before any services are started:
mount -t hugetlbfs nodev /mnt/hugepages
- Set
vm/nr_hugepages=N in /etc/sysctl.conf where N is the maximum number of pages the
system may allocate.
- Reboot to have the changes take effect.
Note that further information about huge pages may be found in the Linux kernel documentation file
hugetlbpage.txt.
ulimit -s <n>
Sets the stack size to n kbytes, or unlimited to allow the stack size to grow without limit.
ulimit -l <n>
Sets the maximum size of memory that may be locked into physical memory.
powersave -f (on SuSE)
Makes the powersave daemon set the CPUs to the highest supported frequency.
/etc/init.d/cpuspeed stop (on Red Hat)
Disables the cpu frequency scaling program in order to set the CPUs to the highest supported frequency.
LD_LIBRARY_PATH
An environment variable that indicates the location in the filesystem of bundled libraries to use when running the
benchmark binaries.
kernel/randomize_va_space
This option can be used to select the type of process address space randomization that is used in the system, for
architectures that support this feature.
- 0 - Turn the process address space randomization off. This is the default for architectures that do not support
this feature anyway, and kernels that are booted with the "
norandmaps" parameter.
- 1 - Make the addresses of mmap base, stack and VDSO page randomized. This, among other things, implies that
shared libraries will be loaded to random addresses. Also for PIE-linked binaries, the location of code start is
randomized. This is the default if the
CONFIG_COMPAT_BRK option is enabled.
- 2 - Additionally enable heap randomization. This is the default if
CONFIG_COMPAT_BRK is
disabled.
MALLOC_CONF
An environment variable set to tune the jemalloc allocation strategy during the execution of the binaries. This
environment variable setting is not needed when building the binaries on the system under test.
Operating System and Software Tuning Parameters
- ulimit -s <n>
-
Sets the stack size to n kbytes, or unlimited to allow the stack size
to grow without limit.
- numactl --interleave=all "runspec command"
-
Launching a process with numactl --interleave=all sets the memory interleave policy so that memory will be allocated using
round robin on nodes. When memory cannot be allocated on the current interleave target fall back to other nodes.
- Free the file system page cache
-
The command "echo 1> /proc/sys/vm/drop_caches" is used to free up the filesystem page cache.
- Using numactl to bind processes and memory to cores
-
For multi-copy runs or single copy runs on systems with multiple sockets, it is advantageous to bind a process to a particular core. Otherwise, the OS may arbitrarily move your process from one core to another. This can affect performance. To help, SPEC allows the use of a "submit" command where users can specify a utility to use to bind processes. We have found the utility 'numactl' to be the best choice.
numactl runs processes with a specific NUMA scheduling or memory placement policy. The policy is set for a command and inherited by all of its children. The numactl flag "--physcpubind" specifies which core(s) to bind the process. "-l" instructs numactl to keep a process memory on the local node while "-m" specifies which node(s) to place a process memory. For full details on using numactl, please refer to your Linux documentation, 'man numactl'
- dirty_background_ratio
-
This is the percentage of the total amount of free and reclaimable memory. When the amount of dirty pagecache exceeds this percentage, writeback threads start writing back dirty memory. This setting can help Linux disk caching and performance by setting the percentage of system memory that can be filled with dirty pages. This can be set through a command like "echo 40 > /proc/sys/vm/dirty_background_ratio".
- swappiness
-
This control is used to define how aggressively the kernel swaps out anonymous memory relative to pagecache and other caches. Increasing the value increases the amount of swapping. The default value is 60. A value of 1 tells the kernel to only swap processes to disk if absolutely necessary. This can be set through a command like "echo 1 > /proc/sys/vm/swappiness".
- Zone Reclaim Mode
-
This parameter controls whether memory reclaim is performed on a local NUMA node even if there is plenty of memory free on other nodes. This parameter is automatically turned on on machines with more pronounced NUMA characteristics. To tell the kernel to free local node memory rather than grabbing free memory from remote nodes, use a command like "echo 1 > /proc/sys/vm/zone_reclaim_mode".
- dirty_ratio
-
A percentage value. When this percentage of total system memory is modified, the system begins writing the modifications to disk with the pdflush operation. The default value is 20 percent. To tell the kernel to free local node memory rather than grabbing free memory from remote nodes, use a command like "echo 1 > /proc/sys/vm/zone_reclaim_mode". This can be set through a command "echo 8 > /proc/sys/vm/dirty_ratio".
- Linux Huge Page settings
-
In order to take advantage of large pages, your system must be configured to use large pages.
To configure your system for huge pages perform the following steps:
Create a mount point for the huge pages: "mkdir /mnt/hugepages"
The huge page file system needs to be mounted when the systems reboots. Add the following to a system boot configuration file before any services are started: "mount -t hugetlbfs nodev /mnt/hugepages"
Set vm/nr_hugepages=N in your /etc/sysctl.conf file where N is the maximum number of pages the system may allocate.
Reboot to have the changes take effect. (Not necessary on some operating systems like RedHat Enterprise Linux 5.5).
Note that further information about huge pages may be found in your Linux documentation file: /usr/src/linux/Documentation/vm/hugetlbpage.txt
Transparent Huge Pages
On RedHat EL 6 and later, Transparent Hugepages increases the memory page size from 4 kilobytes to 2 megabytes. Transparent Hugepages provides significant performance advantages on systems with highly contended resources
and large memory workloads. If memory utilization is too high or memory is badly fragmented which prevents hugepages being allocated, the kernel will assign smaller 4k pages instead. Hugepages are used by default
if /sys/kernel/mm/redhat_transparent_hugepage/enabled is set to always.
- HUGETLB_MORECORE
-
Set this environment variable to "yes" to enable applications to use large pages.
- KMP_STACKSIZE
-
Specify stack size to be allocated for each thread.
- KMP_AFFINITY
-
KMP_AFFINITY = < physical | logical >, starting-core-id
specifies the static mapping of user threads to physical cores. For example,
if you have a system configured with 8 cores, OMP_NUM_THREADS=8 and
KMP_AFFINITY=physical,0 then thread 0 will mapped to core 0, thread 1 will be mapped to core 1, and
so on in a round-robin fashion.
KMP_AFFINITY = granularity=fine,scatter
The value for the environment variable KMP_AFFINITY affects how the threads from an auto-parallelized program are scheduled across processors.
Specifying granularity=fine selects the finest granularity level, causes each OpenMP thread to be bound to a single thread context.
This ensures that there is only one thread per core on cores supporting HyperThreading Technology
Specifying scatter distributes the threads as evenly as possible across the entire system.
Hence a combination of these two options, will spread the threads evenly across sockets, with one thread per physical core.
- OMP_NUM_THREADS
-
Sets the maximum number of threads to use for OpenMP* parallel regions if no
other value is specified in the application. This environment variable
applies to both -openmp and -parallel (Linux and Mac OS X) or /Qopenmp and /Qparallel (Windows).
Example syntax on a Linux system with 8 cores:
export OMP_NUM_THREADS=8
- Determinism Slider
-
This option allows the processor to use a given performance level as the max cap, or to let the processor operate as close to the thermal design point (TDP) as possible. Values for this BIOS option can be:
Power: Processor operates as close to the TDP as possible.
Performance: Processor operates at a capped performance level as the max operating state.
- NUMA Nodes Per Socket:
-
NUMA nodes per socket (NPS) field allows you to configure the memory NUMA domains per socket. The configuration can consist of one whole domain (NPS1), two domains (NPS2), or four domains (NPS4). In the case of a two-socket platform, an additional NPS profile is available to have whole system memory to be mapped as single NUMA domain (NPS0).
- High Bandwidth:
-
Enabling this option allows the chipset to defer memory transactions and
process them out of order for optimal performance.
- submit= MYMASK=`printf '0x%x' \$((1<< \$SPECCOPYNUM))`; /usr/bin/taskset \$MYMASK $command
-
When running multiple copies of benchmarks, the SPEC config file feature submit is sometimes used to cause individual jobs to be bound to specific processors. This specific submit command is used for Linux. The description of the elements of the command are:
/usr/bin/taskset [options] [mask] [pid | command [arg] ... ]
:
taskset is used to set or retreive the CPU affinity of a running process given its PID or to launch a new COMMAND with a given CPU affinity. The CPU affinity is represented as a bitmask, with the lowest order bit corresponding to the first logical CPU and highest order bit corresponding to the last logical CPU. When the taskset returns, it is guaranteed that the given program has been scheduled to a legal CPU. :
The default behaviour of taskset is to run a new command with a given affinity mask: :
taskset [mask] [command] [arguments]
- $MYMASK: The bitmask (in hexadecimal) corresponding to a specific SPECCOPYNUM. For example, $MYMASK value for the first copy of a rate run will be 0x00000001, for the second copy of the rate will be 0x00000002 etc. Thus, the first copy of the rate run will have a CPU affinity of CPU0, the second copy will have the affinity CPU1 etc.
- $command: Program to be started, in this case, the benchmark instance to be started. :
For questions about the meanings of these flags, please contact the tester.
For other inquiries, please contact info@spec.org
Copyright 2017-2020 Standard Performance Evaluation Corporation
Tested with SPEC CPU2017 v1.1.0.
Report generated on 2020-09-15 14:37:38 by SPEC CPU2017 flags formatter v5178.
![[benchmark]](https://www.spec.org/auto/cpu2017/images/benchmark.png)