aocc130-flags-revA2 AMD Optimizing C/C++ Compiler Suite Version 1.3.0 SPEC CPU2017 Flag Description Using numactl to bind processes and memory to cores

For multi-copy runs or single copy runs on systems with multiple sockets, it is advantageous to bind a process to a particular core. Otherwise, the OS may arbitrarily move your process from one core to another. This can affect performance. To help, SPEC allows the use of a "submit" command where users can specify a utility to use to bind processes. We have found the utility 'numactl' to be the best choice.

numactl runs processes with a specific NUMA scheduling or memory placement policy. The policy is set for a command and inherited by all of its children. The numactl flag "--physcpubind" specifies which core(s) to bind the process. "-l" instructs numactl to keep a process's memory on the local node while "-m" specifies which node(s) to place a process's memory. For full details on using numactl, please refer to your Linux documentation, 'man numactl'

Note that some older versions of numactl incorrectly interpret application arguments as its own. For example, with the command "numactl --physcpubind=0 -l a.out -m a", numactl will interpret a.out's "-m" option as its own "-m" option. To work around this problem, we put the command to be run in a shell script and then run the shell script using numactl. For example: "echo 'a.out -m a' > run.sh ; numactl --physcpubind=0 bash run.sh"

]]> Transparent Huge Pages (THP)

THP is an abstraction layer that automates most aspects of creating, managing, and using huge pages. THP is designed to hide much of the complexity in using huge pages from system administrators and developers, as normal huge pages must be assigned at boot time, can be difficult to manage manually, and often require significant changes to code in order to be used effectively. Most recent Linux OS releases have THP enabled by default.

Linux Huge Page settings

If you need finer control you can manually set huge pages using the following steps:

Note that further information about huge pages may be found in the Linux kernel documentation file hugetlbpage.txt.

ulimit -s <n>

Sets the stack size to n kbytes, or unlimited to allow the stack size to grow without limit.

ulimit -l <n>

Sets the maximum size of memory that may be locked into physical memory.

cpupower frequency-set -r -g performance (on Ubuntu, SLES, RHEL)

Sets the CPU governor to "performance" to enable the highest supported performance state for all cores.

powersave -f (on SuSE)

Makes the powersave daemon set the CPUs to the highest supported frequency.

/etc/init.d/cpuspeed stop (on Red Hat)

Disables the cpu frequency scaling program in order to set the CPUs to the highest supported frequency.

OMP_NUM_THREADS

Sets the maximum number of OpenMP parallel threads applications based on OpenMP may use.

LD_LIBRARY_PATH

An environment variable that indicates the location in the filesystem of bundled libraries to use when running the benchmark binaries.

kernel/randomize_va_space

This option can be used to select the type of process address space randomization that is used in the system, for architectures that support this feature.

MALLOC_CONF

An environment variable set to tune the jemalloc allocation strategy during the execution of the binaries. This environment variable setting is not needed when building the binaries on the system under test.

]]>
Compilers: AMD Optimizing C/C++ Compiler Suite 1.3.0 ]]>
Splitter rule for plugin arguments: -fplugin-arg-dragonegg-llvm-option="-flag[ -flag...]" Consumer rule for the tail of split up plugin arguments: -fplugin-arg-dragonegg-llvm-option="" -O Set the optimization level to -O2.

If multiple "O" options are used, with or without level numbers, the last such option is the one that is effective.

]]> -O0 Means "no optimization". This level compiles the fastest and generates the most debuggable code.

If multiple "O" options are used, with or without level numbers, the last such option is the one that is effective.

]]> -O1 Somewhere between -O0 and -O2.

If multiple "O" options are used, with or without level numbers, the last such option is the one that is effective.

]]> -O2 Moderate level of optimization which enables most optimizations. This is the default when no "-O" option is specified, or if no value is specified (i.e. "-O").

If multiple "O" options are used, with or without level numbers, the last such option is the one that is effective.

]]> -O3 Like -O2, except that it enables optimizations that take longer to perform or that may generate larger code (in an attempt to make the program run faster).

If multiple "O" options are used, with or without level numbers, the last such option is the one that is effective.

]]> -Os Like -O2 with extra optimizations to reduce code size.

If multiple "O" options are used, with or without level numbers, the last such option is the one that is effective.

]]> -Oz Like -Os (and thus -O2), but reduces code size further.

If multiple "O" options are used, with or without level numbers, the last such option is the one that is effective.

]]> -O4 Equivalent to -O3.

If multiple "O" options are used, with or without level numbers, the last such option is the one that is effective.

]]> -Ofast Enables all the optimizations from -O3 along with other aggressive optimizations that may violate strict compliance with language standards. Refer to the AOCC options document for the language you're using for more detailed documentation of optimizations enabled under -Ofast.

]]> -march=znver1 Specify that Clang should generate code for a specific processor family member and later. For example, if you specify -march=znver1, the compiler is allowed to generate instructions that are valid on AMD Zen processors, but which may not exist on earlier products.

]]> -fno-exceptions (For C++ only) Enable generation of unwind information. This allows exceptions to be thrown through Clang compiled stack frames. This is on by default in x86-64. -fno-exceptions disables C++ exception handling.

]]> -flto Generate output files in LLVM formats suitable for link time optimization. When used with -S this generates LLVM intermediate language assembly files, otherwise this generates LLVM bitcode format object files (which may be passed to the linker depending on the stage selection options).

]]> -m32 Generate code for a 32-bit environment. The 32-bit environment sets int, long and pointer to 32 bits and generates code that runs on any i386 system. The compiler generates x86 or IA32 32-bit ABI. The default on a 32-bit host is 32-bit ABI. The default on a 64-bit host is 64-bit ABI if the target platform specified is 64-bit, otherwise the default is 32-bit.

]]> -m64 Generate code for a 64-bit environment. The 64-bit environment sets int to 32 bits and long and pointer to 64 bits and generates code for AMD's x86-64 architecture. The compiler generates AMD64, INTEL64, x86-64 64-bit ABI. The default on a 32-bit host is 32-bit ABI. The default on a 64-bit host is 64-bit ABI if the target platform specified is 64-bit, otherwise the default is 32-bit.

]]> -frepack-arrays In some circumstances GNU Fortran may pass assumed shape array sections via a descriptor describing a noncontiguous area of memory. This option adds code to the function prologue to repack the data into a contiguous block at runtime.

This should result in faster accesses to the array. However it can introduce significant overhead to the function call, especially when the passed data is noncontiguous.

]]> -fuse-ld=lld Instructs the compiler to use the LLVM linker and loader (lld) instead of the system linker when linking object files.

]]> -fuse-ld=gold Instructs the compiler to use the GNU gold linker (gold) instead of the system linker when linking object files.

]]> -fno-stack-arrays Instructs the compiler not to allocate arrays from the stack and instead use heap memory.

]]> -freciprocal-math Given the expression "a = b / c", instructs the compiler to calculate "a = b * (1/c)".

]]> -ffast-math Enables a range of optimizations that provide faster, though sometimes less precise, mathematical operations that may not conform to the IEEE-754 specifications. When this option is specified, the __STDC_IEC_559__ macro is ignored even if set by the system headers.

]]> This option enables floating-point expression contraction such as forming of fused multiply-add operations if the target has native support for them.

]]> -fplugin=dragonegg.so Load compiler plugin code from the dragonegg.so file.

]]> -specs=integrated-as.specs The DragonEgg plugin generates LLVM intermediate representation for link-time optimization which must be passed to the LLVM assembler. integrated-as.specs tells the compiler to use the LLVM integrated assembler rather than the system assembler.

]]> -lm Instructs the compiler to link with system math libraries.

]]> -lamdlibm Instructs the compiler to link with AMD-supported optimized math library.

]]> This option instructs the linker to link the executable with the pthread library.

]]> This option instructs the linker to link the executable with libdl, the interface to the dynamic loader.

]]> -z muldefs Instructs the linker to use the first definition encountered for a symbol, and ignore all others.

]]> -ljemalloc Use the jemalloc library, which is a general purpose malloc(3) implementation that emphasizes fragmentation avoidance and scalable concurrency support.

]]> Use the SmartHeap library, which is a fast, portable, reliable, ANSI-compliant malloc/operator new library.

]]> -lomp Instructs the compiler to link with the OpenMP runtime libraries.

]]> -lgfortran Instructs the compiler to link with the gfortran Fortran runtime libraries.

]]> -lflang Instructs the compiler to link with flang Fortran runtime libraries.

]]> The option combines compare/test instructions with branches wherever possible.

]]> > The induction variable simplification optimization transforms induction variables to simpler forms. The option disables this optimization.

]]> Sets the default integer and logical types to an 8 byte type. It does not promote variables with explicit KIND declarations.

]]> "-Wl,-mllvm -Wl,-enable-gvn-hoist" or "-fplugin-arg-dragonegg-llvm-option=-enable-gvn-hoist" This option enables the GVN hoist pass, which is used to hoist computations from branches.

]]> "-Wl,-mllvm -Wl,-inline-threshold:1000" or "-fplugin-arg-dragonegg-llvm-option=-inline-threshold:1000" Sets the compiler's inlining threshold level to the value passed. The inline threshold is used in the inliner heuristics to decide which functions should be inlined.

]]> "-Wl,-mllvm -Wl,-enable-vectorize-compares" or "-fplugin-arg-dragonegg-llvm-option=-enable-vectorize-compares" This option enables vectorization of loops with conditional breaks. In some extreme situations this may result in unsafe behavior. Use this option only if it is safe to execute such loops even if conditional breaks are skipped or otherwise not executed.

]]> "-Wl,-mllvm -Wl,-enable-vectorize-compares:false" or "-fplugin-arg-dragonegg-llvm-option=-enable-vectorize-compares:false" Certain loops with breaks may be vectorized by default at -O2 and above. In some extreme situations this may result in unsafe behavior. Use this option to disable vectorization of such loops.

]]> "-Wl,-mllvm -Wl,-enable-iv-split" or "-fplugin-arg-dragonegg-llvm-option=-enable-iv-split" Enables splitting of long live ranges of loop induction variables which span loop boundaries. This helps reduce register pressure and can help avoid needless spills to memory and reloads from memory.

]]> "-Wl,-mllvm -Wl,-enable-partial-unswitch" or "-fplugin-arg-dragonegg-llvm-option=-enable-partial-unswitch" This optimization does partial unswitching of loops where some part of the unswitched control flow remains in the loop.

]]> This option enables an optimization that generates and calls specialized function versions when the loops inside function are vectorizable and the arguments are not aliased with each other. This optimization helps in function inlining and vectorization.

]]> This option enables an optimization that generates and calls specialized function versions when they are called with constant arguments. This optimization helps in function inlining.

]]> This option enables an optimization that transforms the data layout of a single dimensional array to provide better cache locality by analyzing the access patterns.

]]> -fstruct-layout=3 This option enables transformation of the layout of arrays of structure types and their fields to improve the cache locality. Aggressive analysis and transformations are performed at higher levels. This option is effective only with -flto as whole program analysis is required to perform this optimization.

Possible values:

]]> Turns on LLVM's instrumentation based profiling.

]]> Uses the profiling files generated from a program compiled with -fprofile-instr-generate to guide optimization decisions.

]]> -fgnu89-inline In the 502/602.gcc benchmark description, "multiple definitions of symbols" is listed under the "Known Portability Issues" section, and this option is one of the suggested workarounds. This option causes Clang to revert to the same inlining behavior that GCC does when in pre-C99 mode.

]]> Sets the compiler's inlining heuristics to an aggressive level by increasing the inline thresholds.

]]> -Wl,-mllvm -Wl,-inline-threshold=99 Sets the compiler's inlining threshold level to the value passed as the argument. The inline threshold is used in the inliner heuristics to decide which functions should be inlined.

]]> -Wl,-mllvm -Wl,-inline-recursion=4 Enables inlining for recursive functions based on heuristics, with level 4 being most aggressive. Higher levels may lead to code bloat due to expansion of recursive functions at call sites.

Levels:

]]> This option enables aggressive loop unswitching heuristic based on usage of branch conditions.

]]> "-Wl,-mllvm -Wl,-lsr-in-nested-loop" or "-fplugin-arg-dragonegg-llvm-option=-lsr-in-nested-loop" Enables loop strength reduction for nested loop structures. By default, the compiler will do loop strength reduction only for the innermost loop.

]]> "-Wl,-mllvm -Wl,-merge-constant" or "-fplugin-arg-dragonegg-llvm-option=-merge-constant" This optimization merges duplicate constant uses into a register to reduce instruction width.

]]> This option enables AVX2 (Advanced Vector Extensions, 2nd generation) support.

]]> This option restricts the optimization and code generation to first-generation AVX instructions.

]]> This option enables generation of the adx instruction.

]]> This option disables generation of the adx instruction.

]]> This option enables the generation of SSE4a instructions.

]]> This option disables the generation of SSE4a instructions.

]]> "-mllvm -unroll-threshold=100" or "-fplugin-arg-dragonegg-llvm-option=-unroll-threshold:100" Sets the limit at which loops will be unrolled. For example, if unroll threshold is set to 100 then only loops with 100 or fewer instructions will be unrolled.

]]> "-mllvm -unroll-aggressive" or "-fplugin-arg-dragonegg-llvm-option=-unroll-aggressive" Enables aggressive heuristics to get loop unrolling.

]]> -mllvm -unroll-count=8 The unroll count can be specified explicitly with -unroll_count=<value> where <value> is a positive integer. If this value is greater than the trip count, the loop will be fully unrolled.

]]> -funroll-loops This option instructs the compiler to unroll loops wherever possible.

]]> "-mllvm -vectorize-memory-aggressively" or "-fplugin-arg-dragonegg-llvm-option=-vectorize-memory-aggressively" This option avoids runtime memory dependency checks to enable aggressive vectorization.

]]> "-mllvm -x86-use-vzeroupper=false" or "-fplugin-arg-dragonegg-llvm-option=-x86-use-vzeroupper:false" This option causes AOCC to not emit a vzeroupper instruction before a transfer of control flow, in order to minimize the AVX to SSE transition penalty.

]]> "-mllvm -x86-use-vzeroupper" or "-fplugin-arg-dragonegg-llvm-option=-x86-use-vzeroupper:true" This option causes AOCC to emit a vzeroupper instruction before a transfer of control flow.

]]> -std=gnu89 Selects the C language dialect.

]]> -std=c++98 Selects the C++ language dialect.

]]> -fconvert=big-endian The binary datasets for some of the Fortran benchmarks in the SPEC CPU suites are stored in big-endian format. This option is necessary for those datasets to be read in correctly.

]]> Specifies size of off_t data type.

]]> /path/to/{clang|clang++|flang|gfortran} This flag is just to trim the path from the compiler line. clang clang is a C compiler which encompasses preprocessing, parsing, optimization, code generation, assembly, and linking. Depending on which high-level mode setting is passed, Clang will stop before doing a full link.

]]> clang++ clang++ C++ compiler which encompasses preprocessing, parsing, optimization, code generation, assembly, and linking. Depending on which high-level mode setting is passed, Clang will stop before doing a full link.

]]> flang Flang is a Fortran compiler which encompasses parsing, optimization, code generation, assembly, and linking. Depending on which high-level mode setting is passed, Flang will stop before doing a full link.

]]> gfortran [optimization flags] -fplugin=path/dragonegg.so [plugin optimization flags] Invoke the GNU Fortran compiler with the optimizers and code generators replaced by those from AOCC. The "dragonegg.so" referenced here is the "AOCC Fortran Plugin".

]]> -L/path/to/libs Specifies a directory to search for libraries. Use -L to add directories to the search path for library files. Multiple -L options are valid. However, the position of multiple -L options is important relative to -l options supplied.

]]> -I /path/to/include Specifies a directory to search for include files. Use -I to add directories to the search path for include files. Multiple -I options are valid.

]]> -DUSE_OPENMP Switch to enable OpenMP.

]]>