manually disable AVX512 / wrong CPUID on AMD EPYC 7282

[schnusch]

I run a virtual private server (hosted with Contabo) where I recently ran into issues running some Haskell software. NixOS/nixfmt#276 (comment)

The server seems to run on an AMD EPYC 7282 and it (or the hypervisor) seems to return a wrong feature set through CPUID. This later causes measure_off_avx to fail with SIGILL.

/proc/cpuinfo

processor       : 0                                                 
vendor_id       : AuthenticAMD                                      
cpu family      : 23                                                
model           : 49                                                                                                                                                                                                                                                              
model name      : AMD EPYC 7282 16-Core Processor                                                                                                                                                                                                                                 
stepping        : 0                                                 
microcode       : 0x830107a                                                                                                              
cpu MHz         : 2794.748                                          
cache size      : 512 KB                                            
physical id     : 0                                                 
siblings        : 4                                                 
core id         : 0                                                 
cpu cores       : 4                                                 
apicid          : 0                                                 
initial apicid  : 0                                                 
fpu             : yes                                               
fpu_exception   : yes                                               
cpuid level     : 16                                                
wp              : yes                                               
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm rep_good nopl cpuid extd_apicid tsc_known_freq pni pclmulqdq ssse3 fma cx16 sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm cmp_legacy cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw perfctr_core ssbd ibrs ibpb stibp vmmcall fsgsbase tsc_adjust bmi1 avx2 smep bmi2 rdseed adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 clzero xsaveerptr wbnoinvd arat umip rdpid arch_capabilities          
bugs            : sysret_ss_attrs null_seg spectre_v1 spectre_v2 spec_store_bypass retbleed smt_rsb srso                                 
bogomips        : 5589.49                                           
TLB size        : 1024 4K pages                                     
clflush size    : 64                                                
cache_alignment : 64                                                
address sizes   : 40 bits physical, 48 bits virtual                 
power management:

The call to __get_cpuid_count seems to report wrong results, because the AMD EPYC 7282 is not supposed to support AVX512 per its specs, but the following test program returns support for AVX512:

#include <cpuid.h>
#include <stdint.h>
#include <stdio.h>

static void print_register_bits(uint32_t x, const char *s) {
    for(size_t i = 0; i < 32; ++i) {
        printf("%s & (1 << %zu) = %lu\n", s, i, (unsigned long)(x & ((uint32_t)1 << i)));
    }
}

int main(void) {
    uint32_t eax = 0, ebx = 0, ecx = 0, edx = 0;
    __get_cpuid_count(7, 0, &eax, &ebx, &ecx, &edx);
    // https://en.wikipedia.org/wiki/CPUID#EAX=7,_ECX=0:_Extended_Features
    print_register_bits(ebx, "ebx");
    return 0;
}

Namely the following extended feature flags which should not be supported:

• avx512-f ebx & (1 << 16)
• avx512-dq ebx & (1 << 17)
• avx512-ifma ebx & (1 << 21)
• avx512-cd ebx & (1 << 28)
• avx512-bw ebx & (1 << 30)
• avx512-vl ebx & (1 << 31)

Complete output of the test program

ebx & (1 << 0) = 1         
ebx & (1 << 1) = 2          
ebx & (1 << 2) = 0          
ebx & (1 << 3) = 8
ebx & (1 << 4) = 0                                               
ebx & (1 << 5) = 32
ebx & (1 << 6) = 0
ebx & (1 << 7) = 128
ebx & (1 << 8) = 256
ebx & (1 << 9) = 512
ebx & (1 << 10) = 1024
ebx & (1 << 11) = 0
ebx & (1 << 12) = 0
ebx & (1 << 13) = 0
ebx & (1 << 14) = 0
ebx & (1 << 15) = 0
ebx & (1 << 16) = 65536
ebx & (1 << 17) = 131072
ebx & (1 << 18) = 262144
ebx & (1 << 19) = 524288
ebx & (1 << 20) = 1048576
ebx & (1 << 21) = 2097152
ebx & (1 << 22) = 0
ebx & (1 << 23) = 8388608
ebx & (1 << 24) = 16777216
ebx & (1 << 25) = 0
ebx & (1 << 26) = 0
ebx & (1 << 27) = 0
ebx & (1 << 28) = 268435456
ebx & (1 << 29) = 536870912
ebx & (1 << 30) = 1073741824
ebx & (1 << 31) = 2147483648

Even though this seems to be a CPU issue, I did not find a way to disable the AVX512 detection during compile time. My currently solution is to patch has_avx512_vl_bw to always return false.

It would be nice to have a compile time option to disable AVX512 detection manually.

[Bodigrim]

An external workaround to disable AVX512 is to pass -D__STDC_NO_ATOMICS__, e. g., via cabal.project.

[schnusch]

Wouldn't that disable the SSE implementation as well?

text/cbits/measure_off.c

Lines 213 to 224 in 1ae86be

	#ifndef __STDC_NO_ATOMICS__
	static _Atomic measure_off_t s_impl = (measure_off_t)NULL;
	measure_off_t impl = atomic_load_explicit(&s_impl, memory_order_relaxed);
	if (!impl) {
	#if defined(__x86_64__) && defined(COMPILER_SUPPORTS_AVX512)
	impl = has_avx512_vl_bw() ? measure_off_avx : measure_off_sse;
	#else
	impl = measure_off_sse;
	#endif
	atomic_store_explicit(&s_impl, impl, memory_order_relaxed);
	}
	#else

[Bodigrim]

Wouldn't that disable the SSE implementation as well?

Why? The very next line says measure_off_t impl = measure_off_sse.