Re: CURRENT: FreeBSD not reporting AES-NI on Intel(R) Xeon(R) CPU E5-1650 v3

From: Allan Jude <allanjude_at_freebsd.org>
Date: Fri, 17 Mar 2017 16:42:28 -0400
On 2017-03-17 16:30, Patrick Kelsey wrote:
> On Fri, Mar 17, 2017 at 1:31 PM, O. Hartmann <ohartmann_at_walstatt.org> wrote:
> 
>> Am Fri, 17 Mar 2017 20:07:35 +0300
>> Slawa Olhovchenkov <slw_at_zxy.spb.ru> schrieb:
>>
>>> On Fri, Mar 17, 2017 at 05:53:24PM +0100, O. Hartmann wrote:
>>>
>>>> Am Fri, 17 Mar 2017 15:04:29 +0300
>>>> Slawa Olhovchenkov <slw_at_zxy.spb.ru> schrieb:
>>>>
>>>>> On Fri, Mar 17, 2017 at 12:36:25PM +0100, O. Hartmann wrote:
>>>>>
>>>>>> Running recent CURRENT on a Fujitsu Celsius M740 equipted with an
>> Intel(R)
>>>>>> Xeon(R) CPU E5-1650 v3 _at_ 3.50GHz CPU makes me some trouble.
>>>>>>
>>>>>> FreeBSD does not report the existence or availability of AES-NI
>> feature, which
>>>>>> is supposed to be a feature of this type of CPU:
>>>>>
>>>>> What reassons to detect AES-NI by FreeBSD?
>>>>
>>>> What do you mean? I do not understand! FreeBSD is supposed to read the
>> CPUID and
>>>> therefore the capabilities as every other OS, too. But there may some
>> circumstances
>>>> why FBSD won't. I do not know, that is the reason why I'm asking here.
>>>
>>> This sample can have disabled AES-NI by vendor, in BIOS, for example.
>>> As I show by links this is posible.
>>>
>>> CPUID in you example don't show AES-NI capabilities, for example
>>> 1650v4 w/ AES-NI
>>>
>>> CPU: Intel(R) Xeon(R) CPU E5-1650 v4 _at_ 3.60GHz (3600.07-MHz K8-class CPU)
>>>   Origin="GenuineIntel"  Id=0x406f1  Family=0x6  Model=0x4f  Stepping=1
>>>   Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,
>> APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,
>> MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
>>>   Features2=0x7ffefbff<SSE3,PCLMULQDQ,DTES64,MON,DS_CPL,
>> VMX,SMX,EST,TM2,SSSE3,SDBG,FMA,CX16,xTPR,PDCM,PCID,DCA,
>> SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,TSCDLT,AESNI,XSAVE,
>> OSXSAVE,AVX,F16C,RDRAND>
>>>
>>
>>   ^^^^^^
>>>   AMD Features=0x2c100800<SYSCALL,NX,Page1GB,RDTSCP,LM>
>>>   AMD Features2=0x121<LAHF,ABM,Prefetch>
>>>   Structured Extended
>>> Features=0x21cbfbb<FSGSBASE,TSCADJ,BMI1,HLE,AVX2,SMEP,
>> BMI2,ERMS,INVPCID,RTM,PQM,NFPUSG,PQE,RDSEED,ADX,SMAP,PROCTRACE>
>>> XSAVE Features=0x1<XSAVEOPT> VT-x: PAT,HLT,MTF,PAUSE,EPT,UG,VPID,
>> VID,PostIntr
>>>   TSC: P-state invariant, performance statistics
>>>
>>> In you sample: "TSCDLT,XSAVE"
>>>
>>> May be AES-NI disabled by vendor and FreeBSD correct show this. Or some
>> bug in FreeBSD,
>>> AES-NI work and other OS show AES-NI capabilities.
>>>
>>> _______________________________________________
>>> freebsd-current_at_freebsd.org mailing list
>>> https://lists.freebsd.org/mailman/listinfo/freebsd-current
>>> To unsubscribe, send any mail to "freebsd-current-unsubscribe_at_
>> freebsd.org"
>>
>> We have some LGA1151 XEON based 19 inch rack server, also equipted with
>> Haswell
>> E3-12XX-v3 XEONs and FreeBSD, also CURRENT, does show AES-NI.
>>
>> You're right, the vendor could have disabled AES-NI by intention - but
>> they offered this
>> box especially with AES-NI capabilities.
>>
>> See here:
>>
>>   http://freebsd.1045724.x6.nabble.com/r285947-broken-
>> AESNI-support-No-aesni0-on-Intel-XEON-E5-1650-v3-on-Fujitsu-Celsius-M740-
>> td6028895.html
>>
>> I feel a bit pissed off right now due to Fujitsu, because we started
>> testing some
>> encrypting features and I'd like to use AES-NI and I run into this issue
>> again.
>>
>> I need to know that FreeBSD is not the issue with this specific CPU type.
>> I'm still
>> frustrated by that stupid comment "UNIX is not supoorted" I got that time
>> then when I
>> reported 2015 the issue to Fujitsu.
>>
>>
>>
> It's pretty straightforward to gain confidence that FreeBSD is not the
> issue here.  The 'Features2=' line is printed by printcpuinfo() in
> sys/x86/x86/identcpu.c based on the bits set in a variable called
> cpu_feature2 (the printf is currently at line 802).  The value of
> cpu_feature2 is set in identify_cpu() identcpu.c (for amd64, currently at
> line 1401) based on the result of the cpuid instruction that is executed by
> a call to do_cpuid(), which itself resides in sys/amd64/include/cpufunc.h.
> In other words, a single asm instruction is executed and the set bits from
> the result are printed.
> 
> Based on some poking around in open source bits (tianocore, coreboot), it
> appears that AES-NI is something the BIOS can irreversibly
> disable-until-next-reset by twiddling bits in the appropriate MSR
> register.  There is no code that does this in FreeBSD on purpose, so there
> would have to be a bug introduced in -CURRENT that somehow clobbers those
> MSR bits early on - a bug that was also not merged to 11-STABLE (since
> Slawa shows AESNI enabled on the same processor under 11-STABLE).
> 
> I will also say that I have dealt with a manufacturer of Xeon hardware in
> Europe who will not provide a stock BIOS that allows you to enable AES-NI,
> out of concerns over violating export/import rules governing encryption
> technology.  With that vendor, you have to pass an end-user verification
> and then they will make you a custom BIOS that gives you the option to
> enable AES-NI.  It took quite some time working through the outer layers of
> their support organization to even determine that this was the underlying
> (bureaucratic) issue.
> 
> Here is another data point: 11.0-RELEASE-p1 running on E5-1650 v3 with
> AESNI recognized as enabled.  You could install that stock FreeBSD release
> and if it does not show AESNI as enabled on your system, you will clearly
> know it is not a FreeBSD problem without any question as to whether there
> is a bug in CURRENT or having to build and install the exact rev of
> 11-STABLE that Slawa is using.
> 
> uname:
> FreeBSD ttest2 11.0-RELEASE-p1 FreeBSD 11.0-RELEASE-p1 #0 r306420: Thu Sep
> 29 01:43:23 UTC 2016
> root_at_releng2.nyi.freebsd.org:/usr/obj/usr/src/sys/GENERIC
>  amd64
> 
> dmesg:
> CPU: Intel(R) Xeon(R) CPU E5-1650 v3 _at_ 3.50GHz (3500.08-MHz K8-class CPU)
>   Origin="GenuineIntel"  Id=0x306f2  Family=0x6  Model=0x3f  Stepping=2
> 
> Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
>  Features2=0x7ffefbff<SSE3,PCLMULQDQ,DTES64,MON,DS_CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,TSCDLT,AESNI,XSAVE,OSXSAVE,AVX,F16C,RDRAND>
> 
> 
> -Patrick
> _______________________________________________
> freebsd-current_at_freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-current
> To unsubscribe, send any mail to "freebsd-current-unsubscribe_at_freebsd.org"
> 

I can also confirm it works fine on FreeBSD 11.0-RELEASE-p1 on the
E5-1650 v3, as I did all of the testing for my recent AsiaBSDCon paper
of SSH performance, on a pair of identical E5-1650 v3's in the FreeBSD
Test Cluster _at_ Sentex.

-- 
Allan Jude
Received on Fri Mar 17 2017 - 19:42:37 UTC

This archive was generated by hypermail 2.4.0 : Wed May 19 2021 - 11:41:10 UTC