Re: Make drm drivers use MTRR write-combine

On Tue, Aug 14, 2018 at 08:55:36AM -0500, Eric van Gyzen wrote:
> On 8/14/18 4:12 AM, Johannes Lundberg wrote:
> > Hi
> > 
> > Something that we have seen for a long time on FreeBSD is the boot message
> > 
> > Failed to add WC MTRR for [0xd0000000-0xdfffffff]: -22; performance may
> > suffer
> > 
> > Taking a closer look at this with memcontrol I can see that the 256 MB
> > region that DRM wants to set as WC is already covered by this entry
> > 0xc0000000/0x40000000 BIOS uncacheable set-by-firmware active
> > 
> > Similar on both my Skylake and Broadwell systems.
> I see something similar on my Dell XPS 13 with a Kaby Lake R:
> 
> Failed to add WC MTRR for [0x90000000-0x9fffffff]: -22; performance may 
> suffer
> 
> 0x80000000/0x80000000 BIOS uncacheable set-by-firmware active
> 
> The only mappings in this range are MMIO:
> 
> machdep.efi_map:
>            Type     Physical      Virtual   #Pages Attr
> [snip]
> MemoryMappedIO 0000e0000000   0xe0000000 00010000 RUNTIME
> MemoryMappedIO 0000fe000000   0xfe000000 00000011 UC RUNTIME
> MemoryMappedIO 0000fec00000   0xfec00000 00000001 UC RUNTIME
> MemoryMappedIO 0000fee00000   0xfee00000 00000001 UC WT WB WP RUNTIME
> MemoryMappedIO 0000ff000000   0xff000000 00001000 UC WT WB WP RUNTIME

Yes, the cause of the message is that current x86 mtrr code is not
sufficient to handle this situation. You have BIOS-configured variable
range MTRR which covers upper half of the low 4G, as uncacheable (UC).
It is reasonable for BIOS to set it up this way because this is where
PCIe BARs and other devices MMIO regions are located.

One of the BARs there is the GPU aperture that really should be WC
(write-combining). There are two ways to achieve this: split the UC
variable-length MTRR range into three, UC/WC/UC, which would require
three MTRRs to cover. This is what current x86_mem.c code does not
support.

Another way is to set WC mode in the page table entries (PTEs) using
Page Attribute Table (PAT), for all PTEs. According to the rules of
combination of the memory access types between MTRR and PAT, WC in PAT
and any access mode in MTRR gives effective WC.

I saw the same warning when I initially ported GEM. My code used WC PAT
type, which makes the warning cosmetical, and which made me to not add
MTRR split. If new drm driver also consistently uses WC memattr when
creating aperture mappings, then the warning can be ignored as well.