Re: minidump size on amd64

on 30/09/2010 01:26 Alan Cox said the following:
>  On 9/29/2010 3:41 PM, Andriy Gapon wrote:
>> So perhaps we need to add another level of indirection?
>> I.e. first dump contiguous array of "pseudo-pde" entries that would point to
>> chunks of "pseudo-pte" entries, so that "pseudo-pte" entries could be sparse.
>> This is instead of dumping 1GB of contiguous "pseudo-ptes" as we do now.
>>
> 
> That would be the best approach.  That said, for the foreseeable future, the
> kernel page table on amd64 will have two valid ranges, no more, no less.  So, if
> it's much easier to modify minidump to deal with a page table that is assumed to
> have two contiguous parts, just do it.  That assumption should remain valid for
> a few years.

I tried to go for a full thing, at least how I understood it.
Two hours of hacking frenzy and here is a result:

http://people.freebsd.org/~avg/amd64-minidump.diff

I hope that the code is not too ugly.  At least my testing shows that it's not
(badly) broken.

The idea.  We dump contiguously only pages with PDEs (which means both valid and
invalid PDEs), valid pages with PTEs are dumped the same way as data physical
pages (i.e. via dump_add_page, etc); no fake PTEs for 2MB pages.
PDE area of the dump takes about 20MB as opposed to 1GB for PTE area (the math
is obvious, but just in case).

libkva is changed to treat former PTE area as PDE area and is also taught to
understand PG_PS in PDE.
There is now an overhead of having to first read a PTE page in V-to-P-to-offset
lookup for !PG_PS case.  Perhaps we could cache all PTEs in memory and have a
lookup table for them, but I didn't bother with this possibly premature
optimization at this time.

There is an unrelated change in minidumpsys - "bitmap_frozen".
I had to do it despite having a patch in my local tree to stop other CPUs on
panic->dump.  Code in dump path (peripheral disk driver, CAM, SIM driver,
something else?) seems to do some memory allocations and change dump bitmap,
which leads to a mismatch between dump size and dump bitmap; and also
potentially to inconsistencies in the bitmap itself.  So I decided that it's a
good idea to freeze the bitmap once we decided what pages we want to dump.

Some variables and structure fields with 'pte' in them should probably be
renamed to have 'pde' instead.

What do you think?
I will appreciate reviews, testing, comments, etc.
Thanks!
-- 
Andriy Gapon