Re: [RFC] [PATCH] VM & VFS changes

On  1 Jun, Andre Guibert de Bruet wrote:
> 
> On Wed, 1 Jun 2005, Don Lewis wrote:
>> On  1 Jun, Andre Guibert de Bruet wrote:
>>> On Wed, 1 Jun 2005, Alexander Leidinger wrote:
>>>> Poul-Henning Kamp <phk_at_phk.freebsd.dk> wrote:
>>>>
>>>>> Maybe the simplest solution is also the best:  keep track of the
>>>>> dependencies and do the cleanup leaf->root on the resulting tree.
>>
>> It might not even be necessary to use a tree.  It might be possible to
>> just use a list like vfs_unmountall().

I just thought of a case where this might not work.  It is possible to
create a swap-backed md, use it to back a file system, add a bunch of
swap, and then fill the file system, consuming the swap space.  if you
do the cleanup processing in reverse order, the first operation would be
to remove the swap device, which you might not be able to do because of
a lack of RAM and alternate swap space.

> I do some similar magic in my diff, to check for devfs. I can write a 
> function that unmounts all mds first.
>
>>>> How many userland processes have to be running and consuming memory which
>>>> isn't available as physical RAM at this point in the shutdown sequence?
>>>>
>>>> Wouldn't a loop like the following be enough?
>>>> while swap
>>>>     umount unbusy-FS
>>>>     swap-off swap
>>>>
>>>> This assumes that swap-off doesn't turns off the swap if it isn't able to put
>>>> everything back into other swap or physical RAM areas.
>>>
>>> I would think that one would want to disable swapping before the unmount
>>> of filesystems for the very fact you could have vnode-backed swapspace in
>>> use.
>>
>> This order doesn't work either because you might only have 128 MB of
>> RAM, but 1 GB of data in /tmp, which is stored on a swap-backed memory
>> disk.  In this case you'll have to unmount /tmp and toss the md contents
>> before you disable swap.
> 
> I could modify my patchset to get a first pass at MDs, then disable swap, 
> then unmount UFS/FFS/ext2/etc, then devfs. The question becomes: Is this 
> the correct process that we should follow? It makes sense to me. I would 
> like to get input from our VM & VFS gurus on this before I schedule a 
> hack-and-testathon... :-)

Taking care of the md's first is a good idea for the sake of efficiency,
because it eliminates the need to page in any of their contents that
have been paged out to swap.  The problem is that if they are used to
back file systems, any dependent file systems must be unmounted first,
and that might not be possible if one of the dependent file systems
contains a swap file.  An example would be using an md to back /tmp,
mounting /dev/adXXX on /tmp/foo, and adding /tmp/foo/swapfile as a swap
device.  It might not be possible to clean up this arrangement at
shutdown without deadlocking.

If a dependency tree is maintained, it should be possible prevent the
troublesome cases from happening.  If swapping to a swap-backed md or
swapping to vnode-backed md that resides in a file system that is
dependent on a file system that resides on a swap-backed md are
forbidden, I think that is sufficient to prevent deadlock.

Actually, I think the dependencies form a directed graph instead of
tree.  A simple example is a vnode-backed md /a/b that is used to back a
file system that is mounted on /c/d.  In this case, d is dependent on
both /a and /c.

A slightly more complex example is two vnode-backed md's, m1 and m2,
backed by /a/v1 and /b/v2, with m1 mounted on /b/d1 and m2 mounted on
/a/d2.  The reverse order traversal in vfs_unmountall() is smart enough
to handle this case (if it knew to unconfigure the md's), but a one-pass
recursive tree walk would not work.

A more complicated example would be two swap-backed md's mds1 and mds2
used to back file systems that are mounted on /a/mds1 and /a/mds2. Files
are created in each of these file systems /a/mds1/v1a, /a/mds1/v1b, and
/a/mds2/v2, that are used to back vnode-backed md's mdv1a, mdv1b, and
mdv2, that are used to back file systems that are mounted on
/a/mds2/d1a, /a/mds2/d1a/d1b, and /a/mds1/d2.

I think this can be unwound in one pass if a list of the dependency
pairs is kept and then properly sorted before processing.  The types of
dependencies are:
	md depends on file system (vnode backed md)
	md depends on swap (swap backed file system)
	file system depends on md (md backed file system)
	swap depends on md (swap on an md)
	file system depends on file system (mount relationship)
First undo any dependencies that ultimately depend on swap, unconfigure
the swap devices, and finally undo any dependencies that swap depended
on.


In this case, it is probably wise to design before coding ...