Re: MPSAFE VFS -- List of upcoming actions

On Tue, Sep 18, 2012 at 7:48 PM, Attilio Rao <attilio_at_freebsd.org> wrote:
> On Fri, Jul 13, 2012 at 12:18 AM, Attilio Rao <attilio_at_freebsd.org> wrote:
>> 2012/7/4 Attilio Rao <attilio_at_freebsd.org>:
>>> 2012/6/29 Attilio Rao <attilio_at_freebsd.org>:
>>>> As already published several times, according to the following plan:
>>>> http://wiki.freebsd.org/NONMPSAFE_DEORBIT_VFS
>>>>
>>>
>>> I still haven't heard from Vivien or Edward, anyway as NTFS is
>>> basically only used RO these days (also the mount_ntfs code just
>>> permits RO mounting) I stripped all the uncomplete/bogus write support
>>> with the following patch:
>>> http://www.freebsd.org/~attilio/ntfs_remove_write.patch
>>>
>>> This is an attempt to make the code smaller and possibly just focus on
>>> the locking that really matter (as read-only filesystem).
>>> On some points of the patch I'm a bit less sure as we could easily
>>> take into account also write for things like vaccess() arguments, and
>>> make easier to re-add correct write support at some point in the
>>> future, but still force RO, even if the approach used in the patch is
>>> more correct IMHO.
>>> As an added bonus this patch cleans some dirty code in the mount
>>> operation and fixes a bug as vfs_mountedfrom() is called before real
>>> mounting is completed and can still fail.
>>
>> A quick update on this.
>> It looks like NTFS won't be completed for this GSoC thus I seriously
>> need to find an alternative to not loose the NTFS support entirely.
>>
>> I tried to look into the NTFS implementation right now and it is
>> really a poor support. As Peter has also verified, it can deadlock in
>> no-time, it compeltely violates VFS rules, etc. IMHO it deserves a
>> complete rewrite if we would still support in-kernel NTFS. I also
>> tried to look at the NetBSD implementation. Their code is someway
>> similar to our, but they used very complicated (and very dirty) code
>> to do the locking. Even if I don't know well enough NetBSD VFS, I have
>> the impression not all the races are correctly handled. Definitively,
>> not something I would like to port.
>>
>> Considering all that the only viable option would be meaning an
>> userland filesystem implementation. My preferred choice would be to
>> import PUFFS and librefuse on top of it but honestly it requires a lot
>> of time to be completed, time which I don't currently have as in 2
>> months Giant must be gone by the VFS.
>>
>> I then decided to switch to gnn's rewamp of FUSE patches. You can find
>> his initial e-mail here:
>> http://lists.freebsd.org/pipermail/freebsd-fs/2012-March/013876.html
>>
>> I've precisely got the second version of George's patch and created
>> this dolphin branch:
>> svn://svn.freebsd.org/base/projects/fuse
>>
>> I'm fixing low hanging fruit for the moment (see r238411 for example)
>> and I still have to make a throughful review.
>> However my idea is to commit the support once:
>> - ntfs-3g is well stress-tested and proves to be bug-free
>> - there is no major/big technical issue pending after the reviews
>
> In the last weeks Peter, Florian, Gustau and I have been working in
> stabilizing fuse support. In the specific, Peter has worked hard on
> producing several utilities to nit stress-test fuse and in particular
> ntfs, Florian has improved fuse related ports (as explained later) and
> Gustau has done sparse testing. I feel moderately satisfied by the
> level of stability of fuse now to propose to wider usage, in
> particular given the huge amount of complaints I'm hearing around
> about occasional fuse users.
>
> The final target of the project is to completely import into base the
> content of fusefs-kmod starting from earlier posted patches by George.
> So far, we took care only of importing in the fuse branch the kernel
> part, so that fusefs-kmod userland part is still needed to be
> installed from ports, but I was studying the mount_fusefs licensing
> before to process with the import for the userland bits of it.
>
> The fixing has been happening here:
> svn://svn.freebsd.org/base/projects/fuse/
>
> which is essentially an HEAD branch + fuse kernel components. In order
> to get fuse, please compile a kernel from this branch with FUSE option
> or simply build and load fuse module.
> Alternatively, a kernel patch that should work with HEAD_at_240684 is here:
> http://www.freebsd.org/~attilio/fuse_import/fuse_240684.patch
>
> I guess the patch can easilly apply to all FreeBSD branches, really,
> but it is not tested to anything else different then -CURRENT.
>
> As said you still need currently to build fusefs-kmod port. However
> you need these further patches, to be put in the fusefs-kmod/files/
> directory::
> http://www.freebsd.org/~attilio/fuse_import/patch-Makefile
> http://www.freebsd.org/~attilio/fuse_import/patch-mount_fusefs__mount_fusefs2.c
>
> They both disable the old kernel building/linking and import new
> functionality to let the new kernel support work well in presence of
> many consumers.
>
> In addition to fusefs-kmod, Bryan and Florian have also updated
> fusefs-lib and fusefs-ntfs ports. For instance, please refer to this
> e-mail:
> http://lists.freebsd.org/pipermail/freebsd-ports/2012-August/077950.html
>
> Even if this work is someway independent by the fusefs-kmod import, I
> warmly suggest to all of you to use their patches (and this what we
> have been testing so far too).
>
> At this point what I'm looking for are reviews and further testing.
> I would like to spend some words on what you should expect from this work:
> *Fuse is far from being perfect*.
> I cannot stress this enough. Peter stress-tests could break also Fuse
> on Linux generally and by Fuse authors admissions the modules can
> never guarantee to be completely starvation-free. However, they tend
> to be designed in a way that sleeps can be at least interrupted
> easily, making at least easy to recover from deadlocks. This is mostly
> retained also in FreeBSD, for what I can tell. Also, sometimes fuse
> seems to leave a small amount of hidden files, when it find references
> on files it wants to delete. This happens also under Linux and it is
> part of FUSE design, not much we can do.
> However, if deadlocks can be someway tollerated, things you should
> really pay attention are dumps of fuse modules (like ntfs-3g binary)
> and kernel panics. They must not happen and if they do they need to be
> fixed promptly.
> However, the good new is that ntfs seems doing exceptionally good.
> Florian could use ntfs as a backend for postgresql test. I think this
> is by far a big improvement if compared to current in-kernel ntfs
> which is completely torned.
>
> So far we have almost entirely tested only ntfs-3g. I know Gustau also
> used other modules like sshfs and George used GlusterFS with his older
> patches, but I encourage you to test as many modules as you want, as
> they may expose different bugs. Of course, I don't plan to spend much
> more time on FUSE, but I can occasionally look at bugs as they fall in
> the filesystems category and I'm always interested in keeping a good
> open eye on such issues.
>
> A few operational informations:
> - In the next days I will import the userland bits of fusefs-kmod to
> the fuse project branch making the port obsolete. When this happens I
> will make this clear to the user of this thread.
> - If no major bug is remained by the early October, I will commit this
> to -CURRENT
> - I expect Bryan and Florian to commit libfuse and ntfs updates soon.
> They can do independently from the fusefs-kmod retiral, but I would
> prefer their patches to go on first.
> - After that I will handover fusefs maintainership to gnn as agreed in
> precedence but I will be around helping with analysis and fixing,
> depending on time availability
>
> In the end I have really 2 minor questions:
> - One is about importing the mount_fusefs userland bits. I don't think
> we need a vendor import at all because they were developed by a
> FreeBSD GSoC student and kept in his git repo (or someone else's).
> Anyway, i'd just commit as new files once I do a good sweep. I hope
> nobody objects to that.
> - Another one is: fusefs-kmod right now is only amd64/i386 specific. I
> have no idea why as it has not any MD specific code. However I'm sure
> it has not been tested on other arches so far. Anyway I left it usable
> by all the arches. I think this is the correct choice. If someone
> objects with valid argument I can bring it back to be usable only on
> i386 and amd64.
>
> That's all, for any question please don't hesitate to contact me and
> the other people involved in this work.

Attilio (and the crew),

Thanks for working on fusefs-ntfs. It's been increasingly worrying to
me that we might lose it and I really depend on it. I really hope to
be able to use rsync to update files without killing my system some
day.

I tried the new fusefs-libs and fusefs-ntfs ports from Florian and
Bryan, but ran into trouble as I could no longer build the kmod after
installing the updated fusefs-libs. It had an unresolved symbol:
cc -O2 -pipe -fno-strict-aliasing -Werror -D_KERNEL -DKLD_MODULE
-nostdinc  -I../include -I. -I_at_ -I_at_/contrib/altq -finline-limit=8000
--param inline-unit-growth=100 --param large-function-growth=1000
-fno-common  -fno-omit-frame-pointer  -mcmodel=kernel -mno-red-zone
-mno-mmx -mno-sse -msoft-float  -fno-asynchronous-unwind-tables
-ffreestanding -fstack-protector -std=iso9899:1999 -fstack-protector
-Wall -Wredundant-decls -Wnested-externs -Wstrict-prototypes
-Wmissing-prototypes -Wpointer-arith -Winline -Wcast-qual  -Wundef
-Wno-pointer-sign -fformat-extensions  -Wmissing-include-dirs
-fdiagnostics-show-option   -c fuse_vnops.c
fuse_vnops.c: In function 'create_filehandle':
fuse_vnops.c:1586: error: 'struct fuse_open_in' has no member named 'mode'
*** [fuse_vnops.o] Error code 1

This was on amd64 9-Stable r239879 until/unless this issue is
resolved, please keep the existing port available and/or mark the new
one to not install on pre-10 systems.
-- 
R. Kevin Oberman, Network Engineer
E-mail: kob6558_at_gmail.com