Re: MPSAFE VFS -- List of upcoming actions

On 9/19/12, Kevin Oberman <kob6558_at_gmail.com> wrote:
> On Wed, Sep 19, 2012 at 12:30 AM, Attilio Rao <attilio_at_freebsd.org> wrote:
>> On Wed, Sep 19, 2012 at 4:47 AM, Kevin Oberman <kob6558_at_gmail.com> wrote:
>>> On Tue, Sep 18, 2012 at 7:48 PM, Attilio Rao <attilio_at_freebsd.org>
>>> wrote:
>>>> On Fri, Jul 13, 2012 at 12:18 AM, Attilio Rao <attilio_at_freebsd.org>
>>>> wrote:
>>>>> 2012/7/4 Attilio Rao <attilio_at_freebsd.org>:
>>>>>> 2012/6/29 Attilio Rao <attilio_at_freebsd.org>:
>>>>>>> As already published several times, according to the following plan:
>>>>>>> http://wiki.freebsd.org/NONMPSAFE_DEORBIT_VFS
>>>>>>>
>>>>>>
>>>>>> I still haven't heard from Vivien or Edward, anyway as NTFS is
>>>>>> basically only used RO these days (also the mount_ntfs code just
>>>>>> permits RO mounting) I stripped all the uncomplete/bogus write
>>>>>> support
>>>>>> with the following patch:
>>>>>> http://www.freebsd.org/~attilio/ntfs_remove_write.patch
>>>>>>
>>>>>> This is an attempt to make the code smaller and possibly just focus
>>>>>> on
>>>>>> the locking that really matter (as read-only filesystem).
>>>>>> On some points of the patch I'm a bit less sure as we could easily
>>>>>> take into account also write for things like vaccess() arguments, and
>>>>>> make easier to re-add correct write support at some point in the
>>>>>> future, but still force RO, even if the approach used in the patch is
>>>>>> more correct IMHO.
>>>>>> As an added bonus this patch cleans some dirty code in the mount
>>>>>> operation and fixes a bug as vfs_mountedfrom() is called before real
>>>>>> mounting is completed and can still fail.
>>>>>
>>>>> A quick update on this.
>>>>> It looks like NTFS won't be completed for this GSoC thus I seriously
>>>>> need to find an alternative to not loose the NTFS support entirely.
>>>>>
>>>>> I tried to look into the NTFS implementation right now and it is
>>>>> really a poor support. As Peter has also verified, it can deadlock in
>>>>> no-time, it compeltely violates VFS rules, etc. IMHO it deserves a
>>>>> complete rewrite if we would still support in-kernel NTFS. I also
>>>>> tried to look at the NetBSD implementation. Their code is someway
>>>>> similar to our, but they used very complicated (and very dirty) code
>>>>> to do the locking. Even if I don't know well enough NetBSD VFS, I have
>>>>> the impression not all the races are correctly handled. Definitively,
>>>>> not something I would like to port.
>>>>>
>>>>> Considering all that the only viable option would be meaning an
>>>>> userland filesystem implementation. My preferred choice would be to
>>>>> import PUFFS and librefuse on top of it but honestly it requires a lot
>>>>> of time to be completed, time which I don't currently have as in 2
>>>>> months Giant must be gone by the VFS.
>>>>>
>>>>> I then decided to switch to gnn's rewamp of FUSE patches. You can find
>>>>> his initial e-mail here:
>>>>> http://lists.freebsd.org/pipermail/freebsd-fs/2012-March/013876.html
>>>>>
>>>>> I've precisely got the second version of George's patch and created
>>>>> this dolphin branch:
>>>>> svn://svn.freebsd.org/base/projects/fuse
>>>>>
>>>>> I'm fixing low hanging fruit for the moment (see r238411 for example)
>>>>> and I still have to make a throughful review.
>>>>> However my idea is to commit the support once:
>>>>> - ntfs-3g is well stress-tested and proves to be bug-free
>>>>> - there is no major/big technical issue pending after the reviews
>>>>
>>>> In the last weeks Peter, Florian, Gustau and I have been working in
>>>> stabilizing fuse support. In the specific, Peter has worked hard on
>>>> producing several utilities to nit stress-test fuse and in particular
>>>> ntfs, Florian has improved fuse related ports (as explained later) and
>>>> Gustau has done sparse testing. I feel moderately satisfied by the
>>>> level of stability of fuse now to propose to wider usage, in
>>>> particular given the huge amount of complaints I'm hearing around
>>>> about occasional fuse users.
>>>>
>>>> The final target of the project is to completely import into base the
>>>> content of fusefs-kmod starting from earlier posted patches by George.
>>>> So far, we took care only of importing in the fuse branch the kernel
>>>> part, so that fusefs-kmod userland part is still needed to be
>>>> installed from ports, but I was studying the mount_fusefs licensing
>>>> before to process with the import for the userland bits of it.
>>>>
>>>> The fixing has been happening here:
>>>> svn://svn.freebsd.org/base/projects/fuse/
>>>>
>>>> which is essentially an HEAD branch + fuse kernel components. In order
>>>> to get fuse, please compile a kernel from this branch with FUSE option
>>>> or simply build and load fuse module.
>>>> Alternatively, a kernel patch that should work with HEAD_at_240684 is
>>>> here:
>>>> http://www.freebsd.org/~attilio/fuse_import/fuse_240684.patch
>>>>
>>>> I guess the patch can easilly apply to all FreeBSD branches, really,
>>>> but it is not tested to anything else different then -CURRENT.
>>>>
>>>> As said you still need currently to build fusefs-kmod port. However
>>>> you need these further patches, to be put in the fusefs-kmod/files/
>>>> directory::
>>>> http://www.freebsd.org/~attilio/fuse_import/patch-Makefile
>>>> http://www.freebsd.org/~attilio/fuse_import/patch-mount_fusefs__mount_fusefs2.c
>>>>
>>>> They both disable the old kernel building/linking and import new
>>>> functionality to let the new kernel support work well in presence of
>>>> many consumers.
>>>>
>>>> In addition to fusefs-kmod, Bryan and Florian have also updated
>>>> fusefs-lib and fusefs-ntfs ports. For instance, please refer to this
>>>> e-mail:
>>>> http://lists.freebsd.org/pipermail/freebsd-ports/2012-August/077950.html
>>>>
>>>> Even if this work is someway independent by the fusefs-kmod import, I
>>>> warmly suggest to all of you to use their patches (and this what we
>>>> have been testing so far too).
>>>>
>>>> At this point what I'm looking for are reviews and further testing.
>>>> I would like to spend some words on what you should expect from this
>>>> work:
>>>> *Fuse is far from being perfect*.
>>>> I cannot stress this enough. Peter stress-tests could break also Fuse
>>>> on Linux generally and by Fuse authors admissions the modules can
>>>> never guarantee to be completely starvation-free. However, they tend
>>>> to be designed in a way that sleeps can be at least interrupted
>>>> easily, making at least easy to recover from deadlocks. This is mostly
>>>> retained also in FreeBSD, for what I can tell. Also, sometimes fuse
>>>> seems to leave a small amount of hidden files, when it find references
>>>> on files it wants to delete. This happens also under Linux and it is
>>>> part of FUSE design, not much we can do.
>>>> However, if deadlocks can be someway tollerated, things you should
>>>> really pay attention are dumps of fuse modules (like ntfs-3g binary)
>>>> and kernel panics. They must not happen and if they do they need to be
>>>> fixed promptly.
>>>> However, the good new is that ntfs seems doing exceptionally good.
>>>> Florian could use ntfs as a backend for postgresql test. I think this
>>>> is by far a big improvement if compared to current in-kernel ntfs
>>>> which is completely torned.
>>>>
>>>> So far we have almost entirely tested only ntfs-3g. I know Gustau also
>>>> used other modules like sshfs and George used GlusterFS with his older
>>>> patches, but I encourage you to test as many modules as you want, as
>>>> they may expose different bugs. Of course, I don't plan to spend much
>>>> more time on FUSE, but I can occasionally look at bugs as they fall in
>>>> the filesystems category and I'm always interested in keeping a good
>>>> open eye on such issues.
>>>>
>>>> A few operational informations:
>>>> - In the next days I will import the userland bits of fusefs-kmod to
>>>> the fuse project branch making the port obsolete. When this happens I
>>>> will make this clear to the user of this thread.
>>>> - If no major bug is remained by the early October, I will commit this
>>>> to -CURRENT
>>>> - I expect Bryan and Florian to commit libfuse and ntfs updates soon.
>>>> They can do independently from the fusefs-kmod retiral, but I would
>>>> prefer their patches to go on first.
>>>> - After that I will handover fusefs maintainership to gnn as agreed in
>>>> precedence but I will be around helping with analysis and fixing,
>>>> depending on time availability
>>>>
>>>> In the end I have really 2 minor questions:
>>>> - One is about importing the mount_fusefs userland bits. I don't think
>>>> we need a vendor import at all because they were developed by a
>>>> FreeBSD GSoC student and kept in his git repo (or someone else's).
>>>> Anyway, i'd just commit as new files once I do a good sweep. I hope
>>>> nobody objects to that.
>>>> - Another one is: fusefs-kmod right now is only amd64/i386 specific. I
>>>> have no idea why as it has not any MD specific code. However I'm sure
>>>> it has not been tested on other arches so far. Anyway I left it usable
>>>> by all the arches. I think this is the correct choice. If someone
>>>> objects with valid argument I can bring it back to be usable only on
>>>> i386 and amd64.
>>>>
>>>> That's all, for any question please don't hesitate to contact me and
>>>> the other people involved in this work.
>>>
>>> Attilio (and the crew),
>>>
>>> Thanks for working on fusefs-ntfs. It's been increasingly worrying to
>>> me that we might lose it and I really depend on it. I really hope to
>>> be able to use rsync to update files without killing my system some
>>> day.
>>>
>>> I tried the new fusefs-libs and fusefs-ntfs ports from Florian and
>>> Bryan, but ran into trouble as I could no longer build the kmod after
>>> installing the updated fusefs-libs. It had an unresolved symbol:
>>> cc -O2 -pipe -fno-strict-aliasing -Werror -D_KERNEL -DKLD_MODULE
>>> -nostdinc  -I../include -I. -I_at_ -I_at_/contrib/altq -finline-limit=8000
>>> --param inline-unit-growth=100 --param large-function-growth=1000
>>> -fno-common  -fno-omit-frame-pointer  -mcmodel=kernel -mno-red-zone
>>> -mno-mmx -mno-sse -msoft-float  -fno-asynchronous-unwind-tables
>>> -ffreestanding -fstack-protector -std=iso9899:1999 -fstack-protector
>>> -Wall -Wredundant-decls -Wnested-externs -Wstrict-prototypes
>>> -Wmissing-prototypes -Wpointer-arith -Winline -Wcast-qual  -Wundef
>>> -Wno-pointer-sign -fformat-extensions  -Wmissing-include-dirs
>>> -fdiagnostics-show-option   -c fuse_vnops.c
>>> fuse_vnops.c: In function 'create_filehandle':
>>> fuse_vnops.c:1586: error: 'struct fuse_open_in' has no member named
>>> 'mode'
>>> *** [fuse_vnops.o] Error code 1
>>>
>>> This was on amd64 9-Stable r239879 until/unless this issue is
>>> resolved, please keep the existing port available and/or mark the new
>>> one to not install on pre-10 systems.
>>
>> If you follow the rule I described in this e-mail, the fusefs-kmod
>> kernel part won't be build anymore, so you won't run into this.
>> If it is build yet, please let me know because there is a bug in the 2
>> patches I posted for fusefs-kmod port.
>
> Attilo,
>
> I assumed that your new kernel module was only tested/working with
> current, so I did not try to use it. I was only referring to the use
> of the updated of fusefs-libs and fusefs-ntfs that Florian and Bryan
> provided. I had tested these on 9-stable and found that after
> installing the updated fusefs-libs, the old fusefs-kmod port would no
> longer compile.
>
> Today Florian sent me a one line patch to fuse-modue/fuse-vnops.c in
> the current fusefs-kmod port which appears to have fixed the problem.
> It compiled fine and it is currently running on the system on which I
> am typing this. I have done a bit of light testing and it works to
> this point. I'll do some heavier testing later today. So it looks like
> this there is probably no issue with Florian committing the new
> fusefs-libs and fusefs-ntfs ports for those of us not running current.

Thanks for let us know. I think that Bryan and Florian should really
update the ports as soon as possible.

Also, I hope that someone will sync the fusefs-kmod port (in
particular the kernel part) with the kernel code that our branch
brings along. I think Florian volountereed for this, so there should
not be a problem on that.

Attilio


-- 
Peace can only be achieved by understanding - A. Einstein