Re: The future of ZFS in FreeBSD

> On 19 Dec 2018, at 08:35, Matthew Macy <mmacy_at_freebsd.org> wrote:
> 
> On Tue, Dec 18, 2018 at 11:49 PM Enji Cooper <yaneurabeya_at_gmail.com <mailto:yaneurabeya_at_gmail.com>> wrote:
>> 
>> Hello Matthew,
>> 
>> I appreciate the long write up, as someone who still uses FreeBSD ZFS on my NAS box and knowing some of the history with ZFS on *Solaris, etc. Something like this was bound to happen with post the Oracle buyout.
>> 
>>> On Dec 18, 2018, at 10:49 PM, Matthew Macy <mmacy_at_freebsd.org> wrote:
>>> 
>>> The sources for FreeBSD's ZFS support are currently taken directly
>>> from Illumos with local ifdefs to support the peculiarities of FreeBSD
>>> where the Solaris Portability Layer (SPL) shims fall short. FreeBSD
>>> has regularly pulled changes from Illumos and tried to push back any
>>> bug fixes and new features done in the context of FreeBSD. In the past
>>> few years the vast majority of new development in ZFS has taken place
>>> in DelphixOS and zfsonlinux (ZoL). Earlier this year Delphix announced
>>> that they will be moving to ZoL
>>> https://www.delphix.com/blog/kickoff-future-eko-2018 This shift means
>>> that there will be little to no net new development of Illumos. While
>>> working through the git history of ZoL I have also discovered that
>>> many races and locking bugs have been fixed in ZoL and never made it
>>> back to Illumos and thus FreeBSD. This state of affairs has led to a
>>> general agreement among the stakeholders that I have spoken to that it
>>> makes sense to rebase FreeBSD's ZFS on ZoL. Brian Behlendorf
>>> has graciously encouraged me to add FreeBSD support directly to ZoL
>>> https://github.com/zfsonfreebsd/ZoF so that we might all have a single
>>> shared code base.
>>> 
>>> A port for ZoF can be found at https://github.com/miwi-fbsd/zof-port
>>> Before it can be committed some additional functionality needs to be
>>> added to the FreeBSD opencrypto framework. These can be found at
>>> https://reviews.freebsd.org/D18520
>>> 
>>> This port will provide FreeBSD users with multi modifier protection,
>>> project quotas, encrypted datasets, allocation classes, vectorized
>>> raidz, vectorized checksums, and various command line improvements.
>>> 
>>> Before ZoF can be merged back in to ZoL several steps need to be taken:
>>> - Integrate FreeBSD support into ZoL CI
>>> - Have most of the ZFS test suite passing
>>> - Complete additional QA testing at iX
>> 
>> Can you please describe the testing process that will be employed to verify the sanity of the ZoL on FreeBSD port? Should other large companies who use ZFS on FreeBSD (Panzura?) chime in and the ZFS on FreeBSD community (as a whole) collaborate to better suss out issues with the ZoL port?
> 
> The ZFS test suite itself provides ~80% coverage
> https://codecov.io/gh/zfsonlinux/zfs/branch/master <https://codecov.io/gh/zfsonlinux/zfs/branch/master> - FreeBSD currently
> lacks equivalent gcov support, but presumably it would provide
> comparable coverage here. Andrew Turner has some form of kernel gcov
> support that he uses with syzkaller. However, I believe that it isn't
> sufficient for this purpose.

The code I have is to trace the kernel part of a single thread, e.g. what happens in the kernel when you make a system call. It can trace function either function entry or places in the code with a conditional statement, e.g. an if, while, or switch statement. If this is enough for your case a tool could be written to track coverage from the tests.

I expect to update the review soon. I’m working on a man page, but have been too busy with work and other projects recently to finish writing it.

Andrew