FreeBSD Project Quarterly Status Report - 1st Quarter 2017 While a few of these projects indicate they are a "plan B" or an "attempt III", many are still hewing to their original plans, and all have produced impressive results. Please enjoy this vibrant collection of reports, covering the first quarter of 2017. --Benjamin Kaduk __________________________________________________________________ The deadline for submissions covering the period from April to June 2017 is July 7, 2017. __________________________________________________________________ FreeBSD Team Reports * The FreeBSD Core Team * The FreeBSD Foundation * The FreeBSD Ports Collection * The FreeBSD Release Engineering Team Projects * Ceph on FreeBSD * OpenBSM * Porting Software to CloudABI: Sandboxed Bitcoin! * Support for eMMC Flash and Faster SD Card Modes * TrustedBSD Kernel * FreeBSD on Hyper-V and Azure * Intel 10G and 40G Network Driver Updates * Linuxulator * MMC Stack Using the CAM Framework * pNFS Server Plan B Architectures * 64-bit PowerPC Book-E Support * FreeBSD on Marvell Armada38x * FreeBSD/s390x Attempt III Ports * MySQL * Rust Documentation * The FreeBSD Dutch Documentation Project __________________________________________________________________ FreeBSD Team Reports The FreeBSD Core Team Contact: FreeBSD Core Team <core_at_FreeBSD.org> Core's primary function is to ensure the long-term viability of the FreeBSD project. A very large part of that is to ensure that the interactions between developers remain cordial, and consequently that the project appears welcoming to newcomers. Normally, most of Core's activities around this are done in private -- a quiet word in the right ear, some discrete peacemaking, occasional reading of the riot act. Most of the time, this is all that is necessary. Unfortunately, this quarter we had an instance where such private measures failed to achieve the desired result, and we ended up ejecting a developer. This developer is an extremely talented programmer and has made significant contributions to the Ports Collection. Despite this, portmgr found him to be sufficiently disruptive and abrasive that in their judgement, the project was better off overall to sever his connection to itself, and core backed them up in that. We are sorry that events came to this sad conclusion, but we remain convinced that this was a necessary step to safeguard the character of our community. In a more positive light, Core has been working on a proposal to recognise notable contributors to the FreeBSD project who are not (or perhaps not yet) suitable to be put forward as new committers. In addition to the usual routes of recognising people that write numbers of good bug reports or that supply patches or that volunteer to maintain ports, this will also allow recognition of people who contribute by such things as organising FreeBSD events or who promote FreeBSD through social media. A formal announcement of Core's proposal is imminent. During January, the core secretary held an exercise to contact all source committers who had been inactive for more than 18 months and persuade them to hand in their commit bits if they were not planning to resume working on FreeBSD in the near future. This is meant to be a routine function -- the "grim reaper" -- that aims to keep the list of people with the ability to commit pretty much in synchrony with the list of people that are actively committing. The regular process had fallen out of activity several years ago, and we needed to clear the decks before restarting. Ultimately, this resulted in some 20 developers-emeritus handing in their commit bits. No new commit bits were awarded during this quarter. Core is also taking soundings on producing a 10.4-RELEASE. This is not in the current plan, but a number of developers and important FreeBSD users would be keen to see it happen, given some of the work that has gone into the stable/10 branch since 10.3-RELEASE. On the other hand, this would represent an additional support burden for the Security Team, including maintaining versions of software that have been declared obsolete upstream, in particular OpenSSL. As an even-numbered release, 10.4-RELEASE would have a "normal" rather than an "extended" lifetime which means it should not result in extending the support lifetime of the stable/10 branch. In other news, Core arranged for the old and largely inactive marketing_at_FreeBSD.org mailing list to be wound up, and for any remaining activities to be transferred to the FreeBSD Foundation. Core also asked clusteradm to turn off Internet-wide access to the finger server on freefall.freebsd.org. Many developers have included details such as phone numbers into the GECOS field of their FreeBSD password database entries, and these would be revealed by the finger server -- details which are nowadays generally felt inadvisable to expose publicly. finger is still available internally within freefall.freebsd.org. Core recommends that GECOS data is limited to just your full name, and we have updated the standard "new committer" e-mail template to reflect that. Core is looking for new volunteers to help out with several of the teams that manage various aspects of the project. In particular, Postmaster and the Security Team are in need of new blood. Recruiting for a new member of the Security Team is well under way, but anyone interested in joining any of the teams is encouraged to make themselves known either to Core or directly to the teams concerned. __________________________________________________________________ The FreeBSD Foundation Links FreeBSD Foundation Website URL: https://www.FreeBSDFoundation.org/ Quarterly Newsletter URL: https://www.FreeBSDfoundation.org/wp-content/uploads/2017/03/FreeBSD-Foundation-Q1-2017-Update.pdf 2017 Storage Summit URL: https://wiki.FreeBSD.org/201702StorageSummit Contact: Deb Goodkin <deb_at_FreeBSDFoundation.org> The FreeBSD Foundation is a 501(c)(3) non-profit organization dedicated to supporting and promoting the FreeBSD Project and community worldwide. Funding comes from individual and corporate donations and is used to fund and manage software development projects, conferences and developer summits, and provide travel grants to FreeBSD contributors. The Foundation purchases and supports hardware to improve and maintain FreeBSD infrastructure; publishes marketing material to promote, educate, and advocate for the FreeBSD Project; facilitates collaboration between commercial vendors and FreeBSD developers; and finally, represents the FreeBSD Project in executing contracts, license agreements, and other legal arrangements that require a recognized legal entity. Our work is 100% funded by your donations. We kicked off the new year with some large contributions from Intel and NetApp, to help us raise over $400,000 last quarter! We engaged in discussions with new and old commercial users to help facilitate collaboration, explain how the Project works, and to ask for financial contributions to help us keep FreeBSD the innovative, secure, and reliable operating system they depend on. Please consider making a donation today! https://www.FreeBSDfoundation.org/donate/. The Foundation improves the FreeBSD operating system by employing our technical staff to maintain and improve critical kernel subsystems, add features and functionality, and fix problems. Our contributions also include funding separate project grants like the arm64 port, blacklistd access control daemon, and integration of VIMAGE support, to make sure FreeBSD remains a viable solution for research, education, computing, products and more. This quarter's project development highlights include: * 168 commits sponsored by the FreeBSD Foundation in the src tree (base system) development branch, across three staff members and four grant recipients/other developers. * Multiple funded grants, including the cfumass project, now committed to FreeBSD-HEAD, and improvements to the blacklistd daemon and FreeBSD/arm64 port. * Staff contributions including improvements to toolchain and build tool components, run time libraries, arm64, mips64 and 32- and 64-bit x86 architectures, release image build tooling, packaged base, and VM subsystem bug fixes. * Significant progress on the 64-bit inode project, which is nearly ready for commit. FreeBSD Advocacy and Education A large part of our efforts are dedicated to advocating for the Project. This includes promoting work being done by others with FreeBSD; producing advocacy literature to teach people about FreeBSD and help make the path to starting to use FreeBSD or contribute to the Project easier; and attending and getting other FreeBSD contributors to volunteer to run FreeBSD events, staff FreeBSD tables, and give FreeBSD presentations. Some of the highlights of our advocacy and education work over the last quarter: * Promoted FreeBSD at: FOSDEM, SCALE, AsiaBSDcon, and FOSSASIA * Promoted BSDCan, SCALE, USENIX LISA, vBSDcon and EuroBSDcon Calls for Participation * Promoted Google Summer of Code participation on social media and created a flyer for people to post at their universities * Published a New Faces of FreeBSD Story: Joseph Kong * Set up a Marketing Partnership with the USENIX Association and SNIA * Published and Promoted the Jan/Feb 2017 issue of the FreeBSD Journal: https://www.FreeBSDfoundation.org/journal/ * Published monthly Development Projects Updates on our blog * Secured a FreeBSD table at OSCON and promoted available discounts Conferences and Events The FreeBSD Foundation sponsors many conferences, events, and summits around the globe. These events can be BSD-related, open source, or technology events geared towards underrepresented groups. We support the FreeBSD-focused events to help provide a venue for sharing knowledge, to work together on projects, and to facilitate collaboration between developers and commercial users; this all helps provide a healthy ecosystem. We support the non-FreeBSD events to promote and raise awareness about FreeBSD, to increase the use of FreeBSD in different applications, and to recruit more contributors to the Project. We also sponsored and/or attended the following events last quarter: * FOSDEM FreeBSD developer summit (sponsor) * AsiaBSDCon -- Tokyo, Japan (sponsor) * Organized and ran the FreeBSD Storage Summit in Santa Clara, CA * Board member Philip Paeps gave a FreeBSD presentation at FOSSASIA * Attended FOSSASIA, FOSDEM, and SCALE Release Engineering The Foundation provides a full-time staff member to lead the release engineering efforts. This has provided timely and reliable releases over the last few years. Some highlights from last quarter include: * Continued the production of weekly development snapshots for the 12-CURRENT, 11-STABLE, and 10-STABLE branches. * Published the initial FreeBSD 11.1-RELEASE schedule to the Project website. Legal/FreeBSD IP The Foundation owns the FreeBSD trademarks, and it is our responsibility to protect them. We continued to review requests and grant permission to use the trademarks. Many more details about how we supported FreeBSD last quarter can be found in our Q1 newsletter! __________________________________________________________________ The FreeBSD Ports Collection Links About FreeBSD Ports URL: https://www.FreeBSD.org/ports/ Contributing to Ports URL: https://www.FreeBSD.org/doc/en_US.ISO8859-1/articles/contributing/ports-contributing.html FreeBSD Ports Monitoring URL: http://portsmon.FreeBSD.org/index.html Ports Management Team URL: https://www.FreeBSD.org/portmgr/index.html FreeBSD portmgr on Twitter (_at_FreeBSD_portmgr) URL: https://twitter.com/FreeBSD_portmgr/ FreeBSD Ports Management Team on Facebook URL: https://www.facebook.com/portmgr FreeBSD Ports Management Team on Google+ URL: https://plus.google.com/communities/108335846196454338383 Contact: René Ladan <portmgr-secretary_at_FreeBSD.org> Contact: FreeBSD Ports Management Team <portmgr_at_FreeBSD.org> The number of ports is currently just 500 short of 30,000. The current number of PRs is close to 2,400, of which 620 are unassigned. The last quarter saw 6656 commits from 167 comitters. Both the number of ports and the number of unassigned PRs have increased in the last quarter. In the last quarter, we welcomed 7 new committers: Eugene Grosbein (eugen), Johannes Dieterich (jmd), Larry Rosenman (ler), Mahdi Mokhtari (mmohki), Matthew Rezny (rezny), Tobias Kortkamp (tobik), and Vladimir Kondratyev (wulf). dumbbell_at_ was already a src committer and got an extension for the Ports Tree. We also welcomed back krion_at_ and miwi_at_. We took 6 bits in for safe-keeping: itetcu_at_, leeym_at_, mva_at_, olivierd_at_, pgollucci_at_, and sanpei_at_. There were no changes to the membership of portmgr. antoine_at_ worked on USES=samba to prepare for the removal of the long-outdated Samba 3.6 ports and replace them with modern versions. The new default versions are: FreePascal 3.0.2, Ruby 2.3, and Samba 4.4. A new variable USE_LOCALE was created to add the LANG and LC_ALL environment variables to all builds. Out-of-tree patches can now be added with the new EXTRA_PATCH_TREE variable. The error messages for invalid OPTIONS_SINGLE options were improved. Some of the major port updates last quarter were: pkg 1.10.1, linux c6_64, Firefox 52.0.2, Chromium 57.0.2987.110, GCC 4.9.4, Gnome 3.18.0, Xorg 1.18.4, Qt 4.8.7 and 5.7.1, and PHP 7.1. antoine_at_ ran 31 exp-runs to test version updates and under-the-hood changes. Open tasks: 1. The number of unassigned and open PRs is still growing, so if you have some spare time, please close some of those. __________________________________________________________________ The FreeBSD Release Engineering Team Links FreeBSD 11.1-RELEASE Schedule URL: https://www.freebsd.org/releases/11.1R/schedule.html FreeBSD development Snapshots URL: http://ftp.freebsd.org/pub/FreeBSD/snapshots/ISO-IMAGES/ Contact: FreeBSD Release Engineering Team <re_at_FreeBSD.org> The FreeBSD Release Engineering Team is responsible for setting and publishing release schedules for official project releases of FreeBSD, announcing code freezes, and maintaining the respective branches, among other things. The FreeBSD Release Engineering Team continued producing weekly development snapshots for the 12-CURRENT, 11-STABLE, and 10-STABLE branches. In addition, the FreeBSD 11.1-RELEASE schedule was added to the Project website. Please note, however, the schedule on the website is still subject to change. This project was sponsored by The FreeBSD Foundation. __________________________________________________________________ Projects Ceph on FreeBSD Links Ceph Main Site URL: http://ceph.com Main Repository URL: https://github.com/ceph/ceph My FreeBSD Fork URL: https://github.com/wjwithagen/ceph Contact: Willem Jan Withagen <wjw_at_digiware.nl> Ceph is a distributed object store and file system designed to provide excellent performance, reliability and scalability. * Object Storage Ceph provides seamless access to objects using native language bindings or radosgw, a REST interface that is compatible with applications written for S3 and Swift. * Block Storage Ceph's RADOS Block Device (RBD) provides access to block device images that are striped and replicated across the entire storage cluster. * File System Ceph provides a POSIX-compliant network file system that aims for high performance, large data storage, and maximum compatibility with legacy applications. I started looking into Ceph because the HAST solution with CARP and ggate did not really do what I was looking for. But I aim to run a Ceph storage cluster of storage nodes that are running ZFS. User stations would be running bhyve on RBD disks that are stored in Ceph. Compiling for FreeBSD will now build most of the tools available in Ceph. Notable progress since the last report: * The most important change is that a port has been submitted: net/ceph-devel. However, it does not yet contain ceph-fuse. * Regular updates to the ceph-devel port are expected, with the next one coming in April. * ceph-fuse works, allowing one to mount a CephFS filesystem on a FreeBSD system and perform normal operations. * ceph-disk prepare and activate work for FileStore on ZFS, allowing for easy creation of OSDs. * RBD is actually buildable and can be used to manage RADOS BLOCK DEVICEs. * Most of the awkward dependencies on Linux-isms are deleted -- only /bin/bash is here to stay. To get things running on a FreeBSD system, run pkg install net/ceph-devel or clone https://github.com/wjwithagen/ceph and build manually by running ./do_freebsd.sh in the checkout root. Parts not (yet) included: * KRBD: Kernel Rados Block Devices are implemented in the Linux kernel, but not yet in the FreeBSD kernel. It is possible that ggated could be used as a template, since it does similar things, just between two disks. It also has a userspace counterpart, which could ease development. * BlueStore: FreeBSD and Linux have different AIO APIs, and that incompatibility needs to resolved somehow. Additionally, there is discussion in FreeBSD about aio_cancel not working for all devicetypes. * CephFS as native filesystem: though ceph-fuse works, it can be advantageous to have an in-kernel implementation for heavy workloads. Cython tries to access an internal field in struct dirent, which does not compile. Open tasks: 1. Run integration tests to see if the FreeBSD daemons will work with a Linux Ceph platform. 2. Compile and test the userspace RBD (Rados Block Device). This currently works but testing has been limitted. 3. Investigate and see if an in-kernel RBD device could be developed akin to ggate. 4. Investigate the keystore, which can be embedded in the kernel on Linux and currently prevents building Cephfs and some other parts. The first question is whether it is really required, or only KRBD requires it. 5. Scheduler information is not used at the moment, because the schedulers work rather differently between Linux and FreeBSD. But at a certain point in time, this will need some attention (in src/common/Thread.cc). 6. Improve the FreeBSD init scripts in the Ceph stack, both for testing purposes and for running Ceph on production machines. Work on ceph-disk and ceph-deploy to make it more FreeBSD- and ZFS-compatible. 7. Build a test cluster and start running some of the teuthology integration tests on it. Teuthology wants to build its own libvirt and that does not quite work with all the packages FreeBSD already has in place. There are many details to work out here. 8. Design a vitual disk implementation that can be used with bhyve and attached to an RBD image. __________________________________________________________________ OpenBSM Links OpenBSM: Open Source Basic Security Module (BSM) Audit Implementation URL: http://www.openbsm.org OpenBSM on GitHub URL: https://github.com/openbsm/openbsm FreeBSD Audit Handbook Chapter URL: https://www.FreeBSD.org/doc/en_US.ISO8859-1/books/handbook/audit.html DTrace Audit Provider URL: https://reviews.FreeBSD.org/D10149 DARPA CADETS project URL: https://www.cl.cam.ac.uk/research/security/cadets/ TODO List on GitHub URL: https://github.com/openbsm/openbsm/blob/master/TODO Contact: Christian Brueffer <brueffer_at_FreeBSD.org> Contact: Robert Watson <rwatson_at_FreeBSD.org> Contact: TrustedBSD audit mailing list <trustedbsd-audit_at_TrustedBSD.org> OpenBSM is a BSD-licensed implementation of Sun's Basic Security Module (BSM) API and file format. It is the userspace side of the CAPP Audit implementations in FreeBSD and Mac OS X. Additionally, the audit trail processing tools are expected to work on Linux. During this quarter, experimental support for UUIDs in BSM trails was added to OpenBSM. A DTrace audit provider using this functionality has been developed as part of the DARPA CADETS project and is in review (https://reviews.FreeBSD.org/D10149). In the OpenBSM GitHub repository, support for Coverity static analysis was added via TravisCI. Additionally, the OpenBSM 1.2-alpha5 release has been merged into the FreeBSD HEAD branch. This project was sponsored by DARPA/AFRL (in part). Open tasks: 1. Test the latest release on different versions of FreeBSD, Mac OS X and Linux. Testing on the latest versions of Mac OS X would be particularly appreciated. 2. Fix problems that have been reported via GitHub and the FreeBSD bug tracker. 3. Implement the features mentioned in the TODO list on GitHub. __________________________________________________________________ Porting Software to CloudABI: Sandboxed Bitcoin! Links How to Use CloudABI on FreeBSD URL: https://nuxi.nl/cloudabi/freebsd/ LevelDB for CloudABI URL: https://nuxi.nl/blog/2017/02/18/porting-leveldb-to-cloudabi.html Memcached for CloudABI URL: https://nuxi.nl/blog/2017/03/15/sandboxed-memcached.html Bitcoin for CloudABI URL: https://laanwj.github.io/2017/03/02/porting-bitcoin-core-to-cloudabi.html Contact: Ed Schouten <ed_at_FreeBSD.org> CloudABI is a framework that allows you to develop strongly sandboxed applications a lot more easily. It is a programming environment that exclusively uses FreeBSD's Capsicum facilities. Any features incompatible with Capsicum have been removed entirely, which means that it is easier to determine how code needs to be adjusted to behave correctly while sandboxed. In essence, you only need to patch up the code until it builds. Last year we have managed to port a lot of exciting libraries over to CloudABI. Highlights include sandboxing aware versions of Boost and LevelDB. Now that these libraries are readily available, we are at the point where we can shift our focus towards porting full applications. In late February one of the lead developers of the Bitcoin reference implementation got in touch, as he is very interested in creating a copy of Bitcoin that is better protected against security bugs. You do not want a security bug in the networking/consensus code to allow an attacker to steal coins from your local wallet! As I think that this is a use case that demonstrates the strength of CloudABI well, I've made addressing any issues reported by the Bitcoin developers a top priority. Once the Bitcoin port is complete, we want to provide binary packages of it as well. This project was sponsored by Nuxi, the Netherlands. Open tasks: 1. Though getting Bitcoin to work is pretty awesome, don't let that distract us from porting other pieces of software over as well! Are you the maintainer of a piece of software that could benefit from sandboxing? Be sure to try building it using the CloudABI toolchain! 2. One of the pieces of software that got ported over to CloudABI some time ago is Memcached. Are you a user of Memcached? If so, feel free to give the sandboxed version of Memcached for CloudABI a try! 3. So far, CloudABI can be used to run software written in C, C++ and Python. Would you like to see any other programming language work on CloudABI as well? Be sure to help out! __________________________________________________________________ Support for eMMC Flash and Faster SD Card Modes Contact: Marius Strobl <marius_at_FreeBSD.org> In r315430, support for eMMC partitions has been added to mmc(4) and mmcsd(4) in FreeBSD 12. Besides the user data area, i.e., the default partition, eMMC v4.41 and later devices can additionally provide up to: * 1 enhanced user data area partition * 2 boot partitions * 1 RPMB (Replay Protected Memory Block) partition * 4 general purpose partitions (optionally with an enhanced or extended attribute) Apart from simply subdividing eMMC flash devices or having UEFI code in the boot partition, as is done on some Intel NUCs, another use case for partition support is the activation of pseudo-SLC mode, which manufacturers of eMMC chips typically associate with the enhanced user data area and/or the "enhanced" attribute of general purpose partitions. In order to be able to partition eMMC devices, r315430 also added a Linux-compatible ioctl(2) interface to mmcsd(4). This allows the use of the GNU mmc-utils (found in ports as sysutils/mmc-utils) on FreeBSD. Besides partitioning eMMC devices, the mmc tool can also be used to query for lifetime estimates and pre-EOL information of eMMC flash, as well as to query some basic information from SD cards. CAVEAT EMPTOR: Partitioning eMMC devices is a one-time operation. Additionally, in order to make eMMC flash devices more usable, support for DDR (Dual Data Rate) bus speed mode at a maximum of 52 MHz (DDR52) has been added to mmc(4) and sdhci(4) in r315598, which will appear in FreeBSD 12. Compared to high speed mode (the previous maximum) at 52 MHz, DDR52 mode increases the performance of the tested eMMC chips from ~45 MB/s to ~80 MB/s. So far, support for DDR52 mode has been enabled for the eMMC controllers found in the Intel Apollo Lake, Bay Trail and Braswell chipsets. Note, however, that the eMMC and SDHCI controllers of the Apollo Lake variant occasionally lock up due to a silicon bug (which is independent of running in DDR52 mode). The only viable workaround for that problem appears to be the implementation of support for ADMA2 mode in sdhci(4) (currently, sdhci(4) supports only the encumbered SDMA mode or no DMA at all). However, r315598 also brought in infrastructure and a fair amount of code for using even faster transfer modes with eMMC devices and SD cards respectively, i.e., up to HS400ES with eMMC and the UHS-I modes up to SDR104 with SD cards. The intent is to merge these changes back to FreeBSD 10 and 11. Open tasks: 1. Add support for eMMC HS200, HS400, and HS400ES transfer modes. 2. Add support for SD card UHS-I transfer modes (SDR12 to SDR104). 3. Make mmcsd(4) more robust and correctly follow the relevant specifications for existing features, e.g., calculate and handle erase timeouts, do a SEND_STATUS when CMD6 is invoked with an R1B response to the extent not already fixed as part of r315430, get the remainder of the existing code to properly check and handle return codes, etc. __________________________________________________________________ TrustedBSD Links TrustedBSD Website URL: http://www.trustedbsd.org TrustedBSD on GitHub URL: https://github.com/trustedbsd Contact: Christian Brueffer <brueffer_at_FreeBSD.org> Contact: Robert Watson <rwatson_at_FreeBSD.org> Contact: TrustedBSD announce mailing list <trustedbsd-announce_at_TrustedBSD.org> The TrustedBSD Project is an open-source community developing advanced security features for the open-source FreeBSD operating system. Started in April 2000, the project developed support for extended attributes, access control lists (ACLs), UFS2, OpenPAM, security event auditing, OpenBSM, a flexible kernel access control framework, mandatory access control, and the GEOM storage layer. The results of this work may be found not just in FreeBSD, but also NetBSD, OpenBSD, Linux, and Apple's Mac OS X and iOS operating systems. Today, the project continues to maintain and enhance these mature features in FreeBSD. During this quarter, the TrustedBSD project transitioned from the FreeBSD Perforce server to GitHub. This was made possible by Alexis Sarghel, who owned the user "trustedbsd" on GitHub and graciously transferred this account to the TrustedBSD project. To date, the repositories hosting the TrustedBSD website and the SEBSD repository have been moved. __________________________________________________________________ Kernel FreeBSD on Hyper-V and Azure Links FreeBSD Virtual Machines on Microsoft Hyper-V URL: https://wiki.FreeBSD.org/HyperV Supported Linux and FreeBSD Virtual Machines for Hyper-V on Windows URL: https://technet.microsoft.com/en-us/library/dn531030.aspx Contact: Sepherosa Ziehau <sepherosa_at_gmail.com> Contact: Hongjiang Zhang <honzhan_at_microsoft.com> Contact: Dexuan Cui <decui_at_microsoft.com> Contact: Kylie Liang <kyliel_at_microsoft.com> SR-IOV support for NICs is implemented. So far, we have only tested with the Mellanox ConnectX-3 VF card, which works despite some issues (Bug 216493: https://bugs.FreeBSD.org/bugzilla/show_bug.cgi?id=216493). Updates for UEFI VMs (i.e., Hyper-V Generation 2 VMs): 1. After the loader issue (Bug 211746) is fixed, UEFI VMs can now boot with Secure Boot disabled; 2. A synthetic keyboard driver has been added. Currently it is only in HEAD, but MFCs to stable/10 and stable/11 are planned to occur soon; 3. A SCSI DVD detection issue (Bug 218248) was fixed. Without the fix, the VM would fail to boot. This project was sponsored by Microsoft. __________________________________________________________________ Intel 10G and 40G Network Driver Updates Links Commit adding X553 ix/ixv Support for iflib URL: https://reviews.FreeBSD.org/D9851 Commit converting ixgbe to iflib URL: https://reviews.FreeBSD.org/D5213 Contact: Jeb Cramer <jeb.j.cramer_at_intel.com> Contact: Eric Joyner <eric.joyner_at_intel.com> Contact: Krzysztof Galazka <krzysztof.galazka_at_intel.com> This driver update is for the Intel ix/ixv and ixl/ixlv network drivers, and includes support for several new hardware releases. ix/ixv: * Added support for X553 SoC devices based on the Denverton platform ixl/ixlv: * Added support for X722 10G SoC devices based on the Lewisburg chipset * Added an interface for the upcoming iWarp driver for X722 devices * Added support for XXV710 25G devices Open tasks: 1. ix/ixv iflib support is currently under review in Phabricator. It will be refactored to include D5213. 2. Initial work for ixl/ixlv iflib support is in progress. __________________________________________________________________ Linuxulator Contact: Dimitry Chagin <dchagin_at_FreeBSD.org> Contact: Edward Tomasz Napierała <trasz_at_FreeBSD.org> Contact: Mahdi Mokhtari <mmokhi_at_FreeBSD.org> In this quarter, we are pleased to announce two (of many) works achieved in the Linuxulator. We added a new placeholder marker UNIMPLEMENTED to accompany the previously existing DUMMY, for distinguishing syscalls that the Linux kernel itself does not implement from those that we currently do not implement. Now our linux_dummy.c is clearer for newcomers to follow, and they will quickly know which areas they can start working on. Support for two new syscalls, preadv and pwritev, was added to the Linuxulator. Open tasks: 1. We plan to implement the execveat syscall for the native FreeBSD syscall table and then port/wrap it for use in the Linuxulator. __________________________________________________________________ MMC Stack Using the CAM Framework Links Project Information URL: https://bakulin.de/freebsd/mmccam.html Source Code URL: https://github.com/kibab/FreeBSD/tree/mmccam-collapsed-commits Contact: Ilya Bakulin <kibab_at_FreeBSD.org> The goal of this project is to reimplement the existing MMC/SD stack using the CAM framework. This will permit utilizing the well-tested CAM locking model and debugging features. It will also be possible to process interrupts generated by the inserted card, which is a prerequisite for implementing the SDIO interface. SDIO support is necessary for communicating with the WiFi/BT modules found on many development boards, such as the Raspberry Pi 3. Another feature that the new stack will have is support for sending SD commands from userland applications using cam(3). This will allow for building device drivers in userland and make debugging much easier. The new stack is able to attach to an SD card and bring it to an operational state so that it is possible to read and write to the card. The stack has been tested to work on the Beaglebone Black and Wandboard Quad platforms. Currently the code is being prepared for inclusion in the FreeBSD source tree. cam(3) is being extended to support SDIO-specific functions (reading registers, managing interrupts, etc.). Open tasks: 1. Integrate the code into FreeBSD HEAD to facilitate testing. 2. Begin writing a driver for Broadcom-based WLAN chips (found on the Raspberry Pi 3 and Wandboard). 3. Begin writing a driver for Marvell-based WLAN chips (found on the GlobalScale Dreamplug and some Chromebooks). __________________________________________________________________ pNFS Server Plan B Contact: Rick Macklem <rmacklem_at_FreeBSD.org> Parallel NFS (pNFS) is an extension to the NFSv4 protocol that allows for file accesses within a single logical mount to be performed against multiple file servers, with the potential for data access to occur in parallel. The pNFS "layout" in use specifies how the division occurs, with metadata operations occurring against the main server, and bulk data operations (read/write/setattr/etc.) occurring via a layout-specific scheme between the client and data servers. My first attempt at a pNFS server using GlusterFS was a dud. It worked, but performance was so poor that it was not usable. This attempt that I call "Plan B", only uses FreeBSD, with one FreeBSD server handling the metadata operations and multiple FreeBSD servers configured to serve data. An NFSv4.1 client that supports the pNFS File Layout will be able to read and write to the data servers directly, spreading out the RPC load and allowing growth beyond that of what a single FreeBSD NFS server could achieve. There is no support for the Flex Files Layout or mirroring at this time. I hope to use the Flex Files Layout to add mirroring support over the next year or so. Striping is also not supported, but I have no plans for implementing it at the moment. Plan B is working quite well now and should be available for testing by the end of April. I will announce how to do this on the freebsd-fs_at_FreeBSD.org mailing list when it is available. Open tasks: 1. Testing by others will be needed, once it is available. __________________________________________________________________ Architectures 64-bit PowerPC Book-E Support Contact: Justin Hibbits <jhibbits_at_FreeBSD.org> The Book-E platform target now supports 64-bit mode ("powerpc64"). It includes a 63-bit address space split, but the page table directory list uses holes to expand to the full address space, leaving gaps in the address space where page mappings are repeated. This may change in the future. As with the AIM powerpc64 port, Book-E supports running powerpc (32-bit) binaries as well, and has even been tested with a 32-bit init and 64-bit shell. Several of the SoC drivers are supported, however, the dTSEC ethernet controller is not yet supported. Work is ongoing to support it. A QORIQ64 config is included, targeting the P5 and T* series SoCs from Freescale. Thanks to Juniper Networks for providing patches against an older internally maintained FreeBSD version, which enabled this porting effort, and for providing historical context for quirks of the pmap changes. Open tasks: 1. Port the dTSEC driver to 64-bit. There are assumptions in the reference driver of operating in a 32-bit environment. It may be easier to port the Linux driver instead, which would also give ARM support for this ethernet controller. 2. Take advantage of pointer alignment to squeeze more bits out of the page tables; it should be possible to squeeze at least 3 more bits out, one at each level. __________________________________________________________________ FreeBSD on Marvell Armada38x Contact: Marcin Wojtas <mw_at_semihalf.com> Contact: Zbigniew Bodek <zbb_at_semihalf.com> Final testing and productionization of support for the Marvell Armada38x platform is under way. The rebase and cleanup is going well, with patches functioning on top of HEAD and ready for upstreaming. Specific tasks completed include: * Platform benchmarking and low-level optimizations (internal bus, L1/L2 cache prefetch) -- already submitted) * Enable the PL310 L2 cache controller -- currently under review * NETA tests, optimizations and PHY-handling rework * e6000sw PHY handling rework and fixes * Fix and enable performance counter support * Fix timers and extract watchdog support to a separate driver * Crypto driver fixes -- merged * AHCI controller support -- merged * Thermal driver -- merged * Merge additional support for new boards (SolidRun ClearFog and DB-88F6285-AP) This project was sponsored by Stormshield, and Semihalf. Open tasks: 1. Submit the remaining fixes and drivers. __________________________________________________________________ FreeBSD/s390x Attempt III Contact: Bjoern A. Zeeb <bz_at_FreeBSD.org> A long time ago, in the FreeBSD 5 times, there was an initial port of FreeBSD to s390 (32bit) and s390x (64bit) which booted past init on good days in an emulator. As an attempt to revive the s390x/systemz efforts I started to get FreeBSD s390x to build with clang/llvm 3.9. At this time, it is possible to build world and a GENERIC kernel skeleton (not doing anything yet) using external binutils. The primary idea of this initial work was to allow for incremental addition of the necessary architecture-specific code. Having the build framework in place will allow third-party developers to simply type make, as they are willing to contribute to the port without having to know FreeBSD build specifics. After some cleanup and further updates to a more recent HEAD I am planning to push the current work to a public repo to facilitate collaboration. Open tasks: 1. Write a wiki page with per-architecture specific tasks that need to be done, based on the current work and the experience from arm64 and riscv. 2. Implement both the userspace and kernel per-architecture gaps. 3. Figure out a way to get access to IBM's zPDT or better emulators to ease implementation, testing, and debugging. __________________________________________________________________ Ports MySQL Links MySQL80 Overview URL: https://www.mysql.com/why-mysql/presentations/mysql-80-overview/ MySQL80 InnoDB New Features URL: https://www.mysql.com/why-mysql/presentations/innodb-whats-new-mysql-80/ Contact: Mahdi Mokhtari <mmokhi_at_FreeBSD.org> Contact: Mark Felder <feld_at_FreeBSD.org> This quarter a new -dev version of MySQL landed in the Ports Collection, MySQL 8.0. It introduces many new features, though we had to (re)-patch parts of it which were merged by MySQL from MySQL5.7. We also updated MySQL 5.6 to its latest version and closed many PRs related to it, mostly relating to using FreeBSD-provided ports for libraries instead of the bundled copies. And of course there were plenty of security updates. We can also report that the problem of having to specify ${mysql_optfile}, which some people encountered while using MySQL, is now considered to be solved in all MySQL versions: 5.6, 5.7, and 8.0. Now the init script will search all default locations, for backwards compatibility with the variety of locations used for configuration files, before it gives up and reports an error. Open tasks: 1. Test the new version and report problems. __________________________________________________________________ Rust Links Wiki Portal URL: https://wiki.FreeBSD.org/Rust Guide to Bootstraping Rust on FreeBSD URL: https://gist.github.com/dumbbell/b587da50ef014078da9e732a4331ebad Bug Report to Track Progress on Bootstrapping URL: https://bugs.FreeBSD.org/bugzilla/show_bug.cgi?id=216143 Contact: Jean-Sébastien Pédron <dumbbell_at_FreeBSD.org> Contact: Thomas Zander <riggs_at_FreeBSD.org> In the Ports Collection, Rust was updated to 1.16.0 and Cargo to 0.17.0, the latest versions at the time of this writing. lang/rust-nightly was also updated to a snapshot from February and it is now enabled on i386. It is lagging a bit behind upstream, but Rustup works nicely on FreeBSD if you need to try any versions/channels of Rust. Work has started to bootstrap Rust on non-x86 architectures. Patches to add FreeBSD/aarch64 support were submitted and accepted upstream. FreeBSD/sparc64 is in progress. The lang/rust-nightly port is also being adapted to compile natively on FreeBSD/aarch64. This work is critical, in particular because Firefox will shortly require Rust. If you want to help, please refer to the guide linked above. The compiler, rustc, is crashing sometimes when there is a compilation error. Therefore, there is a bit of work to do to improve stability. There is some code duplication between lang/rust* and devel/cargo. Those Makefiles deserve a bit of cleanup. It might be useful to create a USES=rust Makefile helper. Open tasks: 1. Bootstrap Rust on more platforms. 2. Investigate compiler crashes. 3. Create a USES=rust Makefile helper and simplify the Rust and Cargo ports. 4. Investigate how to speed up lang/rust* compilation time. __________________________________________________________________ Documentation The FreeBSD Dutch Documentation Project Links The Dutch Translation Project URL: https://www.FreeBSD.org/docproj/translations.html#dutch Contact: René Ladan <rene_at_FreeBSD.org> Contact: Remko Lodder <remko_at_FreeBSD.org> Work has started on an initial translation of the FreeBSD Handbook to the Dutch language via the "po" system. While we have an (outdated) version of the Handbook available via the older XML files, we are now trying to get back into shape with the PO file. Rene started working on two articles already and did some translation of strings for the FDP-Primer, while Remko has started working on the Handbook. If you think you can assist with either, please send Rene and Remko an email so that we can start coordinating work. In addition, since we have a translation set already from the XML files, it would be interesting to see whether we can merge them easily into the PO structure. If you have ideas on that, contact us a.s.a.p. This project was sponsored by Snow B.V. (in part). Open tasks: 1. Identify a way to merge the current XML translations into the nl_NL.po files. 2. Merge the translations into the .po files. 3. Update the remaining open items into the po files. 4. Remove the old/outdated translation files from the main repo and use the po and book.xml files to generate the Dutch handbook and other files. 5. Identify whether we can also translate the htdocs pages via the PO system. __________________________________________________________________
This archive was generated by hypermail 2.4.0 : Wed May 19 2021 - 11:41:11 UTC