FreeBSD Status Report Introduction This report covers FreeBSD related projects between June and October 2006. This includes the conclusion of this year's Google Summer of Code with 13 successful students. Some of last year's and the current SoC participants have meanwhile joined the committer ranks, kept working on their projects, and improving FreeBSD in general. This year's EuroBSDCon in Milan, Italy has meanwhile published an exciting program. Many developers will be there to discuss these current and future projects at the Developer Summit prior the conference. Next year's conference calendar has a new entry - in addition to the now well established BSDCan in Ottawa - AsiaBSDCon will take place in Tokyo at the begining of March. As we are closing in on FreeBSD 6.2 release many bugs are being fixed and new features have been MFCed. On the other hand a lot of the projects below already are focusing on FreeBSD 7.0 and promise a lot of exciting news and features to come. Thanks to all the reporters for the excellent work! We hope you enjoy reading. _________________________________________________________________ Google Summer of Code * Analyze and Improve the Interrupt Handling Infrastructure * Bundled PXE Installer * Gvirstor * IPv6 Stack Vulnerabilities * Jail Resource Limits * Nss-LDAP importing and nsswitch subsystem improvement * Porting the seref policy and setools to SEBSD * Porting Xen to FreeBSD * SNMP monitoring (BSNMP) * Summer of Code Summary * Update of the Linux compatibility environment in the kernel Projects * CScout on the FreeBSD Source Code Base * DTrace * Embedded FreeBSD * FreeSBIE * GJournal * iSCSI Initiator * Porting ZFS to FreeBSD * Summer of FreeBSD security development * TrustedBSD Audit * USB FreeBSD Team Reports * FreeBSD Security Officer and Security Team * Ports Collection * Release Engineering * The FreeBSD Foundation Network Infrastructure * Bridge Spanning Tree Protocol Improvements * FAST_IPSEC Upgrade * Highly improved implementations of sendfile(2), sosend_*() and soreceive_stream() * SCTP Integration * TSO - TCP Segmentation Offload committed Kernel * Gvinum improvements * MMC/SD Support * Sound Subsystem Improvements Documentation * Chinese (Simplified) Project * Hungarian translation of the webpages Userland Programs * Libelf * OpenBSD dhclient Architectures * CPU Microcode Update Software * FreeBSD/arm on Atmel AT91RM9200 * Sun Niagara port * Xen Port Ports * Enlightenment DR17 support in the ports tree * FreshPorts * Improving FreeBSD Ports Collection Infrastructure * OCaml language support in ports Miscellaneous * AsiaBSDCon 2007 * BSDCan 2007 * EuroBSDCon 2006 * FreeBSD Multimedia Resources List _________________________________________________________________ Analyze and Improve the Interrupt Handling Infrastructure URL: http://wikitest.freebsd.org/Interrupts Contact: Paolo Pisati <pisati_at_FreeBSD.org> Contact: John Baldwin <jhb_at_FreeBSD.org> This project consisted in the improvement of the Interrupt Handling System in FreeBSD: while retaining backward compatibility with the previous models (FAST and ITHREAD), a new method called 'Interrupt filtering' was added. With interrupt filtering, the interrupt handler is divided into 2 parts: the filter (that checks if the actual interrupt belong to this device) and the ithread (that is scheduled in case some blocking work has to be done). The main benefits of interrupt filtering are: * Feedback from filters (the system finally knows if any handler has serviced an interrupt or not, and can react consequently). * Lower latency/overhead for shared interrupt line. * Previous experiments with interrupt filtering showed an increase in performance against the plain ithread model Moreover, during the development of interrupt filtering, some MD dependent code was converted into MI code, PPC was fixed to support multiple FAST handlers per line and an interrupt stray storm detection logic was added. While the framework is done, there are still machine dependent bits to be written (the support for ppc, sparc64, arm and itanium has to be written/reviewed) and a serious analysis of the performance of this model against the previous one is a work-in-progress _________________________________________________________________ AsiaBSDCon 2007 URL: http://www.asiabsdcon.org/ Contact: Hiroki Sato <hrs_at_freebsd.org> Contact: George Neville-Neil <gnn_at_freebsd.org> Contact: <secretary_at_asiabsdcon.org> Web site is up and we're soliciting papers and presentations. Some tutorials are already scheduled. Email secretary_at_asibsdcon.org if you have questions or submissions. Open tasks: 1. Send in more papers! _________________________________________________________________ Bridge Spanning Tree Protocol Improvements Contact: Andrew Thompson <thompsa_at_FreeBSD.org> Work is almost finished to implement the Rapid Spanning Tree Protocol (RSTP) which supersedes Spanning Tree Protocol (STP). RSTP has a much faster link failover time of around one second compared to 30-60 seconds for STP, this is very important on modern networks. The code will be posted shortly for testing and feedback. _________________________________________________________________ BSDCan 2007 URL: http://www.bsdcan.org/ Contact: Dan Langille <dan_at_langille.org> The dates for BSDCan 2007 has been set: 11-12 May 2007. As is usual, BSDCan will be held at University of Ottawa, with two days of tutorials prior to the conference starting. The call for papers will go out in mid December. Start thinking about your submissions now! _________________________________________________________________ Bundled PXE Installer URL: http://wikitest.freebsd.org/MarkusBoelter Contact: Markus Boelter <m_at_FreeBSD.org> Contact: Paul Saab <ps_at_FreeBSD.org> For me, the Google Summer of Code was a new and very exciting experience. I got actively involved in doing Open Source Software and giving something back to the community. Facing some challenges within the project forced me to look behind the scenery of FreeBSD. The result was a better understanding of the overall project. Working with a lot of developers directly also gave a very special spirit to the Google Summer of Code. I really enjoyed the time and will continue to work on the project after the deadline. For me, it was a great chance to get involved in active development and not just some scripts and hacks at home. Getting paid for the work was just a small part of the overall feeling. Thanks to the people at the FreeBSD Project and Google for the really, really great time! _________________________________________________________________ Chinese (Simplified) Project URL: http://cnsnap.cn.FreeBSD.org/zh_CN/ URL: http://cnsnap.cn.FreeBSD.org/doc/zh_CN.GB2312/ Contact: Xin LI <delphij_at_FreeBSD.org> In the previous quarter we primarily focused on overall quality of the translation rather than just increasing the number of translations, and we have strived to make sure that these translated stuff are up-to-date with their English revisions. Also, we have merged the translated website into the central repository. In the next quarter we will focus on developing documentation that will help to attract more developers. Open tasks: 1. Translate more development related documentation. 2. Review more of the currently translated documentation. _________________________________________________________________ CPU Microcode Update Software Contact: Stanislav Sedov <stas_at_FreeBSD.org> Last month I was working on a driver/module to update the microcode of Intel or AMD CPUs that support having their microcode updated. As you might know these processors are microcode-driven and this firmware can be updated. Intel(R) often releases microcode updates, and AMD(R) updates can be found in BIOS programs. The work is almost finished now, I just need to find a bit of time to test it on AMD64 systems and perform some code cleanup. The driver also provide a way for userland programs to access the Machine Specific Registers (MSR) and CPUID info for a certain cpu. This will allow some programs like x86info to provide more accurate information about cpus in SMP systems and make assumptions based on the contents of the MSR. Thanks to John Baldwin, Kostik Belousov, John-Mark Gurney and Divacky Roman for helping during development. Open tasks: 1. Perform testing on the AMD64-based systems. 2. Write manpage. 3. Code cleanup/checks. _________________________________________________________________ CScout on the FreeBSD Source Code Base URL: http://wikitest.freebsd.org/CScout Contact: Diomidis Spinellis <dds_at_FreeBSD.org> CScout is a refactoring editor and source code browser for collections of C code. The aim of the project is to make it easy for FreeBSD developers to use CScout and to improve the FreeBSD source code quality through CScout-based queries and refactorings. CScout was first applied to the FreeBSD kernel in 2003. Its application at that point involved substantial tinkering with the build system. The version released in October 2006 makes the running of CScout on the three Tier-1 architectures a fairly straightforward procedure. The current version can also draw a number of call graphs; this might help developers better understand foreign code. Open tasks: 1. Use CScout to locate problematic code areas (for example unused or too liberaly visible objects). 2. Use CScout to globaly rename identifiers in a more consistent fashion. 3. Apply CScout to the userland code. 4. Identify CScout extensions that would help us improve the quality of our code. 5. Arrange for the continous availability of a live CScout kernel session on the current version of the source code. _________________________________________________________________ DTrace Contact: John Birrell <jb_at_freebsd.org> Progress this month has been limited due to my sea-change, moving house to the country. Sun's OpenSolaris developers have followed through and released the DTrace test suite as part of the OpenSolaris distribution. jkoshy_at_'s work on libbsdelf is nearing feature completion for DTrace and will make life easier in FreeBSD for DTrace, given that we have more architectures to support than Sun has. The FreeBSD project has made available a dual processor AMD64 machine for DTrace porting. I am currently working through the diffs between the DTrace project in P4 and -current, committing files to -current if they are ready, _________________________________________________________________ Embedded FreeBSD URL: http://www.embeddedfreebsd.org/ Contact: George Neville-Neil <gnn_at_freebsd.org> Moved the HTML pages into the project CVS tree. Open tasks: 1. Setup the web site to be served from projects CVS so that it can be updated by others. 2. Complete the ARM port. 3. Work on the MIPS port. 4. Update the documentation to include common tasks for embedded engineers. _________________________________________________________________ Enlightenment DR17 support in the ports tree Contact: Stanislav Sedov <stas_at_FreeBSD.org> Integration of the new innovative e17 window manager into the ports tree is almost completed. A lot of new e17-related applications was ported, all old ports were updated to the latest stable cvs snapshot. The special framework (bsd.efl.mk) was created to support the whole thing and simplify the creation of dependent ports. I'll commit the changes in the days before the ports freeze. Thanks to Sergey Matveychuk (sem_at_) for providing a machine to place CVS snapshots on. Without his help it will be impossible. Open tasks: 1. Port Entrance (xdm-like app, but very appealing). 2. Port Net and Wlan e17 module. 3. Develop FreeBSD-specific e17 apps/modules to use The Ports Collection, system configs, etc. _________________________________________________________________ EuroBSDCon 2006 URL: http://www.eurobsdcon.org/ URL: http://www.eurobsdcon.org/register/ Contact: EuroBSDCon Organizing Committee <info_at_eurobsdcon.org> EuroBSDCon 2006 is taking place in Milan (Italy), from the 10th to the 12th of November. EuroBSDCon represents the biggest gathering for BSD developers from the old continent, as well as users and passionates from around the World. It is also a chance to share experiences, know-how, and cultures. The program is rich in talks about FreeBSD, with topics ranging from "How the FreeBSD ports collection works" to "Interrupt Filtering in FreeBSD". This means that both the novice and the hacker can enjoy the conference. Registration is open. The EuroBSDCon Organizing Committee hopes to see you in Milan. _________________________________________________________________ FAST_IPSEC Upgrade URL: www.freebsd.org/~gnn/fast_ipv6.patch Contact: George Neville-Neil <gnn_at_freebsd.org> Contact: Bjoern Zeeb <bz_at_freebsd.org> First working version of code. Does not pass all TAHI tests, but does pass packets correctly and does not panic. Open tasks: 1. More testing of the patch needed. _________________________________________________________________ FreeBSD Multimedia Resources List URL: http://www.mavetju.org/unix/multimedia.php URL: http://www.mavetju.org/unix/multimedia-rss.php Contact: Edwin Groothuis <edwin_at_FreeBSD.org> I have setup the FreeBSD Multimedia Resources List, a one-stop-shop for FreeBSD related podcasts, vodcasts and audio/video resources. Hopefully this list will make it easier for people to find and keep up to date with these recordings. The overview is available as a normal HTML page and as an XML/RSS feed. The ultimate goal is to have this list to reside under the www.FreeBSD.org umbrella. _________________________________________________________________ FreeBSD Security Officer and Security Team URL: http://www.freebsd.org/security/ URL: http://www.freebsd.org/doc/en_US.ISO8859-1/articles/contributors/staff -listing.html#STAFF-SECTEAM URL: http://vuxml.freebsd.org/ Contact: Security Officer <security-officer_at_FreeBSD.org> Contact: Security Team <security-team_at_FreeBSD.org> In the time since the last status report, six security advisories have been issued concerning problems in the base system of FreeBSD; of these, five problems were in "contributed" code, while one was in code maintained within FreeBSD. The Vulnerabilities and Exposures Markup Language (VuXML) document has continued to be updated by the Security Team and Ports Committers documenting new vulnerabilities in the FreeBSD Ports Collection; since the last status report, 57 new entries have been added, bringing the total up to 814. The following FreeBSD releases are supported by the FreeBSD Security Team: FreeBSD 4.11, FreeBSD 5.3, FreeBSD 5.4, FreeBSD 5.5, FreeBSD 6.0, and FreeBSD 6.1. The respective End of Life dates of supported releases are listed on the web site; of particular note, FreeBSD 5.3 and FreeBSD 5.4 will cease to be supported at the end of October 2006, while FreeBSD 6.0 will cease to be supported at the end of November 2006 (or possibly a short time thereafter in order to allow time for upgrades to the upcoming FreeBSD 6.2). _________________________________________________________________ FreeBSD/arm on Atmel AT91RM9200 Contact: Warner Losh <imp_at_freebsd.org> Contact: Olivier Houchard <cognet_at_freebsd.org> The FreeBSD/arm port has grown support for the Atmel AT91RM9200. Boards based on this machine are booting to multiuser off either NFS or an SD card. The onboard serial ports, PIO, ethernet and SD/MMC card controllers are well supported. Support for the SSC, IIC and SPI flash parts in the kernel will be forthcoming shortly. In addition to normal kernel support, the port includes a boot loader that can initialize memory and boot off IIC eeprom, SPI DataFlash, BOOTP/TFTP and SD memory cards. The port will be included in forth coming commercial products. Open tasks: 1. Add support for other members of the AT91 family of arm9 processors. 2. Finish support for AT45D* flash parts. 3. Finish support for USB ports 4. Write support for USB Device functionality _________________________________________________________________ FreeSBIE URL: http://www.FreeSBIE.org URL: http://liste.gufi.org/mailman/listinfo/freesbie URL: http://people.freebsd.org/~matteo/GMV/GMVAnnounce.txt Contact: FreeSBIE Staff <staff_at_FreeSBIE.org> Contact: Matteo Riondato <matteo_at_FreeBSD.org> FreeSBIE is a FreeBSD based LiveCD. On August 19th, Matteo Riondato, a member of the FreeSBIE staff, released an unofficial ISO, codename FreeSBIE GMV, based on FreeBSD -CURRENT (read the Announcement to download it). This is supposed to be the first in a series of four ISOs that will end up with the release of FreeSBIE 2.0. Matteo is now working on another ISO, codename FreeSBIE LVC, which is scheduled to be released October 12th. FreeSBIE 2.0 will be based on FreeBSD 6.2-RELEASE and will hopefully be released at EuroBSDCon 2006 in Milan. It will be available for the i386 and AMD64 platforms. Open tasks: 1. Test the released ISO in preparation for the release. 2. Suggest software to include in the ISO. 3. Submit a simple and clear but complete fluxbox configuration. _________________________________________________________________ FreshPorts URL: http://www.freshports.org/ Contact: Dan Langille <dan_at_langille.org> The new 2U server mentioned in the last report now has a collection of Raptor drives in a RAID-10 configuration. Thanks to very generous donations from the community, I purchased eight of these drives at very good prices. The server will be deployed in the next few weeks. There has been quite a bit of work since the last report in June. Some highlights include: * New news feed formats, including newsfeeds for your watch list. * Better pages caching for faster response. * Sanity Test Failures now available online. * Ability to search for all commits (ports, doc, src, etc) under a given point in the tree. For more detail, please review the FreshPorts Blog . _________________________________________________________________ GJournal URL: http://people.freebsd.org/~pjd/patches/gjournal_20060930.patch URL: http://people.freebsd.org/~pjd/patches/gjournal6_20060930.patch Contact: Pawel Jakub Dawidek <pjd_at_FreeBSD.org> GJournal seems to be finished. I fixed the last serious bug and it is now stable and reliable in our tests. I'm planning to commit it really soon now. The work was sponsored by home.pl _________________________________________________________________ Gvinum improvements URL: http://folk.ntnu.no/lulf/patches/freebsd/gvinum/gvinum_all_current.dif f Contact: Ulf Lilleengen <lulf_at_pvv.ntnu.no> I thought that since I sent a status report the last time, I might as well send one now. Since the last status report I have done work on several of the remaining commands as attach, detach, and finally the concat command to be able to create concatenated volumes with one easy command. The mirror and stripe commands are the next step after this. The most important thing I've been working on is maybe the implementation of drivegroups. I have posted a bit information on this mailinglists, but basically, it's a way to group drives with the same configuration. This way, you can make many commands operate on groups instead of drives, and the group-abstraction will handle how the underlying subdisks are created on the drives. In the future one will be able to move groups to different machines, etc. I've created a patch of all my work that is not in HEAD yet here (this is a snapshot of my developement branch, so how thing's are done might be changed quite fast): http://folk.ntnu.no/lulf/patches/freebsd/gvinum/gvinum_all_current.dif f Be aware that a there will probably be bugs in the code, so don't use it in production yet! Thanks to Greg Lehey for offering to help me on getting this into CVS, and all feedback on this has been good. Open tasks: 1. Remaining components, mirror, stripe and some info commands. _________________________________________________________________ Gvirstor URL: http://wiki.freebsd.org/gvirstor Contact: Ivan Voras <ivoras_at_freebsd.org> Gvirstor is a GEOM class providing virtual ("overcommit") storage devices larger than physical available storage, with possibility to add physical storage on-line when the need arises. Current status is that it's done and waiting commit to HEAD, scheduled for some time after 6.2 is released. Open tasks: 1. The project is in need of testing! If you have the equipment and time, please give it a try so possible bugs can be fixed before it goes into -CURRENT. _________________________________________________________________ Highly improved implementations of sendfile(2), sosend_*() and soreceive_stream() URL: http://lists.freebsd.org/pipermail/freebsd-current/2006-September/0659 97.html URL: http://lists.freebsd.org/pipermail/freebsd-current/2006-September/0661 99.html URL: http://people.freebsd.org/~andre/sendfile+sosend+soreceive-20061006.di ff Contact: Andre Oppermann <andre_at_freebsd.org> The addition of TSO (TCP Segmentation Offload) has highlighted some shortcomings in the sendfile(2) and sosend_*() kernel implementations. The current sendfile(2) code simply loops over the file, turns each 4K page into an mbuf and sends it off. This has the effect that TSO can only generate 2 packets per send instead of up to 44 at its maximum of 64K. kern_sendfile() has been rewritten to work in two loops, the inner which turns as many pages into mbufs as it can -- up to the free send socket buffer space. The outer loop then drops the whole mbuf chain into the send socket buffer, calls tcp_output() on it and then waits until 50% of the socket buffer are free again to repeat the cycle. This way tcp_output() gets the full amount of data to work with and can issue up to 64K sends for TSO to chop up in the network adapter without using any CPU cycles. Thus it gets very efficient especially with the readahead the VM and I/O system do. Looking at the benchmarks we see some very nice improvements: 181% faster with new sendfile vs. old sendfile (non-TSO), 570% faster with new sendfile vs. old sendfile (TSO). The current sosend_*() code uses a sosend_copyin() function that loops over the supplied struct uio and does interleaved mbuf allocations and uiomove() calls. m_getm() has been rewritten to be simpler and to allocate PAGE_SIZE sized jumbo mbuf clusters (4k on most architectures). m_uiotombuf() has been rewritten to use the new m_getm() to obtain all mbuf space in one go. It then loops over it and copies the data into the mbufs by using uiomove(). sosend_dgram() and sosend_generic() have been changed to use m_uiotombuf() instead of sosend_copyin(). Looking at the benchmarks we see some very nice improvements: 290% faster with new sosend vs. old sosend (non-TSO), 280% faster with new sosend vs. old sosend (TSO). Newly written is a specific soreceive_stream() function for stream protocols (primarily TCP) that does only one socket buffer lock per socket read instead of one per data mbuf copied to userland. When doing netperf tests with WITNESS (full lock tracking and validation enabled) the receive performance increases from ~360Mbit/s to ~520Mbit/s. Without WITNESS I could not measure any statistically significant improvement on a otherwise unloaded machine. The reason is two-fold: 1) per packet we do a wakeup and readv() is pretty much as many times as packets come it, thus the general overhead dominates; 2) the packet input path has a pretty high overhead too. On heavily loaded machines which do a lot of high speed receives a performance increase should be measureable. The patches are scheduled to be committed to FreeBSD-current at end of October or early November 2006. This work was sponsored by the TCP/IP Optimization Fundraiser 2005. _________________________________________________________________ Hungarian translation of the webpages URL: http://gabor.t-hosting.hu/data/hu/ Contact: Gábor Kövesdán <gabor_at_FreeBSD.org> Since the last status report, there has been a lot of progress. I investigated a lot of charset issues and found out that HTML tidy breaks some entities when using iso-8859-2, so HTML tidy had to be disabled for Hungarian pages. Open tasks: 1. Translate 4 pages. 2. Review, fix typos and improve the wording where necessary. _________________________________________________________________ Improving FreeBSD Ports Collection Infrastructure URL: http://wikitest.freebsd.org/G%C3%A1borK%C3%B6vesd%C3%A1n Contact: Gábor Kövesdán <gabor_at_FreeBSD.org> Contact: Erwin Lansing <erwin_at_FreeBSD.org> During the Google Summer of Code 2006, Gábor worked on several ideas to improve the ports infrastructure: 1. New handling for i386 binary ports. 2. Cleanup: use ECHO_CMD and ECHO_MSG in bsd.port.mk properly. 3. Add a basic infrastructure support for debugging. 4. Installing ports with different destination (DESTDIR macro). 5. Cleanup: Move fetch shell scripts out of bsd.port.mk. 6. Make ports respect CC and CFLAGS. 7. Cross-compiling Ports. 8. Plist generator tool. The first three items have been completed and the next two items are being worked on. The DESTDIR support was more complicated than presumed and took more time than expected to complete. Gábor will continue working to finish these tasks and other ports related tasks. FreeBSD is happy to have interested him to keep working on ports and ports infrastructure. _________________________________________________________________ IPv6 Stack Vulnerabilities URL: http://wikitest.freebsd.org/ClementLecigne URL: http://pcs.sf.net Contact: George Neville-Neil <gnn_at_FreeBSD.org> Contact: Clement Lecigne <clem1_at_FreeBSD.org> The focus of this project was to review past vulnerabilities, create vulnerability testing tools and to discover new vulnerabilities in the FreeBSD IPv6 stack which is derived from the KAME project code. During the summer Clement took two libraries, the popular libnet, and his mentor's Packet Construction Set (PCS) and created tools to find security problems in the IPv6 code. Several issues were found, bugs filed, and patches created. At the moment Clement and George are editing a 50 page paper that describes the project which will be submitted for conference publication. All of the code from the project, including the tools, is on line and is described in the paper. By all measures, this was a successful project. Both student and mentor gained valuable insight into a previously externally maintained set of code. In addition to the new tools development in this effort, the FreeBSD Project has gained a new developer to help work on the code. _________________________________________________________________ iSCSI Initiator URL: ftp://ftp.cs.huji.ac.il/users/danny/freebsd/iscsi-17.5.tar.bz2 Contact: Damiel Braniss <danny_at_cs.huji.ac.il> This iSCSI initiator kernel module and its companion control program are still under development, but the main parts are working. Open tasks: 1. Network Disconnect Recovery. 2. Sysctl Interface and Instrumentation. 3. Rewrite the userland side of iscontrol. _________________________________________________________________ Jail Resource Limits URL: http://wikitest.freebsd.org/JailResourceLimits Contact: Chris Jones <cdjones_at_freebsd.org> Contact: Kip Macy <kmacy_at_freebsd.org> We now have support for limiting CPU and memory use in jails. This allows fairer sharing of a systems' resources between divergent uses by preventing one jail from monopolizing the available memory and CPU time, if other users and jails have processes to run. The code is currently available as patches against RELENG_6, and Chris is in the process of applying it to -CURRENT. More details can be found at JailResourceLimits on the wiki. Open tasks: 1. Port patches against -CURRENT. _________________________________________________________________ Libelf URL: http://wiki.freebsd.org/LibElf URL: http://wiki.freebsd.org/PmcTools URL: http://people.freebsd.org/~jkoshy/projects/perf-measurement/ Contact: Joseph Koshy <jkoshy_at_FreeBSD.org> Libelf is a BSD-licensed library for ELF parsing & manipulation implementing the SysV/SVR4 (g)ELF[3] API. Current status: Implementation of the library is nearly complete. A TET-based test suite for the API is being worked on. Open tasks: 1. Reviewers are needed for the code and the test suite. If you have extensions to the stock SysV/SVR4 ELF(3) API that you would like to see in -lelf, please send Joseph an email. _________________________________________________________________ MMC/SD Support Contact: Warner Losh <imp_at_freebsd.org> Contact: Bernd Walter <tisco_at_freebsd.org> The MMC/SD stack got a significant boost this quarter. Warner Losh and Bernd Walter have written a generic MMC/SD flash card stack for FreeBSD, and have implemented a host controller for the AT91RM9200 embedded ARM controller they are each using in separate projects. The stack is presently experimental in quality. It is being used as the root file system for these embedded projects. There's been no work done to support hot insertion and removal of cards (neither board wires up the pins necessary, and besides, / disappearing is very bad). There are still many rough edges. This is a freshly written stack. It has been written using the SD 1.0 (and recently 2.0) simplified specification, with the SanDisk MMC application notes supplementing. The Linux stack looks good, although not entirely standards conforming (there's work in progress that I've not seen that is supposed to fix this) and it is contaminated with the GPL. The OpenBSD stack also looks interesting, but Warner's experience porting NEWCARD over from NetBSD suggested that a fresh rewrite may be faster, at least for the bus and driver level. Since MMC is fairly simple, a port of the sdhci driver might be possible. Please see the open tasks list. Open tasks: 1. Write sdhci driver, and integrate it into the current stack. 2. Add support for hot plugging of cards. 3. Add support for MMC cards (SD cards were the first target). 4. Expand SD support to include SDIO cards as well as the new SDHC standard cards. 5. Export stats via sysctl for each of the cards that are found as a debugging and usage monitoring aid. 6. Add support for reading/writing multiple blocks at a time to improve performance. 7. Implement any other host controller. 8. Add proper support for timeouts. _________________________________________________________________ Nss-LDAP importing and nsswitch subsystem improvement URL: http://wikitest.freebsd.org/MichaelBushkov URL: http://wikitest.freebsd.org/LdapCachedOriginalProposal URL: http://wikitest.freebsd.org/LdapCachedDetailedDescription Contact: Michael Bushkov <bushman_at_FreeBSD.org> Contact: Hajimu UMEMOTO <ume_at_FreeBSD.org> The Project consisted of five parts: 1. Nsswitch modules and libc separation. The idea was to move the source code for different nsswitch sources (such as "files", "dns", "nis") out of the libc into the separate shared libraries. This task was successfully finished and the patch is available. 2. Regression tests for nsswitch. A set of regression tests to test the correctness of all nsswitch-related functions and the invariance of their behavior between system upgrades. The task can be considered successfully completed, the patch is available. 3. Rewriting nss_ldap. Though, this task was not clearly mentioned in the original proposal, during the SoC we found it would be easier, not to simply import PADL's nss_ldap, but to rewrite it from scratch (licensing issues were among the basic reasons for this). The resulting module behaves similarly to PADL's module, but has a different architecture that is more flexiable. Though it's basically finished, several useful features from the PADL's nss_ldap still need to be implemented. Despite the lack of some features, this task can be considered successfully completed. Missing features will be implemented as soon as possible, hopefully during September. 4. Importing nss_ldap into the Base System. The task was to prepare a patch, that will allow users to use nss_ldap from the base system. The task was successfully completed (the patch is available), but required importing OpenLDAP into the base in order for nss_ldap to work properly, and it had led to a long discussion in the mailing list. This discussion, however, have concluded with mostly positive opinions about nss_ldap and OpenLDAP importing. 5. Cached performance optimization. The caching daemon performance needs to be as high as possible in order for cached to be as close (in terms of speed) to "files" nsswitch source as possible. Cached's performance analysis was made and nsswitch database pre-caching was introduced as the optimization. This task was completed (the patch is available). However there is room for improvement. More precise and extensive performance analysis should be made and more optimizations need to be introduces. This will be done in the near future. Though none of the code was committed yet into the official FreeBSD tree, my experience from the previous year makes me think that this situation is normal. I hope, that the code will be reviewed and committed in the coming months. _________________________________________________________________ OCaml language support in ports URL: http://www.freebsd.org/cgi/cvsweb.cgi/~checkout~/ports/lang/ocaml/bsd. ocaml.mk?rev=1.3&content-type=text/plain Contact: Stanislav Sedov <stas_at_FreeBSD.org> There were a number of OCaml ports in our tree, and each of them was doing the same work by maintaining OCaml ld.conf in the correct state, installing/removing their files/entries etc. To simplify the task of OCaml-language ports creationm the special framework (bsd.ocamk.mk) was developed and most of the ports was converted to use this framework. This allowed a lot of duplicate code to be removed. This new framework handles all the things required to install an OCaml-language library and properly register it. bsd.ocaml.mk also contains knobs to deal with findlib-powered libraries, modify ld.conf in the proper way, etc. Also, a lot of new Ocaml-related ports were added. _________________________________________________________________ OpenBSD dhclient Contact: Brooks Davis <brooks_at_FreeBSD.org> Most dhclient changes in HEAD have been merged to 6-STABLE for 6.2-RELEASE. The highlight of these changes is a fix for runaway dhclient processes when packets are not 4 byte aligned. Further changes including always sending client identifiers are scheduled for merge before the release. Work is ongoing to improve dhclient's interaction with alternate methods of setting interface addresses. _________________________________________________________________ Porting the seref policy and setools to SEBSD URL: http://wikitest.freebsd.org/DongmeiLiu Contact: Dongmei Liu <dongmei_at_freebsd.org> Contact: Christian Peron <csjp_at_FreeBSD.org> Dongmei Liu spent the summer working on the basic footwork required to port the SEREF policy to SEBSD. This work has been submitted and can be viewed in the soc2006/dongmei_sebsd Perforce branch. This work was originated from the SEBSD branch: //depot/projects/trustedbsd/sebsd. Additionally setools-2.3 was ported from Linux and can be found in contrib/sebsd/setools directory. It is hoped that this work will be merged into the main SEBSD development branch. _________________________________________________________________ Porting Xen to FreeBSD URL: http://www.yuanjue.net/xen/howto.html URL: http://wikitest.freebsd.org/YuanJue Contact: Jue Yuan <yuanjue_at_FreeBSD.org> As a participant of Google's Summer of Code 2006, I am focusing on porting Xen to FreeBSD these months. The result of this summer's work include a domU kernel that could be used for installation, a guide for getting started with FreeBSD on Xen, and some other trivial improvements. But there are still a lot of work needing to be done in this area, e.g, the long-expeted dom0 support. So I will continue my work here and try to keep up with the update of Xen itself. Open tasks: 1. dom0 support is the most urgent _________________________________________________________________ Porting ZFS to FreeBSD URL: http://perforce.freebsd.org/depotTreeBrowser.cgi?FSPC=//depot/user/pjd /zfs URL: http://www.opensolaris.org/os/community/zfs/porting/ URL: http://docs.freebsd.org/cgi/mid.cgi?20060822104516.GB16033 Contact: Pawel Jakub Dawidek <pjd_at_FreeBSD.org> My work is moving slowly forward. ZVOL is, I believe, fully functional (I recently fixed snapshots and clones on zvols), which means you can put UFS on top of RAID-Z volume, take a snapshot of the volume, clone it if needed, etc. Very cool. The hardest part is the ZPL layer, I'm still working on it. Most file system methods work, but probably need detailed review and many fixes. Most of the time these days I'm spending on implementing mmap(2) correctly. It works more or less in simple tests but fails under fsx program. On the other hand, 'fsx -RW' works very stable and reliable. Other test programs (those that don't use mmap(2)) also work quite well. There is still a lot of work to do, mostly in ZPL area, many clean-ups, etc. Some functionality (like ACLs) I haven't even tried to touch yet. _________________________________________________________________ Ports Collection URL: http://www.freebsd.org/ports/ URL: http://www.freebsd.org/doc/en_US.ISO8859-1/articles/contributing-ports / URL: http://people.freebsd.org/~fenner/portsurvey/ URL: http://portsmon.FreeBSD.org/index.html URL: http://www.freebsd.org/portmgr/index.html URL: http://tinderbox.marcuscom.com/ Contact: Mark Linimon <linimon_at_FreeBSD.org> The ports PRs surged (especially due to a large number of new port submissions), but with some hard work we have been able to get back down to around 900. We are rapidly approaching 16,000 ports. Due to this acceleration in adding new ports, portmgr is now very concerned that we are outstripping the capacity of both the build infrastructure and our volunteers to keep up with build errors and port updates. Accordingly, we've added a guideline (not a rule) that ports should be of more than just theoretical use to be added to the Ports Collection (e.g. we can't support all of CPAN + all of Sourceforge + everything else). Basically, use common sense as a guideline; certainly no one wants to see any kind of "gateway" procedure to get incoming ports approved. Seven sets of changes have been added to the infrastructure, mostly refactoring and bugfixing. As part of a Summer of Code project, we have also incorporated some of gabor_at_'s changes to incorporate better DESTDIR support. However, due to some unanticipated side-effects, more work is going to be needed in this area. gabor_at_ is continuing to work on the changes. netchild_at_ and bsam_at_ have been doing a great deal of work to bring the linux emulator ports closer to sanity, including bringing up a regression-test suite. The long-anticipated import of X.Org 7 has stalled due to developer time, mostly to deal with documentation and upgrade instructions. Hopefully this can get done in the early 6.3 development cycle. See the wiki for more information. As a part of that work, the decision has been made to move away from using X11BASE and just put everything into LOCALBASE; /usr/X11R6 is simply an artifact at this point. A plan for a transition process is underway; a great deal of testing will need to be done, but in the end the ports tree will be much cleaner. The GNOME team has already done the work to move all of their ports over, and it will be incorporated after the 6.2 release is shipped. tmclaugh_at_ is looking for someone to take over the C# ports. He has maintained them for over a year and wants more time to be able to work on other projects. Some work has been done to get rid of FreeBSD 2.X cruft in ports. Further work is needed to get the 3.X cruft removed. linimon_at_ did another pass through resetting inactive maintainers. Another list is waiting in the wings. linimon_at_ is also working on adding the ability for portsmon to analyze successful packages (not just failed ones), so that queries such as "show me packages that build on i386 but not amd64" and "show me why dependent package foo was not built on bar". This is currently in alpha testing. We have added 4 new committers since the last report. Open tasks: 1. We still need help getting back to our modern low of 500 PRs. 2. We have nearly 4400 unmaintained ports (see, for instance, the list on portsmon ). Although there has been a welcome upsurge in new maintainers recently which has dropped the percentage down below 28%, we still need much more help. 3. A test run of gcc4.1 on the ports tree showed around 1000 new build errors. Kris_at_ has posted some results so that people can start working on the problems now. In particular, it seems that certain older versions of GCC cannot be built with GCC 4.1, so ports that depend on those older versions are going to have to be fixed as well. Although the import of GCC 4.1 to -CURRENT is not imminent, the time to start planning is now. 4. The state of the packages on AMD64 and sparc64 significantly lags that of i386. In many of these cases, packages are not attempted because NOT_FOR_ARCH is used instead of more accurately only setting BROKEN based on ARCH. (pointyhat can be forced to build packages that are marked BROKEN, but not NOT_FOR_ARCH). NOT_FOR_ARCH is supposed to denote only "will never work on this ARCH". Although we have volunteers who have expressed interest in sparc64 (and ia64), we need more people who are running amd64 (especially as a desktop) to help us get more packages working. _________________________________________________________________ Release Engineering URL: http://www.FreeBSD.org/releng/ URL: http://www.FreeBSD.org/releases/ URL: http://www.FreeBSD.org/snapshots/ Contact: Release Engineering Team <re_at_FreeBSD.org> The FreeBSD Release Engineering team is currently working on FreeBSD 6.2-RELEASE, which is scheduled for release in early November 2006. Some notable features of this release include the debut of security event auditing as an experimental feature, Xbox support, the FreeBSD Update binary updating utility, and of course many fixes and updates for existing programs. Pre-release images for all Tier-1 architectures are available for testing now; feedback on these builds is greatly appreciated. More information about release engineering activities can be found at the links above. _________________________________________________________________ SCTP Integration URL: http://www.sctp.org/ Contact: Randall Stewart <randall_at_freebsd.org> Contact: George Neville-Neil <gnn_at_freebsd.org> There are currently patches available for testing. A planned integration to HEAD is set to happen in October. Open tasks: 1. The code still needs plenty of testing. See patches on sctp.org and in -CURRENT soon. _________________________________________________________________ SNMP monitoring (BSNMP) URL: http://perforce.freebsd.org/depotTreeBrowser.cgi?FSPC=//depot/user/soc %2dshteryana/bsnmp&HIDEDEL=NOe URL: http://wikitest.freebsd.org/CategorySNMP URL: http://wiki.freebsd.org/SnmpBridgeModule URL: http://www.freshports.org/net-mgmt/bsnmptools/ Contact: Shteryana Shopova <shteryana_at_FreeBSD.org> Contact: Bjoern A. Zeeb <bz_at_FreeBSD.org> A BRIDGE monitoring module for FreeBSD's BSNMP daemon has been implemented. In addition to RFC 4188 single bridge support and extending the kernel to get access to all the information, a private MIB was designed in order to be able to monitor multiple bridges supported by FreeBSD. The kernel part has already been committed to -CURRENT (thanks to thompsa_at_), for -STABLE a patch is available (see the wiki), code has already been reviewed. SoC 2005 work on SNMP client tools is now available too via port (net-mgmt/bsnmptools), thanks to Andrew Pantyukhin for the port. Open tasks: 1. More testing is very welcome. 2. if_vlan(4) monitoring module. 3. jail(8) monitoring module. _________________________________________________________________ Sound Subsystem Improvements URL: http://people.FreeBSD.org/~ariff/ URL: http://www.FreeBSD.org/projects/ideas/ URL: http://wiki.FreeBSD.org/soundsystem Contact: Ariff Abdullah <ariff_at_FreeBSD.org> Contact: Alexander Leidinger <netchild_at_FreeBSD.org> Contact: Ryan Beasley <ryanb_at_FreeBSD.org> Contact: Multimedia Mailinglist <multimedia_at_FreeBSD.org> Since the last status report we added basic support for envy24ht chips, imported the emu10kx driver into the base system and added support for High Definition Audio (HDA) compatible chips. Additionally the work of Ryan Beasley as part of his Google Summer of Code 2006 participation is committed. It adds compatibility to the Open Sound System (OSS) v4 API as far as this was possible. This allows for more sophisticated programs to be written. For example it is now possible to synchronize the start of multiple sound channels. It is also possible for a driver to support more than the AC97 mixer devices, but so far no driver has been extended to support this yet. More about it can be found in the wiki and in the official OSS documentation. The wiki page about the sound system was started to describe the current status of the sound system and to provide some information about where we are heading. But more work needs to be done to reach this goal. So far we collected some information about the status of the most recent work in the soundsystem. So if you have a look at it and you think that something important is missing, just tell us about it. While fully prepared content is very welcome, we are even happy about some ideas what we should list on the wiki page. Open tasks: 1. Have a look at the sound related entries on the ideas list. 2. sndctl(1): tool to control non-mixer parts of the sound system (e.g. spdif switching, virtual-3D effects) by an user (instead of the sysctl approach in -current); pcmplay(1), pcmrec(1), pcmutil(1). 3. Plugable FEEDER infrastructure. For ease of debugging various feeder stuff and/or as userland library and test suite. 4. Extend the wiki page. _________________________________________________________________ Summer of Code Summary URL: http://www.FreeBSD.org/projects/summerofcode-2006.html URL: http://wikitest.freebsd.org/SummerOfCode2006 URL: http://perforce.freebsd.org/depotTreeBrowser.cgi?FSPC=//depot/projects /soc2006/ Contact: Murray Stokely <murray_at_FreeBSD.org> We had another successful summer taking part in the Google Summer of Code. By all accounts, the FreeBSD participation in this program was an unqualified success. We received over 150 applications for student projects, amongst which 13 were selected for funding. All successful students received the full $4,500. These student projects included security research, improved installation tools, new utilities, and more. Many of the students have continued working on their FreeBSD projects even after the official close of the program. At least 2 of our FreeBSD mentors will be meeting with Google organizers in Mountain View this month to discuss the program at the Mentor Summit. _________________________________________________________________ Summer of FreeBSD security development URL: http://people.freebsd.org/~cperciva/funding.html URL: http://www.daemonology.net/freebsd-upgrade-6.0-to-6.1/ Contact: Colin Percival <cperciva_at_FreeBSD.org> I spent the months of May through August working on improving Portsnap, FreeBSD Update, and devoting more time to my (continuing) role as Security Officer. FreeBSD Update is now part of the FreeBSD base system and is fully supported by the FreeBSD Security Team; updates are currently only being built for the i386 architecture, but AMD64 updates will become available soon. In an attempt to reduce the number of people running out of date (and unsupported) FreeBSD releases, I wrote an automatic binary upgrade script for upgrading systems from FreeBSD 6.0 to FreeBSD 6.1; I will be releasing a new script for upgrading to FreeBSD 6.2-(RC*|RELEASE) soon (possibly before this status report is published). Further improvements to Portsnap are still ongoing. _________________________________________________________________ Sun Niagara port Contact: Kip Macy <kmacy_at_FreeBSD.org> Support for the UltraSparc T1 (Niagara) continues to improve. The code has recently been checked into public CVS under sys/sun4v. It isn't clear whether or not I will have time to implement full logical domaining support before the APIs become publicly available. Testing indicates that substantial work will be needed before FreeBSD can take full advantage of all 32 threads. Open tasks: 1. Random testing and bug fixes. 2. Import and extend improved mutex profiling support. 3. Virtual network and virtual disk device drivers for logical domains. _________________________________________________________________ The FreeBSD Foundation URL: http://www.freebsdfoundation.org Contact: Deb Goodkin <deb_at_FreeBSD.org> The FreeBSD Foundation continued to support the FreeBSD project and community through various activities. These activities include creating strategies for fund development and actively seeking funding for the FreeBSD community, coordinating a new IBM Bladeserver project, and protecting the image and integrity of FreeBSD by governing the use of the trademarks. We are pleased to be a sponsor of EuroBSDCon and will be sponsoring a few developers to attend the conference through our travel grant program. And finally, we have secured funds for a major project that will be announced later this month. _________________________________________________________________ TrustedBSD Audit URL: http://www.TrustedBSD.org/audit.html URL: http://www.OpenBSM.org/ Contact: Robert Watson <rwatson_at_FreeBSD.org> Contact: Christian Peron <csjp_at_FreeBSD.org> Contact: Wayne Salamon <wsalamon_at_FreeBSD.org> The TrustedBSD audit implementation provides fine-grained security event logging throughout the FreeBSD operating system. The big news for the last quarter is that the TrustedBSD audit implementation has been merged into RELENG_6 branch, and appeared in 6.2-BETA2. Over the past few months, work has also occurred in the following areas: * OpenBSM 1.0 alpha 8 through alpha 12 have been released and merged into FreeBSD CVS. Changes include significant numbers of bug fixes, documentation improvements, and feature enhancements. These include regular expression based matching for auditreduce, auditd management of kernel audit policy (such as maximum trail file size), improvements in printing support for a variety of tokens including execve argument support. * Significant enhancements to the FreeBSD Handbook chapter on Audit. * Full audit support for execve events, including optional auditing of command line arguments and environmental variables, as well as audit support for a broad range of other additional kernel events. * Kqueue support for audit pipes. * Robustness improvements in the presence of low disk space conditions. * Support for system call capture on additional platforms, such as ppc and ia64. * Improved support for very large audit record sizes (as required for extensive execve support). * id(1) now supports a -A argument to query audit state for the process. * An audit_warn(5) event for trail rotation, which can be used for archiving, reduction, and other administrative activities. Lots of testing as part of the 6.2-BETA cycle would be much appreciated. Audit support will be considered an experimental feature in FreeBSD 6.2-RELEASE, but we hope that it will be a production feature in 6.3-RELEASE. Open tasks: 1. Continue expanding auditing of syscall arguments. 2. Continue expanding auditing of administrative tools. 3. More testing! 4. Continue to explore improvements of the administrative model for audit trails, etc. _________________________________________________________________ TSO - TCP Segmentation Offload committed URL: http://lists.freebsd.org/pipermail/cvs-src/2006-September/068524.html URL: http://lists.freebsd.org/pipermail/cvs-src/2006-September/068610.html URL: http://lists.freebsd.org/pipermail/cvs-src/2006-September/069493.html Contact: Andre Oppermann <andre_at_freebsd.org> TSO - TCP Segmentation Offload support has been committed to the network stack of FreeBSD-current in September 2006. With TSO, TCP can send data in the send socket buffer in bulk down to the network card which then does the splitting into MTU sized packets. On bulk high speed sending the performance is increased by 25% (normal writes) to 108% (sendfile). Jack Vogel and Prafulla Deuskar of Intel committed the driver changes for TSO hardware support of em(4) based network cards. These changes are scheduled to be backported to FreeBSD 6-STABLE shortly after FreeBSD 6.2-RELEASE is published to appear in upcoming FreeBSD 6.3 early next year. This work was sponsored by the TCP/IP Optimization Fundraiser 2005. Open tasks: _________________________________________________________________ Update of the Linux compatibility environment in the kernel URL: http://wiki.FreeBSD.org/linux-kernel Contact: Alexander Leidinger <netchild_at_FreeBSD.org> Contact: Roman Divacky <rdivacky_at_FreeBSD.org> Contact: Emulation Mailinglist <emulation_at_FreeBSD.org> Roman Divacky participated in the Google Summer of Code 2006 and implemented a major part of the syscall compatibility to the 2.6.16 Linux kernel. The work has been committed to -CURRENT (the default compatibility still being a 2.4.2 Linux kernel) and we are working on fixing the remaining bugs as time permits. "Intron" submitted an implementation for the linux aio syscalls. His work has been committed to the Perforce repository. We also started to consolidate a list of known bugs, open issues and helpful stuff (e.g. regression tests and their status) in -CURRENT on a page in the FreeBSD wiki (see the links-section). It also contains a link to a more or less up-to-date patch with stuff we have in the Perforce repository so that interested people can help with testing. Thanks to the help of Marcin Cieslak we already fixed some bugs (some of the fixes are already MFCed to -STABLE). Thanks to the nice regression tests of the Linux Test Project (LTP) we have a list of small (and not so small) things which need to be looked at. This list makes up for a quick start into kernel hacking. So if you have a little bit of knowledge about C programming, and if you want to help us a little bit in improving FreeBSD, feel free to have a look at the list and to try to fix a problem or two. Sometimes it is as easy as "if (error condition) return Esomething;" (but you should coordinate with the emulation mailinglist, so that nobody does some work someone else just did too). Even if you do not know how to program, you can help. Have a look at the wiki page and tell us about things which should get mentioned there too. Or download the patch and test it. _________________________________________________________________ USB URL: http://perforce.freebsd.org/depotTreeBrowser.cgi?FSPC=//depot/projects /usb/src/sys/dev/usb&HIDEDEL=NO URL: http://www.turbocat.net/~hselasky/usb4bsd Contact: Hans Petter Sirevaag Selasky <hselasky_at_freebsd.org> During the last three months I have finished reworking nearly all USB device drivers found in FreeBSD-7-CURRENT. Only two USB drivers are left and that is ubser(4) and slhci. Some still use Giant, but most have been brought out of Giant. At the moment I am looking for testers that can test the various USB device drivers. Some have already been tested, and confirmed to work, while others have problems which need to be fixed. If you want to test, checkout the USB perforce tree or download the SVN version of the USB driver that is available on my homepage. At the moment the tarballs are a little out of date. Ideas and comments with regard to the new USB API are welcome at: freebsd-usb_at_freebsd.org. _________________________________________________________________ Xen Port Contact: Kip Macy <kmacy_at_FreeBSD.org> Work on Xen support has slowly been continuing in perforce. The SOC student fixed several bugs and is continuing to work on it. Someone is needed who has the time to complete dom0 support and shepherd it production level stability. Sufficient interest has been expressed in it that it probably makes sense to check it in to public CVS so that more people can try it out. Time permitting, I will bring it up to date and check it in the next month. Open tasks: 1. dom0 support. 2. General testing and bug fixing. _________________________________________________________________ News Home | Status Home Legal Notices | © 1995-2006 The FreeBSD Project. All rights reserved _______________________________________________ freebsd-announce_at_freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-announce To unsubscribe, send any mail to "freebsd-announce-unsubscribe_at_freebsd.org"Received on Thu Oct 19 2006 - 11:57:29 UTC
This archive was generated by hypermail 2.4.0 : Wed May 19 2021 - 11:39:01 UTC