January-April 2005 Status Report Introduction The first quarter of 2005 has been extremely active in both FreeBSD-CURRENT and -STABLE. With FreeBSD 5.4 in the final RC stage and an anticipated branch of FreeBSD-6 this summer we have seen a lot of performance improvements in 5 and a couple of exciting new features in 6. The report turnout was extremely good and it seems that the webform provided by Julian Elischer has made it more enjoyable to write reports. Many thanks to Julian for providing this. We also like to get your attention to the open tasks section provided in some reports. On special note, please take a look at the report about the upcoming BSDCan in Ottawa. There will be lots of interesting FreeBSD related talks and activities. If you enjoy reading these reports, you will love the conference. See you there! Thanks to all the reporters, we hope you enjoy reading. _________________________________________________________________ Projects * Common Address Redundancy Protocol - CARP * FreeBSD Java Project * FreeBSD Release Engineering * GELI - GEOM class for providers encryption * GSHSEC - GEOM class for handling shared secret * Secure Updating Documentation * FreeBSD Dutch Documentation Project Kernel * ATAPI/CAM * Coverity Code Analysis * cpufreq * drm * Filesystem journalling for UFS * Infrastructure Cleanup * Interrupt Latency * Low-overhead performance monitoring for FreeBSD * Many subdirs for UFS * Status Report for FreeBSD ATA driver project * Storage driver SMPng locking Network infrastructure * Dingo * IPv6 Support for IPFW * Move ARP out of routing table * netgraph(4) status report * Removable interface improvements. * Support for telephone hardware (aka Zaptel) * Wireless Networking Support Userland programs * libthread * Pipe namespace added to portalfs Architectures * ARM Support for TS-7200 * PowerPC Port * XenFreeBSD - FreeBSD on Xen Ports * FreshPorts * Ports Collection * Update of the Linux userland infrastructure Vendor / 3rd Party Software * OpenBSD packet filter - pf * twa driver Miscellaneous * BSDCan * FreeBSD Security Officer and Security Team * IMUNES - a FreeBSD based kernel-level network topology emulator _________________________________________________________________ ARM Support for TS-7200 URL: http://www.embeddedarm.com/epc/ts7200-spec-h.html URL: http://perforce.freebsd.org/depotTreeBrowser.cgi?FSPC=//depot/user/jmg /arm&HIDEDEL=NO URL: http://people.freebsd.org/~jmg/dmesg.ts7200 Contact: John-Mark Gurney <jmg_at_FreeBSD.org> I have been working on getting FreeBSD/arm running on the TS-7200. So far the board boots, and has somewhat working ethernet (some unexplained packet loss). I can netboot from a FreeBSD/i386 machine, and I can also mount msdosfs's on CF. Open tasks: 1. Figuring out why some small packets transmit with error 2. EP93xx identification information to properly attach various onboard devices _________________________________________________________________ ATAPI/CAM Contact: Thomas Quinot <thomas_at_freebsd.org> ATAPI/CAM integration with the new ATA (mkIII) framework is now completed. ATAPI/CAM is now available as a loadable module (atapicam.ko). It is also independant from the native ATAPI drivers again, as was the case before mkIII. Thanks to Scott Long and Søren Schmidt for their participation in the integration work. _________________________________________________________________ BSDCan URL: http://www.bsdcan.org/ Contact: Dan Langille <dan_at_langille.org> BSDCan made a strong debut in 2004 . The favorable reception gave us a strong incentive for 2005 . We have been rewarded with a very interesting program and a higher rate of registrations. Percentage-wise, we have more Europeans than last year as they have decided that the trip across the Atlantic is worth taking. We know they won't be disappointed. See you at BSDCan 2005! Open tasks: 1. volunteers needed for the conference _________________________________________________________________ Common Address Redundancy Protocol - CARP URL: http://www.freebsd.org/cgi/man.cgi?query=carp&manpath=FreeBSD+6.0-curr ent URL: http://people.freebsd.org/~mlaier/CARP/ Contact: Max Laier <mlaier_at_FreeBSD.org> Contact: Gleb Smirnoff <glebius_at_FreeBSD.org> CARP is an alternative to VRRP. In contrast to VRRP it has full support for IPv6 and uses crypto to protect the advertisements. It was developed by OpenBSD due to concerns that the HSRP patent might cover VRRP and CISCO might defend its patent. CARP has, since then, improved a lot over VRRP. CARP has been committed to HEAD and MFCed to RELENG_5. It will be available in upcoming 5.4-RELEASE. Big thanks to all users who provided testing and reported bugs to Max and Gleb. Daniel Seuffert has donated hardware to Max for this project. Gleb's work was sponsored by Rambler . Open tasks: 1. Improve vlan(4) support. Test ng_eiface(4). 2. Improve locking, consider removing interface layer. _________________________________________________________________ Coverity Code Analysis URL: http://www.coverity.com/ Contact: Sam Leffler <sam_at_freebsd.org> There has been an ongoing effort to review the kernel source code using Coverity's source code analysis tools (http://www.coverity.com). These tools check for a variety of problems such as null pointer dereference, use-after-free of allocated variables, invalid array references, etc. This work is a joint project between FreeBSD and Coverity. Two passes have been completed over the 6-current kernel source code base and all significant problems have been corrected. These runs were done in February and March of this year. A few reports of minor problems await response from outside groups and will be resolved in time for the first 6.x release. Another analysis run over the kernel will happen soon. We are looking for a way to use these tools on a regular basis as they have been helpful in improving the code base. Thanks to Coverity for their help and especially Ted Unangst. Several developers have been especially helpful in resolving reports: Poul-Henning Kamp, David Schultz, Pawel Jakub Dawidek, George V. Neville-Neil, and Matthew Dodd. _________________________________________________________________ cpufreq URL: http://www.freebsd.org/cgi/man.cgi?query=cpufreq&manpath=FreeBSD+6.0-c urrent&format=html Contact: Nate Lawson <njl> The cpufreq project was committed to 6-CURRENT in early February and has undergone bugfixes and updates. It will soon be MFCd to 5-STABLE. The cpufreq driver provides a unified kernel and user interface to CPU frequency control drivers. It combines multiple drivers offering different settings into a single interface of all possible levels. Users can access this interface directly via sysctl(8), by indicating to power_profile that it should switch settings when the AC line state changes, or by using powerd(8). For example, an absolute driver offering frequencies of 1000 Mhz and 750 Mhz combined with a relative driver offering settings of 100% and 50% would result in cpufreq providing levels of 1000, 750, 500, and 375 Mhz. Colin Percival helped with powerd(8), which provides automatic control of CPU frequencies. The adaptive mode is especially interesting since it attempts to respond to changes in system load while reducing power consumption. Current hardware drivers include acpi_perf (ACPI CPU performance states), est (Intel Enhanced SpeedStep for Pentium-M), ichss (Intel's original SpeedStep for ICH), and powernow (AMD Powernow! K7 and K8 support). Other drivers for relative hardware include acpi_throttle (ACPI CPU throttling) and p4tcc (Pentium 4 Thermal Control Circuitry) Thanks to Bruno Ducrot for the powernow driver, Colin Percival for the est driver, and the many testers who have sent in feedback. Open tasks: 1. We'd appreciate someone with a Transmeta CPU converting the existing longrun driver to the cpufreq framework. It would also be good if someone wrote a VIA Longhaul driver. See the Linux arch/i386/kernel/cpu/cpufreq directory for examples. 2. Various other architectures, including ARM, have CPU power control that could be implemented as a cpufreq driver. 3. The powerd(8) algorithm is rather simple and we'd appreciate more help in testing it and alternative algorithms with various workloads. The -v flag causes powerd to report frequency transitions and print a summary of total energy used upon termination. This should help testers profile their algorithms. _________________________________________________________________ Dingo URL: http://people.freebsd.org/~gnn/Dingo/notebook/60.html URL: http://zoo.unixdaemons.com/index.php?blog=7 Contact: George Neville-Neil <gnn_at_neville-neil.com> On the protocol conformance tool I have finally made some progress getting a scriptable packet library using libnet, and SWIG. This will hopefully become a port that can then be used to do conformance testing on protocol stack changes. Qing Li has separately taken up the ARP rewrite and that will be taken out of the Dingo project pages. Open tasks: 1. Many :-) _________________________________________________________________ drm URL: http://r300.sourceforge.net/ Contact: Eric Anholt <anholt_at_FreeBSD.org> A DRM update was finally committed to -current on 2005-04-15, after jhb_at_ did the necessary fix to vm_mmap. New development drivers were added for mach64 and r300 (see URL for info). The nearly-finished code for savage and i915 were also added, but left disconnected from the build. However, the most visible change is likely the support for texture tiling, color tiling, and HyperZ on Radeons, which (with updated userland) likely provide a 50-75% framerate increase in many applications. Open tasks: 1. Find someone with newbus knowledge to figure out why the i915 won't attach to drmsub0. 2. Finish porting the savage driver. 3. Integrate busdma code from Tonnerre (NetBSD). _________________________________________________________________ Filesystem journalling for UFS URL: http://repoman.freebsd.org/depotTreeBrowser.cgi?FSPC=//depot/user/scot tl/ufsj Contact: Scott Long <scottl_at_freebsd.org> It's time to bite the bullet and admit that fsck is no longer scalable for modern storage capacities. While a healthy debate can still be had on the merits and data integrity guarantees of journalling vs. SoftUpdates, the fact that SoftUpdates still requires a fsck to ensure consistency of the filesystem metadata after an unclean shutdown means uptime is lost. While background fsck is available, it saps system performance and stretched the fsck time out to hours. Journalling provides a way to record transactions that might not have fully been written to disk before the system crashed, and then quickly recover the system back to a consistent state by replaying these transactions. It doesn't guarantee that no data will be lost, but it does guarantee that the filesystem will be back to a consistent state after the replay is performed. This contrasts to SoftUpdates that re-arranges metadata updates so that inconsistencies are minimized and easy to recover from, though recovery still requires the traditional full filesystem scan. Journalling is a key feature of many modern filesystems like NTFS, XFS, JFS, ReiserFS, and Ext3, so the ground is well covered and the risks for UFS/FFS are low. I'm aware that groups from CMU and RPI have attempted similar work in the past, but unfortunately the work is either very outdates, or I haven't had any luck in contacting the groups. Is this absence, I've decided to work on this project myself in hopes of having a functional prototype in time for FreeBSD 6.0. The approach is simple and journals full metadata blocks instead of just deltas or high-level operations. This greatly simplifies the replay code at the cost of requiring more disk space for the journal and more work within the filesystem to identify discreet update points. An important design consideration is whether to make the journal data and code compatible with the UFS2 filesystem, or to start a new UFS3 derivative. Since the latter presents a very high barrier to adoption for most people, I'm going to try to make it a compatible option for UFS2. This means that the journal blocks will likely appear as an unlinked file to legacy filesystem and fsck code, and will be treated as such. This will allow seamless fallback to using fsck, though once the unlinked journal data blocks are reclaimed by fsck, the user will have to take action to re-create the journal file again. One key piece of journalling is ensuring that each journal transaction is fully written to disk before the associated metadata blocks are written to the filesystem. I plan to adopt the buffer 'pinning' mechanism from Alexander Kabaev's XFS work to assist with this. This will allow the journalling subsystem fine-grained control over which blocks get flushed to disk by the buffer daemon without having to further complicate the UFS/FFS code. One consideration is how Softupdates falls into this and whether it is multually exclusive of journalling or if it can help provide transaction ordering functionality to the journal. Research here is on-going. Some preliminary work can be found in Perforce in the //depot/user/scottl/ufsj/... tree or at the URL provided. Hopefully this will quickly accelerate. _________________________________________________________________ FreeBSD Dutch Documentation Project URL: http://www.freebsd.org/doc/nl/books/handbook URL: http://www.evilcoder.org/freebsd_html/ URL: http://www.evilcoder.org/content/section/6/39/ Contact: Remko Lodder <remko_at_FreeBSD.org> The FreeBSD Dutch Documentation Project is a ongoing project in translating the English documentation to the Dutch language. Currently we have translated almost the entire handbook, and more to come. If you want to help out by review the Dutch documents, or you want to help translating the remainders of the handbook or other documents, feel free to contact me at remko_at_FreeBSD.org Open tasks: 1. Translate the English handbook, then review the Dutch handbook 2. Translate the English FAQ, then review the Dutch FAQ 3. Translate the English Articles, then review the Dutch Articles _________________________________________________________________ FreeBSD Java Project URL: http://www.FreeBSD.org/java/ Contact: Greg Lewis <glewis_at_FreeBSD.org> Contact: Alexey Zelkin <phantom_at_FreeBSD.org> The FreeBSD Java Project released its initial support for JDK 1.5.0 with patch set 1 "Sabretooth" in January. The initial release featured support for both FreeBSD 5.3/i386 and 5.3/amd64. Since then preliminary support for FreeBSD 4.11/i386 has been added and several bug fixes have been made. Updates in the coming months will add support for the browser plug in and Java Web Start, which were not in the initial release. Open tasks: 1. Volunteers to look into some serious problems with JDK 1.5.0 on FreeBSD 4.x _________________________________________________________________ FreeBSD Release Engineering URL: http://www.freebsd.org/releng Contact: RE Team <re_at_freebsd.org> FreeBSD 4.11, the final formal release of the 4.x series, was released on 25 Jan 2005. Many thanks to the all of the developers and users over the past 5 years who made it successful. While no more releases are planned, the security team will continue to support it through security update patches until 2007. Developers are also free to commit bug fixes and low-risk features to the RELENG_4 branch for the foreseeable future. FreeBSD 5.4 is going through its final release candidate stages and is expected to be released in late April. Its focus is mostly bug fixes and minor feature and performance improvements, so it is an excellent target for those looking to upgrade from previous versions or to give FreeBSD a try for the first time. FreeBSD 5.5 will be release in about 4-6 months after 5.4. FreeBSD 6.0 is rapidly approaching also. In contrast to FreeBSD 5.0, the goal is to take a more incremental approach to major changes, and not wait for years to get as many features in as possible. FreeBSD 6.0 will largely be an evolutionary change from the 5.x series, with the largest changes centered around multi-threading and streamlining the filesystem and device layers. Feature freeze and code freeze for 6.0 are coming up in May and June, and we hope to have 6.0 stable and ready for release in July or August. The release engineering team has also started doing monthy informal snapshots of the 6-CURRENT and 5-STABLE trees. These are intended to increase the exposure of new features and get more users involved in testing and providing feedback. Snapshots can be found at http://www.freebsd.org/snapshots. _________________________________________________________________ FreeBSD Security Officer and Security Team URL: http://www.freebsd.org/security/ URL: http://www.freebsd.org/doc/en_US.ISO8859-1/articles/contributors/staff -listing.html#STAFF-SECTEAM URL: http://vuxml.freebsd.org/ Contact: Security Officer <security-officer_at_FreeBSD.org> Contact: Security Team <security-team_at_FreeBSD.org> In January 2005, Warner Losh (Security Officer Emeritus) stepped down from the FreeBSD Security Team in order to better devote his time to other projects. In March, Colin Percival was named as a second Deputy Security Officer, joining Dag-Erling Smørgrav in that position. The current Security Team membership is published on the web site. So far in 2005, four security advisories have been issued concerning problems in the base system of FreeBSD, three of which were specific to FreeBSD. The Vulnerabilities and Exposures Markup Language (VuXML) document has continued to be updated by the Security Team and the Ports Committers documenting new vulnerabilities in the FreeBSD Ports Collection. As of April 17, 127 entries have been added in 2005 bringing the FreeBSD VuXML file up to a total of 422 entries. In the past months both the VuXML web site and the FreshPorts VuXML integration have been improved. The VuXML web site has had a face lift and, among other things, each package now has a separate web page which lists all documented vulnerabilities for the particular package. CVE information is now also included directly on the VuXML web site. Finally, the first few months of 2005 also saw FreeBSD 4.8 -- the first release to be offered "extended support" -- reach its designated End of Life. The currently supported releases are FreeBSD 4.10, 4.11, and 5.3. _________________________________________________________________ FreshPorts URL: http://www.freshports.org/ Contact: Dan Langille <dan_at_langille.org> This is the first status report for FreshPorts. FreshPorts started in early 2000 and now contains over 170,000 commits. FreshPorts is primarily concerned with port commits, but actually processes and records all commits to the FreeBSD source tree. Its sister site, FreshSource uses the same database as FreshPorts but has a wider reporting scope. In recent months, FreshPorts has been enhanced to process and include VuXML information. In addition, RESTRICTED and NO_CDROM have been added to list of things that FreshPorts keeps track of. For unmantained ports, we recently added this message: There is no maintainer for this port. Any concerns regarding this port should be directed to the FreeBSD Ports mailing list via ports_at_FreeBSD.org FreshPorts, with direct and indirect support from the FreeBSD community, continues to evolve and to provide a great tool for users and developers alike. Open tasks: 1. Provide a copy/paste method for updating watch lists 2. improvement of query times for "People watching this port, also watch" 3. pagination of commits within a port 4. pagination of watch lists 5. create an RSS feed for individual watch lists _________________________________________________________________ GELI - GEOM class for providers encryption URL: http://perforce.freebsd.org/depotTreeBrowser.cgi?FSPC=//depot/user/pjd /geom%5fclasses/sys/geom/eli&HIDEDEL=NO URL: http://perforce.freebsd.org/depotTreeBrowser.cgi?FSPC=//depot/user/pjd /geom%5fclasses/sbin/geom/class/eli&HIDEDEL=NO Contact: Pawel Jakub Dawidek <pjd_at_FreeBSD.org> GELI is a GEOM class used for GEOM providers encryption. I decided to work on this, as I needed some feature, which cannot be found in similar projects. Here is the list of features, I found interesting: * makes use of crypto(9) * if there is a crypto hardware available, GELI will run cryptography on it automatically; if not, it starts dedicated kernel thread and do crypto software work in there * supports many cryptographic algorithms (AES, Blowfish, 3DES) * is able to take key components from many sources at once (user entered passphrase, random bits from a file, etc.) * allows to encrypt root partition * user will be asked for the passphrase before root file system is mounted * uses "PKCS #5: Password-Based Cryptography Specification Version 2.0" for user passphrase protection (optional) * allows to use two independent keys (e.g. "user key" and "company key") * is fast * GELI does simple sector-to-sector encryption * allows to backup/restore Master Keys, so when user have to quickly destroy keys, it is able to get the data back by restoring keys from the backup * provider can be configured at attach time to automatically detach on last close (so user don't have to remember to detach after unmounting file system) * allows to attach provider with a random, one-time keys * useful for swap partitions and temporary file systems Open tasks: 1. Code audit/review is more than welcome! _________________________________________________________________ GSHSEC - GEOM class for handling shared secret URL: http://www.freebsd.org/cgi/man.cgi?query=gshsec&apropos=0&sektion=0&ma npath=FreeBSD+6.0-current&format=html Contact: Pawel Jakub Dawidek <pjd_at_FreeBSD.org> GSHSEC is a GEOM class used for handling shared secret data between multiple GEOM providers. For every write request, SHSEC class splits the data using XOR operation with random data, so N-1 providers gets just random data and one provider gets the data XORed with the random data from the other providers. All of the configured providers must be present in order to reveal the secret. The class is already committed to HEAD and RELENG_5 branches. _________________________________________________________________ if_bridge from NetBSD URL: http://www.fud.org.nz/~andy/if_bridge.diff Contact: Andrew Thompson <andy_at_fud.org.nz> This project aims to import the bridging code and interface from NetBSD and OpenBSD. The bridge is a cloned interface which can be modified by ifconfig and brconfig. It supports assigning an IP address directly to the bridge (e.g. bridge0) instead of one of the member interfaces, and can be used with tcpdump to inspect the bridged packets. The code also supports spanning tree (802.1D) for loop detection and link redundancy. Any pfil(9) packet filter can be used to filter the bridged packets. Open tasks: 1. Testing performance and functionality against the existing bridge code. Testers welcome! _________________________________________________________________ IMUNES - a FreeBSD based kernel-level network topology emulator URL: http://www.imunes.net/ Contact: Miljenko Mikuc <miljenko_at_tel.fer.hr> Contact: Marko Zec <zec_at_tel.fer.hr> IMUNES is a scalable kernel-level network topology emulator based on FreeBSD. In IMUNES each virtual node operates on its private instance of network stack state variables, such as routing tables, interface addresses, sockets, ipfw rules etc. Most if not all existing FreeBSD application binaries, including routing protocol daemons such as quagga or XORP, can run unmodified within the context of virtual nodes with no noticeable performance penalty. Complex network topologies can be constructed by connecting the virtual nodes through netgraph-based link-layer paths. A GUI tool allows for simple and intuitive network topology specification, deployment and management. The current version of IMUNES is based on FreeBSD 4.11-RELEASE and supports IPv4. _________________________________________________________________ Infrastructure Cleanup Contact: Warner Losh <imp_at_freebsd.org> Contact: Takahashi Yoshihiro <nyan_at_freebsd.org> Unglamorous cleanup of the code base continues. The focus of recent efforts have been to reduce the number of machine #ifdefs that are in the machine independent code. In addition, we're also trying to increase code sharing between pc98 and i386 ports and reduce the number of #ifdef PC98 instances in the tree. In addition, a number of cleanup tasks are underway for different parts of the kernel that are more complicated than necessary. Recently, the pccard code's allocation routines were simplified to reassign ownership of resources more directly than before. The search is on for other areas that can benefit from cleanup. Open tasks: 1. On pc98, there's no such thing as an ISA bus. It is desirable to move to having cbus appear in the probe messages. This would also allow for additional segregation of pc98 specific code in the drivers and eliminate many ifdefs. Ideally, isa and cbus would share a common newbus ancestor class so their similarities can be exploited (they both have PNPBIOS enumeration methods, for example). 2. cbus devices can have complicated resources. There's support for vectors of resources. Yet there's no support for populating a vector of resources from the plug and play information. Doing so would help the complex world of pc98 a lot, and the odd edge cases in i386 (floppy, ata) a little. 3. The hints mechanism provides a way to associate hardware with drivers and resource that would otherwise be completely unknown to the system. A refinement in the hints mechanism to allow matching of driver instances to resources is desirable. This would allow one to hardwire sio0 to 0x2f8, even when the serial device in the plug and play resource list (or acpi resource list) is listed second. A further refinement could also be wiring sio0 to "port B" as defined by acpi or some other enumeration method. Chances are good that these seemingly related concepts may need separate implementations due to the decision points for unit assignment. 4. Pccard, cardbus and usb probe their devices after interrupts are enabled. It would be desirable to hook into new kernel APIs to allow the mounting of root to be put off until those systems know that they are done with their initial probe of the devices present at boot. _________________________________________________________________ Interrupt Latency Contact: Warner Losh <imp_at_freebsd.org> I've setup a test system to measure interrupt latency on FreeBSD 5.3 and current. So far I've measured the baseline latency for a 300MHz embedded cyrix based single board computer. I've tried a number of different strategies to optimize the interrupt path. Most of these strategies resulted in some improvement of the time it takes to get from the start of the interrupt servicing to the driver's ISR. These improvements turned out to be about 1-2% of the processing times on this single board computer, but a wash on faster machines. However, the time between when the interrupt should happen, and when FreeBSD starts to service the interrupt is the dominant factor in these measurements. Despite the fact that these are fast interrupt handlers (so the scheduler is out of the loop), I routinely see average latencies of 18us, with large variations (on the order of 5us standard deviation). Open tasks: 1. I need to measure the latencies with 4.x and current to characterize the differences more precisely. I'm especially interested in the effects on interrupt latency that the elimination of mixed mode will cause. 2. I need to characterize different parts of our ISR routines to see if some of the variation I've seen so far can be reduced by improved coding techniques. 3. I need to re-run my tests with 5.4 and summarize my results in a paper. _________________________________________________________________ IPv6 Support for IPFW URL: http://lists.freebsd.org/pipermail/cvs-all/2005-April/116671.html Contact: Brooks Davis <brooks_at_FreeBSD.org> In April 18th, I committed support for IPv6 to IPFW. This support was written by two student of Luigi's, Mariano Tortoriello and Raffaele De Lorenzo. I updated it to use PFIL_HOOKS and fixed a few minor issues. As of this commit, IP6FW should be considered deprecated in favor of IPFW. It should be possible to MFC this change to 5.x, but that is not currently planned. Open tasks: 1. Testing. 2. IP6FW to IPFW migration guide. 3. Patches relative to 5-STABLE. _________________________________________________________________ libthread Contact: David Xu <davidxu_at_freebsd.org> libthread is a pure 1:1 threading library, it had stayed in my perforce branch for a long time, recent it was imported into source tree and replaced libthr. The purpose of the work is to improve 1:1 threading on FreeBSD, the library is designed in mind that simplest is best, currently it can run almost all of the applications libpthread can run, but gives you better SMP performance. The library size is smaller than libpthread. Currently it supports i386, AMD64, sparc64 and ia64 and may support alpha, powerpc and arm. I didn't do many tests on sparc64 and ia64, I only tested it on FreeBSD cluster machines. For i386, I always used LDT, but know that Peter committed GDT code, and now there is no 8191 threads limitation anymore. libthread_db was updated to support debugging the new libthr. It is an assistant library used by gdb to debug threaded process, that understands internal detail of thread libraries. I have improved it a bit to support event reports for libthr, currently it can report thread creation and death events. That means a thread that was created and died will be reported to the user regardless if you are tracking it or not. Open tasks: 1. I am working on thread creation performance, currently it needs considerable number of libc functions and syscalls to create a thread, I would like to introduce a syscall to create a thread in atomicaly. That means one syscall will setup thread entry, tls, and signal mask and PTHREAD_SCOPE_PROCESS/SYSTEM; in future maybe even CPU affinity masks, when userland entry code is executed, the thread is already fully setup. 2. Process shareable synchronization objects. In Current FreeBSD does not support this specification. The idea about the shareable mutex and others is like other systems did, one can use mmap() to create a shared memory page, and put a pthread synchronization object in the page, multiple processes use the shared object to control resource access. I am not working on it, if someone is interested, please let me know. _________________________________________________________________ Low-overhead performance monitoring for FreeBSD URL: http://people.freebsd.org/~jkoshy/projects/perf-measurement Contact: Joseph Koshy <jkoshy_at_FreeBSD.org> Many modern CPUs have on-chip performance monitoring counters (PMCs) that can be used to count low-level hardware events like instruction retirals, branch mispredictions, cache and TLB misses and the like. PMC architectures and capabilities vary between CPU vendors and between CPU generations from the same vendor, making the creation of portable applications difficult. This project attempts to provide a uniform API for applications to use, and the necessary infrastructure to "virtualize" and manage the available PMC hardware resources. The creation of performance analysis tools that use this infrastructure is also part of the project's goals. Work since the last status report: * Support for Intel Pentium-Pro/Pentium-II/Pentium-III/Pentium-M/Celeron style PMCs has been added. * The Pentium-4/HTT machine dependent layer has been overhauled. * A Python language interface to the C library interface pmc(3) has been written. * Many bugs have been fixed and documentation has been updated. Open tasks: 1. The code needs to be tested on Intel Pentium-M, Celeron, Pentium II and Pentium Pro CPUs. _________________________________________________________________ Many subdirs for UFS URL: http://groups-beta.google.com/group/muc.lists.freebsd.fs/browse_frm/th read/a36d1143d695287e/40cad00cf2c0823b?hl=en#40cad00cf2c0823b Contact: David Malone <dwmalone_at_freebsd.org> I'm currently looking at the limit on the number of subdirectories a directory can have in UFS. There is currently a limit of 32K subdirectories because of the 16 bit link count field in both struct stat and the on-disk inode format. The thread above shows that dirhash provides acceptable performance for directories with 100k subdirectories using a prototype patch. Two options for allowing many subdirectories seem to exist: changing the link counting scheme for directories and expanding the link count field. The prototype patch implements the first scheme and there are plans to investigate the second scheme (which may require an ABI change). _________________________________________________________________ Move ARP out of routing table URL: http://people.freebsd.org/~qingli/ Contact: Qing Li <qingli_at_freebsd.org> I have finished the basic functionality for both IPv4 and IPv6. The userland utilities ("arp" and "ndp") have been updated. I have tested the changes with "make buildworld". I have been testing the new code in a production environment and things appear to be stable. Gleb Smirnoff (glebius_at_FreeBSD.org) has provided review comments and I have incorported these feedback into the patch. I have discussed the IPv6 changes with two of the core KAME developers during the last IETF meeting in March 2005. They indicated that these changes may result in divergence from the KAME project but that is not necessarily a bad thing. Open tasks: 1. I am waiting for review feedback from my mentor Andre. I need locking experts to help me fix my giant-lock shortcut. I am hoping to send out the code for wider review soon. _________________________________________________________________ netgraph(4) status report URL: http://www.freebsd.org/cgi/man.cgi?query=ng_netflow&manpath=FreeBSD+6. 0-current URL: http://www.freebsd.org/cgi/man.cgi?query=ng_ipfw&manpath=FreeBSD+6.0-c urrent URL: http://people.freebsd.org/~glebius/totest/ng_nat/ Contact: Gleb Smirnoff <glebius_at_FreeBSD.org> This report covers period since August 2004 until April 2005. New nodes. Two new nodes have been added to base FreeBSD distribution. ng_netflow(4) node, which implements NetFlow version 5 accounting of IPv4 packets. ng_ipfw(4) node, which diverts packets from ipfw(4) to netgraph(4) and back. A well known ng_ipacct node has been added to ports tree. SMP. Nodes, which need to allocate unique names have been protected with mutex in RELENG_5, and subr_unit allocator in HEAD. Nodes, which need to run periodical jobs were reworked to use mpsafe ng_callout() API. ng_tty(4) node has been overhauled to be compatible with debug.mpsafenet=1. NetGraph ISR and callout are now declared MPSAFE in HEAD. NetGraph flow control. Two nodes ng_ether(4) and ng_cisco(4) have been improved to emit flow control messages to upstream node, when state of link changes. New link failure detection method have been introduced in ng_one2many(4) node - listening to these flow control messages from downstream. Open tasks: 1. more SMP testing of many nodes 2. review locking of graph restructuring 3. ng_nat node - an in-kernel natd(8) 4. make ng_bridge(4) multithreaded _________________________________________________________________ OpenBSD packet filter - pf URL: http://pf4freebsd.love2party.net/ URL: http://people.freebsd.org/pf37/ Contact: Max Laier <mlaier_at_freebsd.org> OpenBSD is about to release version 3.7 . There are patches available to catch up with the development done in OpenBSD 3.6 and 3.7. These patches are in an early stage, but ready for testing, please help. Otherwise there was not much activity on pf, as it already is quite stable. Other work, such as CARP and if_bridge are having impact on pf in FreeBSD however, please see the respective reports. Open tasks: 1. Alpha/Betatesting of the 3.7 import 2. Testing with if_bridge _________________________________________________________________ Pipe namespace added to portalfs URL: http://www.spinellis.gr/blog/20050413/index.html Contact: Diomidis Spinellis <dds_at_FreeBSD.org> A new sub-namespace, called pipe, has been added to portalfs. The pipe namespace executes the named command, starting back at the root directory. The command's arguments can be provided after the command's name, by separating them with spaces or tabs. Files opened for reading in the pipe namespace will receive their input from the command's standard output; files opened for writing will send the data of write operations to the command's standard input. The pipe namespace allows us to perform scatter gather operations without using temporary files, create non-linear pipelines, and implement file views using symbolic links. _________________________________________________________________ Ports Collection URL: http://www.freebsd.org/ports/ URL: http://portsmon.firepipe.net/index.html URL: http://www.freebsd.org/portmgr/index.html Contact: Mark Linimon <linimon_at_FreeBSD.org> As this report was being written, the 5.4 release was ongoing. A new charter for the Ports Management (portmgr) team was approved by core and has been posted at the URL above. In addition, two other new pages describe the policies of the team, and the range of QA activities both during and between releases. Due to being absent from email discussions for some time, Oliver Eikemeier (eik) was moved to non-voting status on portmgr. We have added several new and very active committers recently; this is helping us to keep the PR count low even with the large numbers of new ports that have been added. Several more iterations of infrastructure changes have been tested on the cluster and committed; see /usr/ports/CHANGES for details. Updates have occurred to x.org, GNOME, KDE, and perl. There have been some updates to the Porter's Handbook, but more sections are still in need of updates to include recent changes in practices. The ports collection now contains almost 12,750 ports. Open tasks: 1. Further progress has been made in cracking down on ports that install files outside the approved directories and/or do not deinstall cleanly (see "Extra files not listed in PLIST" on pointyhat ) and this will remain a focus area. We appreciate everyone who has sent in PRs or committed fixes. 2. Demand for new features and revisions for bsd.port.mk is still very high and the portmgr team is trying to work through them all. 3. We still have a large number of PRs that have been assigned to committers for some time (in fact, they constitute the majority). One goal of portmgr in the coming months is to try to reduce this number, and we would like to ask our committers to help us out as much as possible. _________________________________________________________________ PowerPC Port Contact: Peter Grehan <grehan_at_FreeBSD.org> Progress continues. X.Org 6.8.1 server has been up and running on a number of different Macs, and the work is being merged into 6.8.2. There have been successful installs on Mac Minis _________________________________________________________________ Removable interface improvements. URL: http://people.freebsd.org/~brooks/pubs/eurobsdcon2004/ URL: http://www.freebsd.org/projects/dingo/ Contact: Brooks Davis <brooks_at_FreeBSD.org> This project is an attempt to clean up handling of network interfaces in order to allow interfaces to be removed reliably. Current problems include panics if Dummynet is delaying packets to an interface when it is removed. I am currently working to remove struct ifnet's from device driver structures to allow them to be managed properly upon device removal. I believe I have removed all known instances of casting a struct ifnet pointer to something else (except that that are just magic values and not real struct ifnets.) I will begin committing these changes to the tree shortly and will then add a new function if_alloc() that will allocate struct ifnets. if_detach() will be modified to destroy them. _________________________________________________________________ Secure Updating URL: http://www.daemonology.net/portsnap/ URL: http://www.daemonology.net/freebsd-update/ Contact: Colin Percival <cperciva_at_FreeBSD.org> Shortly before the ports freeze for FreeBSD 5.4, I released a new version of Portsnap. In addition to being secure and more efficient than CVSup, this latest version distributes INDEX, INDEX-5, and INDEX-6 files, thereby eliminating the need to run "make fetchindex" and ensuring that the ports INDEX will match the existing ports tree. In addition, portsnap builds have now moved onto hardware managed by the FreeBSD project, thereby sharply increasing portsnap's chances of survival if I get hit by a bus. In early February hardware problems caused both FreeBSD Update and Portsnap to stop functioning for a few days, but those were resolved thanks to a server donated by layeredtech.com. I intend bring Portsnap into the FreeBSD base system before the end of the month, followed by FreeBSD Update a few months later. _________________________________________________________________ Status Report for FreeBSD ATA driver project Contact: Søren Schmidt <sos_at_FreeBSD.org> ATA mkIII has been committed to -current after a couple of month testing as patches post on -current and 5-stable. I will continue to provide patches for 5-stable for those that need up-to-date ATA support there. Here a short rehash of what mkIII brings: ATA is now fully modular so each part can be loaded/unloaded at will to provided the wanted functionality. Much improved SATA support that support hotplug events on controllers that support it (Promise, SiS, nVidia so far) ie the system will automagically detect when SATA devices come and go and add/delete device entries etc. Much improved ATA RAID support. The ata-raid driver has been largely rewritten to take advantage of the features the improved infrastructure provides, including composite ATA operations etc. The rebuild functionality has been changed to rebuild on userland reads, so a simple dd of the entire array will get it rebuild (what atacontrol now does). This means that the resources used for this can be better tailored to the actualy usage pattern if needed. ATA RAID now supports 10+ different RAID metadata formats, so most BIOS defined ATA RAID arrays can be picked up and used. The number of metadata formats that can be created from within FreeBSD is still limitted though and is not a high priority feature right now. The lowlevel infrastructure of the ATA driver has been refined even further to support "strange" chipsets much more easily and in most case transparent to the higher levels. This to easy ports to new platforms where ATA controllers doesn't necessarily have the x86 legacy layout. Lots of bug fixes and corrections all over the driver proper. The rework of the infrastructure has revealed bugs and deficiencies that has been fixed in the process of modulerising ATA and making the infrastructure more generic, and hopefully easier to understand. The work continues to keep ATA on top of new chipsets and other advancements in the ATA camp. SATA ATAPI support is in the works and so are support for NCA/TCQ (tags). Donations of unsupported hardware is the way to get it supported as I'm way out of my budget for new hardware for the next decade or so according to my wife :) Open tasks: 1. Lots of testing wanted, especially SATA and RAID support _________________________________________________________________ Storage driver SMPng locking Contact: Scott Long <scottl_at_freebsd.org> Several storage drivers have been taken out from under the Giant mutex in the past few months. Thanks to sponsorship from FreeBSD Systems, Inc and ImproWare, AG, Switzerland , the LSI MegaRAID (AMR) and IBM/Adaptec ServeRAID (IPS) drivers have been locked. SMPng locking is a key step in improving the performance of system drivers in FreeBSD 5.x and beyond, and both of these drivers are showing the benefits of this. FreeBSD 5.4 will contains these improvements when it is released. Similar work is ongoing with the 3WARE Escalade (TWE) driver, and preliminary patches have been made available to testers. I hope to have this driver complete in time for the next FreeBSD release. Unfortunately, most benefits can only be gained from pure block storage drivers such as the ones mentioned here due to the SCSI subsystem in FreeBSD (CAM) not be locked itself at this time. It is possible, however, to lock a CAM sub-driver and bring the driver's interrupt handler out from under Giant for a partial gain. The Sun FAS366 SCSI driver (ESP) operates like this. Volunteers to lock other drivers or to tackle locking CAM are gladly accepted, so please contact me if you are interested. _________________________________________________________________ Support for telephone hardware (aka Zaptel) URL: http://www.digium.com/index.php?menu=hardware_products Contact: Maxim Sobolev <sobomax_at_FreeBSD.org> Contact: Oleksandr Tymoshenko <gonzo_at_pbxpress.com> Contact: Max Khon <fjoe_at_FreeBSD.org> During the last 2 months lot of progress has been made. Existing support for TDM400 (FXO/FXS) has been significantly improved. Drivers for PRI and BRI cards have been added and now should be considered beta-quality. Open tasks: 1. More testing of PRI/BRI drivers. 2. Add support for channelized DS3 card(s). _________________________________________________________________ twa driver URL: http://www.freebsd.org/cgi/cvsweb.cgi/src/sys/dev/twa/ URL: http://www.freebsd.org/cgi/cvsweb.cgi/src/sys/modules/twa/ Contact: Vinod Kashyap <vkashyap at amcc.com> A newly re-architected twa(4) driver was committed to 6 -CURRENT on 04/12/2005. Highlights of this release are: 1. The driver has been re-architected to use a "Common Layer" (all tw_cl* files), which is a consolidation of all OS-independent parts of the driver. The FreeBSD OS specific portions of the driver go into an "OS Layer" (all tw_osl* files). This re-architecture is to achieve better maintainability, consistency of behavior across OS's, and better portability to new OS's (drivers for new OS's can be written by just adding an OS Layer that's specific to the OS, by complying to a "Common Layer Programming Interface (CLPI)" API. If there's interest in porting the 3ware driver to any other OS, you may contact ctchu at amcc.com to get a copy of the CLPI specifications. 2. The driver takes advantage of multiple processors. It does not need to be Giant protected anymore. 3. The driver has a new firmware image bundled, the new features of which include Online Capacity Expansion and multi-lun support, among others. More details about 3ware's 9.2 release can be found here: http://www.3ware.com/download/Escalade9000Series/9.2/9.2_Release_N otes_Web.pdf _________________________________________________________________ Update of the Linux userland infrastructure Contact: Alexander Leidinger <netchild_at_FreeBSD.org> Contact: Emulation Mailinglist <freebsd-emulation_at_FreeBSD.org> The update to RedHat 8 as discussed in the last status report went smoothly (just some minor glitches which got resolved fast). As a next step a cleanup/streamlining and the possibility of overriding the default linux base is in progress. This depends on changes which need at least one testrun on the ports build cluster, so the final date for those changes depends upon the availability of the cluster resources. Open tasks: 1. Refactoring the common RPM code into bsd.rpm.mk. 2. Determining which up-to-date Linux distribution to use as the next default linux base. Important criteria: + RPM based (to be able to use the existing infrastructure) + good track record regarding availability of security fixes + packages available from several mirror sites + available for several hardware architectures (e.g. i386, amd64, sparc64; Note: not all architectures have a working linuxolator for their native bit with, but as long as there are no userland bits available, no motivation regarding writting the kernel bits will arise) 3. Moving the linuxolator userland to an up-to-date version (see above). _________________________________________________________________ Wireless Networking Support Contact: Sam Leffler <sam_at_freebsd.org> Several new drivers by by Damien Bergamini were brought into the tree: iwi, ipw, ral, and ural. WPA-PSK support for the ndis driver was contributed by Arvind Srinivasa. A new tx rate control algorithm for the ath driver was contributed by John Bicket. It will become the default algorithm shortly. Work on multi-bss support is going on outside the cvs tree. A presentation on this work will be given at BSDCan 2005 and the slides for the talk will be made available after. Open tasks: 1. Drivers other than ath and ndis need updates to support the new security protocols. 2. hostapd needs work to support the IAPP and 802.11i preauthentication protocols (these are simple conversions of existing Linux code). 3. The openbsd dhclient program has been ported but needs a developer that will maintain it once it is brought into cvs. _________________________________________________________________ XenFreeBSD - FreeBSD on Xen URL: http://www.cl.cam.ac.uk/Research/SRG/netos/xen/ URL: http://xen.bkbits.net/ Contact: Kip Macy <kmacy_at_fsmware.com> FreeBSD 5.3 runs on the stable and the development branches of xen and is now checked into both trees. Over the next couple of weeks I will be adding improvements for better batching of page table updates and SMP support. Open tasks: 1. FreeBSD support for running as Domain 0, i.e. running as the hosting operating system. 2. FreeBSD support for VM checkpoint and migration. _________________________________________________________________Received on Thu Apr 21 2005 - 15:52:26 UTC
This archive was generated by hypermail 2.4.0 : Wed May 19 2021 - 11:38:32 UTC