cvs-src summary for August 9-16 - special last-minute shopping edition

From: Mark Johnston <mjohnston_at_skyweb.ca>
Date: Tue, 17 Aug 2004 20:29:35 -0500
Here's this week's giant summary.  As you probably know, a code freeze went 
into effect first thing on the 17th, which is the main reason for the length 
of this summary.

In other news, Omer Faruk Sen has kindly arranged to host a mailing list 
exclusively for the summaries, in case you'd like to get them separately from 
your current_at_ mail.  The address of this list is 
freebsd-cvs-summary_at_lists.enderunix.org; to subscribe, send mail to 
freebsd-cvs-summary-subscribe_at_lists.enderunix.org.  Thanks, Omer!

FreeBSD cvs-src summary for 09/08/04 to 16/08/04
++++++++++++++++++++++++++++++++++++++++++++++++
This is a regular weekly summary of FreeBSD's cutting-edge development.
It is intended to help the FreeBSD community keep up with the fast-paced
work going on in FreeBSD-CURRENT by distilling the deluge of data from
the CVS mailing list into a (hopefully) easy-to-read newsletter.  This
newsletter is marked up in reStructuredText_, so any odd punctuation
that you see is likely intended for the reST parser.

.. _reStructuredText: http://docutils.sourceforge.net/rst.html

You can get old summaries, and an HTML version of this one, at
http://www.xl0.org/FreeBSD/.  Please send any comments to Mark Johnston
(mark at xl0.org).

If you would like to get the summary without subscribing to current_at_,
please send mail to freebsd-cvs-summary-subscribe_at_lists.enderunix.org.
Thanks to Omer Faruk Sen and EnderUNIX for hosting this list.

For Lukasz Dudek and Szymon Roczniak's Polish translations of these
summaries, which may lag the English ones slightly, please see
http://mocart.pinco.pl/FreeBSD/.

.. contents::

======
Errata
======
I reported last week that Nate Lawson (njl) had moved the crypto
distribution into the base system.  This was actually done by Colin
Percival (cperciva).

I also creatively described UMA as standing for "Uniform Memory Access",
when in reality it stands for "Universal Memory Allocator".

Apologies to Colin and Nate, and anyone confused by the UMA mixup.

============
New features
============
ADAPTIVE_GIANT turned on by default
-----------------------------------
Robert Watson (rwatson) enabled ADAPTIVE_GIANT by default in the GENERIC
kernel.  ADAPTIVE_GIANT makes the Giant system mutex adaptive.  That is,
when a thread wants to lock the Giant mutex, but it is already locked by a
thread on another CPU, the first thread will keep running and wait for the
lock to be released.  Normally, the thread would instead go back to sleep
and wait for its next chance to run.  On an SMP system, Robert has noted a
16% improvement on buildworld and a 30% improvement in MySQL performance.

Thanks to Colin Percival (cperciva) for sending me a pointer to this
change.

http://www.freebsd.org/cgi/mid.cgi?200408110134.i7B1YI9Y045791

GEOM support for read-only compressed disks added
-------------------------------------------------
Max Khon (fjoe) committed geom_uzip, a GEOM class that implements
read-only compressed disks.  Using geom_uzip, you can natively mount
disks compressed in cloop 2.0 format.  The FreeSBIE_ project supplied
some timings for application startup on their LiveCD, showing drastic
improvements.  Times are in minutes and seconds.

Boot        1:26    1:04
X             41       9
XFce        1:17      18
xterm         13       2
Firefox     3:02      24

.. _FreeSBIE: http://www.freesbie.org/

http://www.freebsd.org/cgi/mid.cgi?200408130940.i7Dewwq077646

GEOM RAID 3 class added
-----------------------
Pawel Jakub Dawidek (pjd) added a GEOM class for RAID 3 and a utility
named graid3 that is used to manage it.  RAID 3 spreads data across a
number of drives, using another drive to store parity data.  In the event
of any one drive failing, the parity drive can be used to recover it.  In
other words, RAID 3 is like RAID 5, but it uses a dedicated drive for
parity data.

http://www.freebsd.org/cgi/mid.cgi?200408160623.i7G6NExIO57228

Speed enhancements for mmap and pipes
-------------------------------------
Alan Cox (alc) committed code submitted by Mark W. Krentzel that changes
the VM code to use an O(log n) algorithm instead of the old linear search
when finding space to map memory.  This results in much lower overhead
for applications that mmap many regions.  Dag-Erling Smorgrav (des)
followed up, pointing out that the commit should also greatly increase
pipe performance.

http://www.freebsd.org/cgi/mid.cgi?200408130806.i7D86YJZ075107

ipfw rules can now be associated with jails
-------------------------------------------
Christian S. J. Peron (csjp) added a "jail" option to ipfw, allowing rules
to be associated with specific jail IDs.  He gives the following example::

    ipfw add count ip from any to any jail 2

http://www.freebsd.org/cgi/mid.cgi?200408122205.i7CM5lTf044327

GNU sort updated to latest version
----------------------------------
Tim Robbins (tjr) updated GNU sort to the latest version from GNU.  The
new version of sort improves the performance of merging large files with
-m, as well as fixing some POSIX conformance issues.  The semantics of
"-o" have changed so that "-o -" means to write to a file named "-", not
standard output; "-k" with a b argument has changed to skip leading blanks
in some cases.  See `Tim's followup`_ for more details.

.. _`Tim's followup`: 
http://www.freebsd.org/cgi/mid.cgi?20040812063334.GA79720

http://www.freebsd.org/cgi/mid.cgi?200408120537.i7C5bkIs005791

Routing table entries using much less memory now
------------------------------------------------
Andre Oppermann (andre) converted the system routing table to use a UMA
zone to store entries.  The effect of this is to reduce the memory used
by about 50%, from 256 bytes plus overhead to 132 bytes per entry.  In
testing, Andre reports he was able to load roughly 1.2 million routes
before running out of space.

http://www.freebsd.org/cgi/mid.cgi?200408111726.i7BHQuPC078939

Support for new Adaptec architecture and 2130S adapter added
------------------------------------------------------------
Scott Long (scottl) added support for the Adaptec RAID-On-Chip
architecture, introducing support for the Adaptec 2130S adapter.  Scott
thanks Adaptec for providing hardware for this work.

http://www.freebsd.org/cgi/mid.cgi?200408130144.i7D1i9Nc060041

Support for VIA C3 Nehemiah AES encryption
------------------------------------------
Mark Murray (markm) committed support for the Via C3 Nehemiah chip's
embedded "Padlock" AES encryption.  This code was imported from the
yet-unreleased OpenSSL 0.9.5.

http://www.freebsd.org/cgi/mid.cgi?200408141338.i7EDcZTE040442

ACPI locking added
------------------
Nate Lawson (njl) added locking to much of the ACPI code, preparing it to
use on SMP without the Giant mutex.

(many)

32-bit linux emulation on 64-bit platforms introduced
-----------------------------------------------------
Tim Robbins (tjr) committed preliminary support for running 32-bit Linux
binaries on the AMD64 platform.  This is enabled with the COMPAT_LINUX32
option.  More work is still required before both 32-bit and 64-bit Linux
emulation can be used at once.

http://www.freebsd.org/cgi/mid.cgi?200408160755.i7G7t6GG064950

===============
Notable changes
===============
sysinstall changed to X.org instead of XFree86
----------------------------------------------
Ken Smith (kensmith) committed patches from Eric Anholt (anholt) to
change sysinstall from installing XFree86_ to installing the `X.org`_ X
server.  There was a major controversy in February 2004 regarding license
changes in XFree86; `a summary`_ can be found at `Linux Today`_.  FreeBSD
initially switched to X.org in July (see `Eric's HEADSUP`_).

.. _XFree86: http://www.xfree86.org
.. _`X.org`: http://www.x.org
.. _`a summary`: http://linuxtoday.com/developer/2004021803026NWDTLL
.. _`Linux Today`: http://www.linuxtoday.com
.. _`Eric's HEADSUP`: 
http://lists.freebsd.org/pipermail/freebsd-current/2004-July/032267.html

http://www.freebsd.org/cgi/mid.cgi?200408100218.i7A2ISVL090391

pcic for NEWCARD removed
------------------------
Warner Losh (imp) removed the pcic driver for NEWCARD, as it was "badly
broken".

http://www.freebsd.org/cgi/mid.cgi?200408111724.i7BHOgxY078756

RANDOM_IP_ID kernel option becomes a sysctl
-------------------------------------------
David Malone (dwmalone) eliminated the RANDOM_IP_ID kernel option, which
randomized the identifier field on IP packets, and added a new sysctl,
net.inet.ip.random_id.  Setting the sysctl to 1 gives the same effect as
setting RANDOM_IP_ID did.  This change is based on similar work done in
NetBSD.

http://www.freebsd.org/cgi/mid.cgi?200408141532.i7EFWKAT043669

=================
Discussion topics
=================
Preserving the user interface on an MFC
---------------------------------------
Oliver Eikemeier (eik) merged the latest pkg_install tools from -CURRENT
to -STABLE.  As part of this change, he replaced the old Perl version of
pkg_version with the new C version, which no longer supports the -c flag.
In -STABLE, the -c flag outputs commands you could use to update your
packages to the latest versions, although if used as-is, the resulting
commands will destroy dependencies.

Ceri Davies (ceri) followed up, "I don't think that this should have gone
into RELENG_4 [with pkg_version -c not working]."

Alfred Perlstein (alfred) agreed, "We don't break things that used to work
in 4.x unless there is dire need."

Doug Barton (dougb) agreed as well, saying, "This should not have
happened, and I'd like to see it backed out."

Oliver replied to all three, offering, "When anyone from re_at_ thinks it
is worth the effort, I will add an -c option to the C pkg_version code
ASAP."  He also mentioned, "I believe backing this out won't be beneficial
for -STABLE users, since they'll loose a lot of features (and speed).",
asking, "Has anyone on this thread *ever* used that option?"

Doug replied, "I am happy to let re_at_ make the decision [ . . . ]." He
also addressed the second part of Oliver's post, saying, "None of these
arguments are relevant, sorry. One of the key principles for a -stable
branch is not violating trust with our users that 'how things work is
how they will always work' without a really good reason.  'It's better,
stronger, faster!' is not a really good reason."

Ken Smith (kensmith), as a member of re_at_, said, "I would like to see the
-c added please. [ . . . ] Additional features (and speed ;-) are always
welcome/good as long as existing semantics don't change."

Oliver replied, "Ok, I'll write a patch then."

Ken responded, "First thank you, the effort is appreciated. [ . . . ] It
sounds like Oliver is agreeable to adding -c so I'm OK with his previous
commit remaining in RELENG_4 as long as adding -c doesn't take too long."

Bruce Mah (bmah) then followed up, saying "As the original author of
pkg_version (the Perl one), I wish I had never added the -c option.  Of
the users I have talked to who used this option, the majority used it
wrongly (as a package updating script).  In hindsight, this was basically
a foot-shooting device. :-("

Alfred pointed out, "But wouldn't it suck to be that user that actually
does use '-c' correctly, and to have it suddenly gone in the middle of a
'stable' release?"

Will Andrews (will) replied, "It is a broken mis-feature.  Please tell me
how you managed to make it work properly.  Do you have some sort of script
that sorts the output in order by dependency tree?  Or do you actually go
through it every time and sort it by hand? [ . . . ] portupgrade is a much
better tool.  How about simply replacing -c with a message suggesting use
of that instead?"

Ceri answered, "The fact is that the development community have an
agreement with the userbase that we will not do this.  The fact that we
all think that -c is shit doesn't change that."

http://www.freebsd.org/cgi/mid.cgi?200408111616.i7BGGOYZ074677

Network interface flag ambiguities
----------------------------------
Maksim Yevmenkin (emax) committed code to set the IFF_RUNNING flag on the
tap network interface as soon as its control device is opened.

Max Laier (mlaier) responded, "IFF_RUNNING has a different meaning (for all
I know) - 'resources allocated' i.e. address(es) assigned. See if_loop.c
for instance."

Brian Feldman (green) replied, "Having written a driver recently, the best
I could find is that IFF_RUNNING essentially means not dead..."

John Polstra (jdp) replied, "That's pretty much correct.  IFF_UP is an
administrative control that expresses the desired state of the interface.
[ . . . ] IFF_RUNNING is the driver's idea of the _actual_ state."  He
clarified, "Neither IFF_UP nor IFF_RUNNING have anything at all to do with
whether any addresses are assigned to the interface."

Roman Kurakin (rik) pointed out, "PPP state machine can remove IFF_UP. For
example if connection is not persistent and link was broken for any
reason."

John answered, "I call that a bug."

Roman argued, "This is not a bug, this is feature of protocol. Some times
link should go down (or other state from which it could go up only by
administrator (or program) intervention)."

John disagreed, saying, "PPP should clear IFF_RUNNING in that case, but
should leave IFF_UP untouched."

Roman noted, "But in that case we need some other way to bring line up
again, since we unable just to ifconfig XXX up."  He asked, "Could you
describe why is so bad that some administrative action could be canceled
due to protection or some other reasons by device state machine?"

John answered, "Well, it is standard practice to separate administrative
controls from driver status bits.", pointing out that IEEE standards use
"'adminEnable' and 'operEnable' to reflect the administratively requested
state and the actual state, respectively."

Sam Leffler (sam) responded to John's previous post, clarifying,
"IFF_RUNNING was intended to mark a device "ready for use" and should be
managed by the driver.  IFF_UP was to be administratively controlled and
any automated change is contrary to the original intent/design."

Roman replied, saying, "Ok. Since IFF_UP is used this way by many
implementations we have to agree that this is normal behavior that was
introduced by evolution and practice like word spelling could be changed
from its origin."

Sam responded, "The instances that I am aware of are misdesigns that need
correcting."

John also replied, saying, "There's a big difference between human
language and software design.  IFF_UP is misused by some implementations
because the implementors didn't understand it.  That's not evolution, it's
just a plain old bug. :-)"

http://www.freebsd.org/cgi/mid.cgi?200408110012.i7B0CRnp039953

=================
Committer changes
=================
Matt Jacob (mjacob) has returned after a brief absence; he requested his
commit bit be suspended in March.

http://www.freebsd.org/cgi/mid.cgi?200408122116.i7CLGiRW042861

===============
Other bug fixes
===============
Max Laier (mlaier) imported a fix from OpenBSD for pf.  This fix corrects
a problem that caused incorrect packet checksums when using the "scrub
random-id" option.

http://www.freebsd.org/cgi/mid.cgi?200408121359.i7CDxiHD024666

Tim Robbins (tjr) fixed a bug in gzip that could result in a crash when
processing certain corrupt input files.  This fix was also MFC'ed.

http://www.freebsd.org/cgi/mid.cgi?200408130538.i7D5ciLc069647

Martin Blapp (mbr) fixed a bug in the DHCP client that could result in it
neither renewing its old address nor requesting a new one.

http://www.freebsd.org/cgi/mid.cgi?200408161609.i7GG9oKa005749

Bill Paul (wpaul) fixed the NDISulator to support the Texas Instruments
802.11g chipset, as used in the Netgear WG311v2 wireless PCI card.  The
NDISulator allows the use of Windows network drivers under FreeBSD.  Note
that the firmware, supplied with the driver as .bin files, will need to be
installed in /compat/ndis.

http://www.freebsd.org/cgi/mid.cgi?200408161850.i7GIoK6c020520

Eric Anholt (anholt) fixed a problem in the AGP code that was resulting in
a spontaneous reboot with VIA AGPv3 chipsets when an AGPv2 card was used.
This closes `PR 69953`_.

.. _`PR 69953`: http://www.freebsd.org/cgi/query-pr.cgi?pr=69953

http://www.freebsd.org/cgi/mid.cgi?200408092101.i79L1nD8078620

Ian Dowse (iedowse) fixed a bug in the driver for the ASIX Electronics
AX88172 USB Ethernet adapter, as used in the D-Link DUBE100, the Linksys
USB200M, the Netgear FA120, and other cards.  The bug was resulting in the
interface breaking down under load.  This closes `PR 70474`_, which also
had the fix from Andrew Thompson.

.. _`PR 70474`: http://www.freebsd.org/cgi/query-pr.cgi?pr=70474

http://www.freebsd.org/cgi/mid.cgi?200408151051.i7FApLh8081711
Received on Tue Aug 17 2004 - 23:29:36 UTC

This archive was generated by hypermail 2.4.0 : Wed May 19 2021 - 11:38:06 UTC