HEADS UP: Adding /usr/local/etc/rc.d to the base rcorder

I have not heard any negative responses to this post (or the forwards to
-arch and -ports), and I've had a few private, and one public vote of
support, so barring objection I plan to commit this to HEAD Thursday night,
California time (roughly 24 hours from now).

In an ideal world, you shouldn't notice anything bad happen. Of course, we
may not live in an ideal world. :) Obviously, if you have a chance to test
this before tomorrow and provide feedback, it would be greatly appreciated.

Thanks,

Doug

-------- Original Message --------
Subject: Adding /usr/local/etc/rc.d to the base rcorder
Date: Tue, 29 Nov 2005 03:13:37 -0800
From: Doug Barton <dougb_at_FreeBSD.org>
Organization: http://www.FreeBSD.org/
To: freebsd-rc_at_FreeBSD.org

Howdy,

The idea to incorporate the scripts in the local startup directories
(currently defined as /usr/local/etc/rc.d /usr/X11R6/etc/rc.d, or
substitute whatever your PREFIX/LOCALBASE/X11BASE is), into the overall
rcorder that the scripts in /etc/rc.d follow has been around basically since
the new rc.d framework was introduced, but has been very difficult to
implement. A thread on the freebsd-rc list back in June discussed the
ramifications of this change, and how it might possibly be implemented
safely. Since then, I've put together a set of patches that implement one
approach to this change.

Now that 6.0-RELEASE is done, I'd like to move fairly quickly in
implementing this in HEAD, and once the bogons are shaken out, I'd like to
MFC it to RELENG_6 prior to 6.1-RELEASE.

I want to point out several things at the outset. This is _one_ possible
approach to this problem. One that I believe will work well, and minimizes
the pain of the transition. However, I am not necessarily tied into every
detail of my patch, or even the approach generally. However, we need to move
forward on this, and without compelling reasons to do things differently, I
am confident that this approach will work.

1. In order for this to work, we need to first get the disks mounted.
(Thanks to Brooks for this, and other important insights). Therefore I'm
proposing that we split the rcorder function into two parts, one "early"
stage that takes care of everything up to mountcritremote; then redo
rcorder, skipping the scripts that were done in the early stage, and
incorporating the scripts in /usr/local/etc/rc.d, etc. This is not a
"perfect" approach, as theoretically some cases where a local package might
want to insert itself into rcorder before mountcritremote, however given the
various tradeoffs we have to make (for example, diskless booting), this is
at least a reasonable place to start. Compare
http://people.freebsd.org/~dougb/rcorder.all and
http://people.freebsd.org/~dougb/rcorder.early to see how the ordering as it
exists in HEAD at the moment would be affected. Of 127 scripts, 52 would be
in the early stage. (/etc/rc.d/tmp is being forced into the early stage by
my patch in order to avoid non-deterministic behavior when local scripts are
added to rcorder. This could just as easily be forced into the late stage.)

My method for implementing the distinction between the early boot stage and
the later is derived from an idea suggested by J.R. Oldroyd. It stops
processing in the early stage once mountcritremote is done. The "late" stage
first finds scripts in the local directories, then runs rcorder again over
both lists. It starts processing after mountcritremote is reached.

2. The next aspect of this plan is how to manage the transition for the
ports. There are two phases to this. First, adding code to /etc/rc (and
rc.subr) to pick up those scripts in local_startup that are ready to be
added to rcorder. This is done by grep'ing for '^# PROVIDE:' in the script.
This method is not entirely foolproof, as for example the cups.sh startup
script is (from our perspective) an "old style" script, however it contains
the PROVIDE line for NetBSD's purposes. Thus, some care will have to be
taken during the transition period to avoid problems.

By my count, there are roughly 640 ports that install some sort of startup
script. Of these, roughly 345 have transitioned to the new style of rc.d
scripts (based on the presence of USE_RCORDER/USE_RC_SUBR in the Makefile).
Thus, there are roughly 300 ports with scripts that would _potentially_ have
problems. Of these, I'm confident that the vast majority would work without
modification, as the number of possibly fatal error conditions are very
small.

The situation with cups.sh above is the only one I've encountered, but there
may be others. There are two more potential problems with the new style
scripts that are already in the ports tree. First, there are probably some
scripts that have errors in them that will not be exposed until they are run
within the rcorder context. The other problems that are almost sure to arise
are scripts whose ordering needs to be adjusted (via REQUIRE/BEFORE, etc.).
These things will need to happen in order for the transition to be
successful in any case, so although it is sure to be a non-zero amount of
work, it's work that will have to be done regardless of what method is
chosen to incorporate the local_startup scripts into rcorder.

The other part of this transition is to modify the localpkg script. This is
done using some ideas and code from J.R. Oldroyd. Brooks, and myself. First
we sort out the scripts that start with a number (like 000.foo.sh) and run
them in numerical order, as a lot of work has gone into this style of
ordering already. Then we run the scripts that start with a letter. The
original idea here was to use rcorder in localpkg, however in my testing I
found that it's simpler to just run the new style scripts in the base
rcorder, and run everything else in localpkg. Therefore, the function that
searches for these scripts eliminates any that use the new rc.d style first.
These changes dramatically simplify the localpkg script.

3. The last part of this proposal is to apply the same changes to how we
pick up local scripts in rc to the shutdown process, both in rc.shutdown and
in localpkg.

My patch to implement all this is at
http://people.freebsd.org/~dougb/local-rcorder.diff. Comments are welcome.


Regards,

Doug

-- 

    This .signature sanitized for your protection