Re: Problems building FreeBSD 9.2 on FreeBSD 10

On Jun 24, 2014, at 8:43 AM, Glen Barber <gjb_at_FreeBSD.org> wrote:

> Trimmed CC a bit.
> 
> On Mon, Jun 23, 2014 at 11:42:20PM -0600, Warner Losh wrote:
>> On Jun 23, 2014, at 8:24 PM, Glen Barber <gjb_at_freebsd.org> wrote:
>>> I sort of typed what I meant a bit backwards from what I intended to
>>> write.  What I meant (sort of) is, "I would like to discuss our forward
>>> thinking on backward-compatibility."
>>> 
>>> I fully understand forward-compatibility is not feasible.
>> 
>> We already build current back to the stable/8 branch. 7.x is no longer feasible, supported or tested. stable/10 is the only one that is required, but enough people use stable/9 machines it will work. stable/8 has one customer that is keeping it going, so I suspect it will stop working in the coming years, maybe before 11 is branched.
>> 
> 
> To be clear, I am talking about the other direction.  Meaning, being
> able to "reliably" build N-2 from head/, without needing to do silliness
> like 'make make buildworld', or "not using -jN.”

Yea, that’s never been officially supported, but generally works. In fact, in the past, you were required to have exactly the same version on the host as you were building a release for to ensure nothing weird happening. Having the full release process work across multiple major versions and have it produce identical results to the exact version built release is not well tested and caused all kinds of problems back in the day.

To make it work, you’ll need to make it work. And you’ll need to keep it working as people break it. We focus on the project as “I’m updating from version X to version Y, where X < Y” in all our make infrastructure. While we could add bits where X > Y, and for the release, there are likely several items that will need to be fixed to get there. You are currently hitting this turbulence with cross-version races in multiple job builds. By all means fix them, but since this is an unusual use case (from a historical perspective), expect there to be bumps, and expect there to need to be fixes to make it work (and also from a historical perspective, expect people will break it innocently).

>>> I hate to even suggest this, but the ports tree (ab)uses the notion of
>>> using the kern.osreldate for certain things.  This, however, requires
>>> proper bumping of __FreeBSD_version when needed, and maintenance of the
>>> Makefiles for the kern.osreldate-specific things.
>> 
>> We already do that. It mostly works most of the time, so long as the delta isn’t too great, and we don’t have high compiler/tools/make velocity… Except we don’t use the kernel version, but rather the installed tools version as indicated by a .h file. That’s more robust.
>> 
> 
> True.  Thank you for the sanity check.
> 
>>> The benefit to this is that it would help prevent pissing off ports
>>> developers and make their lives a bit easier when userland / kernel
>>> things change.  It would, however, (expectedly) is that it would force
>>> src committers to do the right thing.  Win-win, IMHO.
>> 
>> What should we do we aren’t doing today?
>> 
> 
> There have been a number of times where changes that should deem
> a __FreeBSD_version bump necessary either 1) do not bump
> __FreeBSD_version at all, or 2) bump __FreeBSD_version several days (or
> longer) later.  So, we're left with a window of time where something is
> "different enough", but there is no corresponding version change to
> reference.
> 
> This is somewhat tangential to my original annoyance here though. :)

With -current, a few days is more than enough granularity. There are bumps, and this is one of them.

>>> Personally, and no I won't discuss more on this, I'm in the camp of "I
>>> don't really see clang as a feature."  It caused our ports developers
>>> and maintainers a mountain of headache to convert to the "invisibly new
>>> great thing", it increases our overall buildworld by a non-insignificant
>>> amount of time, and it has personally caused me headaches (still
>>> ongoing) trying to figure out what the correct incantation of evil to
>>> wish over the cauldron to get BeagleBone images to build.  (They're
>>> failing because gcc is not being installed on both head/ and stable/10/,
>>> and despite the game of "musical KNOBS" I've been playing over the past
>>> few days, I'm running out of hair to pull out of my head.)
>> 
>> Yea, if you are using crochet, that’s because crochet uses xdev rather than a ports compiler (which in all fairness didn’t exist when it started) to build u-boot, which basically requires gcc.
>> 
>> The compiler rework in head is still a work in progress. What’s there now is better than before, but still isn’t quite right. I do plan on fixing that before summer is out.
>> 
> 
> It isn't just head that is a problem with crochet, though.  stable/10
> has been a problem since, as far as I can tell, roughly early May.

Building release 9 on 10 falls under the X > Y category above. If if breaks, and you want it to work, you gotta fix it. If there’s several somebodies that want it working, they gotta keep it working and fix what breaks. Since this scenario is quite a bit less supported and has received no love traditionally, it will likely take a while before you can count on it once you’ve mopped up the problems and convinced prime movers to spend the extra time it takes to make this work. Supporting all the normal use cases is hard enough, but adding this to the mix will add lots of extra test time, and given how hard these problems have been to nail, lots of extra debugging and fixing time to the cycle that might be hard to enforce in a volunteer organization.

>>>> But 9.2 will never build on head because it is broken with bmake, which is now standard for head. Since 9.2 cannot be changed, and since we’ve removed (or nearly) fmake in current, chances are quite good it will never build on head again without some special handling.
>>>> 
>>>> In summary, good luck! there’s a lot of use cases here, and it will take time and effort of multiple people over the long haul to keep it working. Best effort may be larger than you estimate… I won’t stand in your way, but I’m afraid my time available to help is limited.
>>>> 
>>> 
>>> As Ozzy once sang:
>>> 
>>>   "I'm just a dreamer
>>>   I dream my life away
>>>   I'm just a dreamer
>>>   Who dreams of better days”
>> 
>> Since I was commenting on the opposite problem of what you were wanting comments on, my harshness is justified.
>> 
> 
> My comment wasn't a comment on your comment. :-)
> 
>> What you want though, we largely already do, though maybe with a few more warts than necessary (which we should try to fix). Most of the warts are due to gcc/clang division being done badly and unsustainable initially and the cleanup taking a bit of time, not specific version issues.
>> 
>> Back to your basic point, the issue becomes a testability one: not all committers can reasonable be expected to have 8 or 9 systems to test every change. Having a 10.x system to test changes is a bit of a stretch as it is, but it is the official policy that many folks play fast and loose with the rules because they haven’t been burned too often by it… VMs, Jails, etc of various flavors can help, but some info does leak through (mostly the info leaks are bugs or kludges that well meaning people shouldn’t have done given the historical knowledge we have about the ill effects of certain ways to do conditional compilation).
>> 
> 
> To be fair, we do have reference machines in the cluster running head/,
> stable/10, and stable/9.

True, but I already have jails with these environments locally. It doesn’t magically give me the time to run the tests, nor find the time to fix subtle problems that crop up. Plus, my work flow needs things like hg and hgsubversion installed on top of the normal tools, which historically has been at best spotty coverage on the project machines.  And despite using FreeBSD for two decades and being a somewhat active developer, I’ve never once typed ‘make release’.

I don’t mean to be difficult or unsympathetic, I’m just trying to paint a realistic picture of the challenges in expanding the current support in builds to include the new things you want to include.

Warner