Re: warning of pending commit attempt.

Bruce M Simpson wrote:
> Just to give everone some background:
> I shared an office in Berkeley with Marko for nearly a month 3 years 
> ago, working on the same project. I bear him no ill will whatsoever, I'm 
> impressed by the work he's done, especially so given he's started a family.
> 
> But I do need to wear a black hat in this argument, because there's more 
> to FreeBSD than the server space, and the future beckons.
> 
> We need to balance the risk of the new feature against its effect on 
> current and future development. I still see loose ends which need to be 
> tied up before I'm 90% happy. I will never be 100% happy with anything 
> in life ever.

I personally want it for my laptop, and embedded devices.
Of course there are loose ends but because it can be compiled out,
they do not need to affect other users when we commit it. They
will only affect those who elect to be affected.


> 
> Julian Elischer wrote:
>> It includes enough of a framework to allow it to cope with loadable 
>> kernel modules and to integrate with jails. Two important requirements
>> for its job. This means that this framework is avialable for other 
>> virtualisation work to use as well.
>>
>> no it doesn't do machine virtualisation.
>>
> 
> OK, so the agenda for vimage hasn't fundamentally changed. Thanks for 
> clarifying this. I got the impression from your message that the agenda 
> had gotten bigger.

No, but we would like to get it integrated with other virtual machine
work that is underway. In the same way it integrates in with jails at 
the moment.

> 
> 
>>>
>>> * Also, do the changes which you intend to introduce, allow for code 
>>> which is currently under development, and which is not aware of 
>>> vimage/IMUNES in any way, to continue to compile and be usable 
>>> without additional developer time?
>>
>> Code not aware of vimage just appears in all virtual machines, just as 
>> it would appear in all jails. I uses and expands on the jail 
>> infrastructure.
>>
> 
> OK. The presence of jail generally doesn't interfere with things I've 
> been looking at.
> 
> I don't really have a problem with this as long as it defaults to "off" 
> for at least a first release.
> 
> I speculate the majority of the user base don't actually need vimage. I 
> speculate it isn't an interesting feature in the near future for 
> embedded platforms. They may not know they want it (or I for that 
> matter) yet. It's a bit like trying to get a child to eat more green 
> vegetables.

I am pushing this for embedded platform use.. That's what I do.
consider.. I have module A and module B that do function X on
a network segment. Now I can make a machine with 4 interfaces and
have function X performed on 4 network segments without fear
of them interacting and producing problems. WITHOUT CHANGING
MY CODE.  I can make a compbined device that has module A and module B
both on the same segment, with no fear that they'll screw each other
up WITHOUT CHANGING MY CODE.
(just some little examples)


> 
> Of course if vimage touches network interfaces, and can present virtual 
> NICs to the stack, then things get more complicated.

it can if you want it to.. but you mis-spoke. You said "the stack".
There is no one special stack. There are many and each virtual 
interface could be feeding to a different stack.

> 
> This of course is the KitchenSinkBSD issue, how can something be all 
> things to all people and yet stay focused and be tight?
> 
>> And there we differ. I think the time for this has come. I'd say about 
>> 6 years is long enough. That is how long people have had to play with 
>> this and find problems. Anybody who has had anything to do with it
>> has only sung its praises.
>>
> I know --  6 years --  blimey, tell me about it.
> 
> But the point I am making is that a lot of that drag could have been 
> avoided.

how? "sticks in the mud can be very stubborn".


> 
> It's not the first time that I've introduced code to the tree, having 
> given people plenty of warning (3-6 months), and then experience my 
> stress levels surge off the scale as my inbox, and lists, are bombarded 
> with messages from people going "WTF?"
> 
> And this set of changes wasn't even on the scale of vimage.

yes but you can compile it out and it will be so by default.
I've done quite a few "large" introductions over the years.
Sometimes not because I really wanted to, but because "some-one
has to do this". (e.g. threading)
This is probably the least dangerous of them all.

> 
> It is reminiscent of the opening chapters of Douglas Adams' 
> "Hitchhiker's Guide to the Galaxy", where notice of the change, to their 
> mind, was posted in a basement filing cabinet behind a locked door with 
> a sign saying "Beware of the leopard".

Oh do come on, this has been presented at several conferences and 
Marco and I have brought it up whenever possible.

> 
> I see problems because people come to expect a higher level of polish 
> from FreeBSD as they do from its alternatives, which is why I mention 
> regression testing. In many ways there is no real substitute for a 
> commercially oriented development model, however as we know that has its 
> cons as well as its pros.

well, turn it off if you want to avoid the risk.. but it will NEVER
get the polish you want of it if we do not put it in after 7.0 is out.

> 
> Granted, there's infinite demand for free goods and services. 
> Unfortunately this is more economically involved than simply saying to 
> the world, "Look everyone, I made some cookies. Want some?" -- unless 
> it's done on a small scale and the effort is tightly focused. I imagine 
> most of us know that real life development of anything is an iterative 
> process.
> 
> Of course finding people who are willing to be committed to test and 
> roll-out is made infinitely easier, if there are other things in it for 
> them, and quite often, that boils down to money...
> 
> Which leads to a head on collision with the ideological battleground 
> that open source sometimes ends up being.
> 
>>>
>>> Whilst CVS is a proven tool, its use in the project, to my mind, 
>>> seems more geared to delivering production code and tracking changes 
>>> in production branches, rather than managing fast-moving new 
>>> development.
>>
>> Things have to get into it at some stage..
> 
> Again, I'm trying to underline a point about CVS, p4 and their use 
> within the project.

P4 is not a suitable place for code to get polish. That's all there is 
to it. We do not make releases from P4.

> 
> Perforce doesn't really cut it for opening up code to the public at 
> large, as we have seen, and has real limitations, both in these terms 
> and in terms of its resource use.

exactly. This has gone about as far as it can in perforce.
it's time for the next step.

> 
> 
>>
>> no. it has tests for its own functionality.
>> No other piece of code in the system EVER has had regeression
>> tests for whole system. Loading this burden on vimage is not a
>> fair requirement. The aim of vimage is for you to not be able to tell, 
>> so any deviation form normal behaviour in any part of the
>> system would be a regression.
> 
> Of course it's not fair -- I'm playing daemon's advocate here. ;^)
> 
> I would be happier to see vimage go in when I see stronger answers to 
> the questions which Andre and Kris raise re synchronization overhead.

there is less synchronisation overhead when you have multiple vimages
than when you have the same number of multiple jails. (because
different interfaces lead to different IP stacks with different locks, 
so the load is spread over several locks so the collision probability
is reduced).
When you have one vimage it has the same overhead as the curtent code, 
and when you
compile it out you effectively HAVE the current code.

Does that make it easier for you?

> 
> Again, I'm making the point that more formal software development 
> practice wouldn't go amiss, particularly when introducing something with 
> wide impact such as vimage.
> 
> I'm on an interesting chapter of Douglas Hofstader's "G*del, Escher, 
> Bach" which discusses just such interdependence just now.
> 
> I'm happy that things aren't specced down to the microsecond in FreeBSD, 
> otherwise we'd never get anything done. Folks just have to remember to 
> perform appropriate system tuning on their own expectations. ;-)
> 
> cheers
> BMS