Re: libXO-ification - Why - and is it a symptom of deeper issues?

From: Allan Jude <allanjude_at_freebsd.org>
Date: Mon, 16 Nov 2015 12:08:45 -0500
On 2015-11-15 14:44, Dan Partelly wrote:
>> I would welcome competing ideas/solutions, but someone would have to actually build them, not just
> 
>> rattle off some ideas on the mailing list.
> 
> Am I missing the point of a mailing list ?  it is a place to present and exchange ideas, ask why some things are the way they are , and get criticism. Rattling ideas is generally where it all starts. You cant realistically expect one to start a pervasive project like transactioanl databases for an OS configuration without not one mailing list discussion, but a lot of consultation. 
> 

Sorry, I didn't mean to say we shouldn't talk about this. The reason for
the less-than-warm response you got from multiple people, is that this
is the third or fourth such discussion on the mailing list (this is what
mailing list archives are for). Once every so many months, someone rants
about how libxo is horrible and is ruining the base system, and wants to
lib-ize everything. And no one every does it. Meanwhile, libxo actually
gets worked on.

> 
>> There are tools available to filter json/ucl output
> yes they are. vim is one . sed is another. awk is the third …  But a pipe needs both ends to talk the same language. I can generate json as output, i can filter it  , but what tool in FreeBSD will accept it as input ? None.  A
> 
>> and my forthcoming uclcmd
> 
> What uclcmd does / will do ?  

It can read, filter, and write UCL (as well as JSON, YAML, and msgpack.
Hopefully it will be able to do NVLists in a few months).

It is designed to be able to view, extract information from, and edit
config files.

AsiaBSDCon Paper:
http://allanjude.com/talks/AsiaBSDCon2015_paper_-_UCL_for_FreeBSD.pdf

BSDCan Dev Summit working group:
	Wiki Page: https://wiki.freebsd.org/201506DevSummit/UCL
	Slides: http://www.allanjude.com/bsd/BSDCan2015_-_UCL_Working_Group.pdf
	Additional Slides about libucl:
http://www.allanjude.com/bsd/cebka_BSDCan2015-UCL.pdf
	Video: <apparently not online yet>

BSDCan Presentation:
	Slides: http://allanjude.com/talks/BSDCan2015_-_UCL_for_FreeBSD.pdf
	Video: https://www.youtube.com/watch?v=8l6bhKIDecg

> 
>> UCL is a good solution to having a universal config file, and is better
>> than the bespoke config files each utility has
> 
> 
> I agree with you, if you manage somehow to port every ad-hoc database FreeBSD has in base to ucl, (including all the bestiary we have in /etc ) and offer tools to parse them in the rc init system to fetch the settings. I dont expect this to be done in one release , but do you tend towards this in your projects ? One config format to rule them all is good. Yet another config format in selected places in base is evil.
> 

The goal is to change everything, and I am making progress. The working
group at BSDCan had quite a wish list of things to convert, including a
bunch that I figured people would prefer not to touch, like login.conf
(I need to polish that patch and get it up for review)

>>  A transactional database
>> might be better (for some uses, likely less so for some people), but I
>> don't hear anyone volunteering to do that work.
> 
> IIRC Solaris uses sqlite. Might be a decent solution. ( a lot of the ad hoc unix databases are relational databases in disguise, with unix tools acting as operators)  And if you abstract a config interface API over UCL, someone could benefit  from it by plugging in a transactional backend one day. All you would have to do is not directly plug in UCL, but work on a orthogonal abstract API. You could even  model it after UCL API.
> Please consider it. 
> 

For bhyve, the basic design they have in mind is UCL config files that
are parsed and presented to the application as an nvlist, which it then
passed between modules of bhyve (like networking, out-of-band access,
graphics, disk (vmdk, qcow, etc)), without the base bhyve application
having to know what config options modules actually support. It seems
like a reasonable design to me.

I don't know that further abuse of sqlite is actually a good idea, but I
am still a bit new at this.

> 
>> I would welcome competing ideas/solutions, but someone would have to actually build them, not just
> 
>>
>> lib-izing all of the utilities instead of using libxo is a better
>> solution. It would likely be gratefully accepted if someone were to do
>> it, but most likely, no one will.
>>
>> libxo exists now, and most applications can be converted very quickly
>> (although care does need to be taken, and it sometimes has not been).
>>
>> There are tools available to filter json/ucl output, namely textproc/jq
>> and my forthcoming uclcmd
>>
>> One of the major other consumers of the json/xml output of libxo, is web
>> control panels. This is why Juniper is doing the work, and I can think
>> of a list of other appliance vendors who would love to replace fragile
>> per-application parsers with a json parser to extract data from various
>> command line tools.
>>
>> UCL is a good solution to having a universal config file, and is better
>> than the bespoke config files each utility has. A transactional database
>> might be better (for some uses, likely less so for some people), but I
>> don't hear anyone volunteering to do that work.
>>
>> So, libxo and libucl may not be the very best solutions, but they are
>> the ones that are moving forward. I would welcome competing
>> ideas/solutions, but someone would have to actually build them, not just
>> rattle off some ideas on the mailing list.
>>
>> -- 
>> Allan Jude
>>
> 


-- 
Allan Jude


Received on Mon Nov 16 2015 - 16:08:55 UTC

This archive was generated by hypermail 2.4.0 : Wed May 19 2021 - 11:41:01 UTC