Re: Massive libxo-zation that breaks everything

On 3 Mar 2015, at 01:32, Andrey Chernov <ache_at_FreeBSD.org> wrote:
> 
> So, why you ever need to modify wc? Just load wc inside your
> json/xml/etc writer, replacing its printf at the ld-elf.so level.

You can't get structured output from printf() because printf() takes unstructured input.  It's a string with some variables pasted in, but no awareness of context.

The libxo changes to the tools are simply marking the output as having a defined structure.  The library then translates this abstract structure into something that can be parsed easily by external tools.

If your argument is that the UNIX philosophy is simple tools doing one thing well, then please remember the context of this philosophy: It dates back to the original UNIX systems *that did not support shared libraries* and was an argument used to justify not implementing them.  This is why globbing is in the shell instead of a shared library and why some variant of mv *.a *.b works on every command-line interface except for a UNIX shell.

Even with that in mind, small changes to individual tools are a *lot* simpler than one massive monolithic tool that understands the output formats of every other tool in the base system and can transform them.  Why do you think a few library calls in each application is more complex *than an entire parser per tool*?

I am reminded of the time about ten years ago when I was writing a system info widget for a DE.  Getting the total amount of RAM, number of cores, and CPU speed on FreeBSD, OpenBSD, NetBSD, and Solaris proved to be a single sysctl for each attribute.  On Linux, the information was all there, in /proc/cpuinfo, so I had to write a parser.  Unfortunately, the format changed across kernel revisions and across architectures.  In the end, I had to link in a 300KB library to parse information that the kernel already had in a machine-readable format and present it to me in a machine-readable format.

Hopefully there's a lesson here that we can learn from: human-readable formats do not make good intermediate representations when communicating between tools.

If your argument is about maintainability of these changes, then please point to concrete instances where the changes are complex and difficult to maintain.

If your argument is about binary size, then it would be relatively easy for us to add a version of libxo for static linking into the versions in /rescue that only supported plain-text output, but again, please quantify your objections: What is the size increase that you're seeing and what kind of devices is this causing a problem for?


The general approach of making tools produce machine-readable output has been under discussion for *years* and the integration of libxo has been underway for several months, prior to which it was used inside Juniper.  If you had a better approach, then the correct time to raise it was back when the requirement to produce machine-readable output was raised.  The subject of this thread is just plain trolling (is everything broken by libxo?  No.  Is anything broken by libxo?  Occasionally, but new features often come with temporary breakage that's found and fixed in testing: that's why we have -CURRENT).

David