Re: The futur of the roff toolchain

I like all of this.  Thanks for your very thorough research and effort.

Eric


On 05/21/2017 07:57, Baptiste Daroussin wrote:
> Hi all,
>
> I have been working for a while to try to import a modern roff toolchain into
> base.
>
> I didn't like the initial approach that consisted in simply removing all roff
> toolchain in base.
>
> Recap of the situation in base:
> * We have GNU roff version 1.19.2 in base (latest GPLv2 version). Lots of bug
>   fixes has been made upstream in newer version (GPLv3) in particular regarding
>   unicode but not only. (and we cannot update it anymore)
> * GNU roff is now only used to generate the documentation in share/doc and as a
>   fallback for manpages which mandoc does not support.
>
> On the manpages front:
> * No manpages in base are not supported by mandoc except groff manpages
>   themselves
> * man(1) can fallback on ports version of groff if installed (for ports not
>   providing manpages not compatible with mandoc)
>
> Alternatives to GNU roff:
> * Heirloom doctools (which I tried to import) licensed both CDDL/BSD (in C)
> * neartoff http://litcave.rudi.ir/ BSD licensed (in C)
>
> I went the road of using heirloom doctools it is 90% compatible with GNU roff,
> good enough for all our base roff based documents.
>
> After getting down that road for a while, including lots of patches sent
> upstream (thanks them for being so reactive and integrating them quickly as well
> as fixing the issues I wasn't able to fix myself quickly).
>
> The problem is there are lot of corner small corner cases where heirloom is
> different from GNU roff and hard to make it compatible. While this is corner
> cases, it breaks document generation for some large documents people are
> writing. Those users could use (and actually would benefit a lot from it) GNU
> roff from the ports tree, but have to be careful about the path of the tool they
> call to ensure only calling the one from GNU roff and not the one (with the same
> name) from heirloom doctools.
>
> Concerning neatroff it is barely compatible with GNU roff, so not an option
> (last I tested at least).
>
> I would like to change this approach and get back to the initial approach taken
> by others before I jumped in and I would like just entirely remove the roff
> toolchain from base and let people rely on GNU roff from ports.
>
> man(1) is already asking the user to install groff from ports if the manpage
> cannot be read with mandoc.
>
> No the problem left is documentations available in share/doc.
>
> I would like to push them elsewhere. Those documents are mostly useful for
> historical reason (hence we want to keep them) but not really for daily use of
> modern FreeBSD.
> Another issue with those documentation, they are installed as text/ascii version
> in base, which makes most of them not really readable (as the documents has not
> be written for a ascii/text target but more for a PDF/html view - using pic(1)
> for example)
>
> A plan was to push as sources in the svn doc repository and continue to build
> them. This approach also have an issue: over the time roff evolved a bit and
> while working on heirloom doctools import I had to fix a bunch of markup to make
> the rendering of those documents clean (also meaning almost noone should read
> them considering some were not really readable).
>
> What I want to propose now, it to render them as PDF (html?) once and push them
> somewhere (to be defined) as static document on our documentation website.
> Please doceng_at_ provide me a location where to push them.
>
> And then remove bsd.doc.mk from FreeBSD 12.0 along with the removal of groff.
> I also want to remove most of roff related tools (the one provided by toolchains
> available in ports) for which we kept a BSD version (not really maintained in
> base):
> namely:
> - checknr
> - vgrind
> - colcrt
>
> Only keeping:
> - col (useful in other places than roff)
> - soelim (also used for manpages and we have a clean BSD licensed version which
>   is also now parts of mandoc)
>
> Best regards,
> Bapt