The futur of the roff toolchain

Hi all,

I have been working for a while to try to import a modern roff toolchain into
base.

I didn't like the initial approach that consisted in simply removing all roff
toolchain in base.

Recap of the situation in base:
* We have GNU roff version 1.19.2 in base (latest GPLv2 version). Lots of bug
  fixes has been made upstream in newer version (GPLv3) in particular regarding
  unicode but not only. (and we cannot update it anymore)
* GNU roff is now only used to generate the documentation in share/doc and as a
  fallback for manpages which mandoc does not support.

On the manpages front:
* No manpages in base are not supported by mandoc except groff manpages
  themselves
* man(1) can fallback on ports version of groff if installed (for ports not
  providing manpages not compatible with mandoc)

Alternatives to GNU roff:
* Heirloom doctools (which I tried to import) licensed both CDDL/BSD (in C)
* neartoff http://litcave.rudi.ir/ BSD licensed (in C)

I went the road of using heirloom doctools it is 90% compatible with GNU roff,
good enough for all our base roff based documents.

After getting down that road for a while, including lots of patches sent
upstream (thanks them for being so reactive and integrating them quickly as well
as fixing the issues I wasn't able to fix myself quickly).

The problem is there are lot of corner small corner cases where heirloom is
different from GNU roff and hard to make it compatible. While this is corner
cases, it breaks document generation for some large documents people are
writing. Those users could use (and actually would benefit a lot from it) GNU
roff from the ports tree, but have to be careful about the path of the tool they
call to ensure only calling the one from GNU roff and not the one (with the same
name) from heirloom doctools.

Concerning neatroff it is barely compatible with GNU roff, so not an option
(last I tested at least).

I would like to change this approach and get back to the initial approach taken
by others before I jumped in and I would like just entirely remove the roff
toolchain from base and let people rely on GNU roff from ports.

man(1) is already asking the user to install groff from ports if the manpage
cannot be read with mandoc.

No the problem left is documentations available in share/doc.

I would like to push them elsewhere. Those documents are mostly useful for
historical reason (hence we want to keep them) but not really for daily use of
modern FreeBSD.
Another issue with those documentation, they are installed as text/ascii version
in base, which makes most of them not really readable (as the documents has not
be written for a ascii/text target but more for a PDF/html view - using pic(1)
for example)

A plan was to push as sources in the svn doc repository and continue to build
them. This approach also have an issue: over the time roff evolved a bit and
while working on heirloom doctools import I had to fix a bunch of markup to make
the rendering of those documents clean (also meaning almost noone should read
them considering some were not really readable).

What I want to propose now, it to render them as PDF (html?) once and push them
somewhere (to be defined) as static document on our documentation website.
Please doceng_at_ provide me a location where to push them.

And then remove bsd.doc.mk from FreeBSD 12.0 along with the removal of groff.
I also want to remove most of roff related tools (the one provided by toolchains
available in ports) for which we kept a BSD version (not really maintained in
base):
namely:
- checknr
- vgrind
- colcrt

Only keeping:
- col (useful in other places than roff)
- soelim (also used for manpages and we have a clean BSD licensed version which
  is also now parts of mandoc)

Best regards,
Bapt