Re: [RFC] Replace gnu groff in base by heirloom doctools

From: Baptiste Daroussin <bapt_at_FreeBSD.org>
Date: Tue, 19 May 2015 19:21:34 +0200
On Tue, May 19, 2015 at 06:52:40PM +0200, Steffen Nurpmeso wrote:
> Hello,
> 
> Baptiste Daroussin <bapt_at_freebsd.org> wrote:
>  |On Tue, May 19, 2015 at 02:37:22PM +0200, Steffen Nurpmeso wrote:
>  |> Baptiste Daroussin <bapt_at_freebsd.org> wrote:
>  |>|On Sat, May 16, 2015 at 01:42:26AM +0200, Julian H. Stacey wrote:
>  |> 
>  |>|>> I think keeping a fully functionnal roff(7) toolchain part of the
>  |>|>> base system is very good on a unix.
>  |> 
>  |>|>> From what I could check I cannot find any regression when \
>  |>|>> migrating from gnu
>  |>|>> groff to heirloom doctools, if there is a particular area \
>  |>|>> when you think extra
>  |>|>> care is needed please share it.
>  |> 
>  |> It seems you haven't checked at all.
>  |> It seems to me that e.g. mdoc(7) of n-t-r seems to require quite
>  |> a bit of work in order to be at all usable.
>  |
>  |Lots of work has been done recently on heirloom in particular regarding
>  |the support of mdoc(7) and I have opened tickets for all issues \
>  |I could find and
>  |they have been fixed. Please point me to issues you can have \
>  |regarding mdoc(7).
> 
> .Ns doesn't work right, so is why this strong emphasis of mine in
> the first sentence.  Carsten has this example of mine (last week):
> 
>   .Dd Apr 30, 2015
>   .Dt xxx 1
>   .Os
>   .Sh NAME
>   .Nm xxx
>   .Nd yyy
>   .
>   Read the system mailbox of
>   .Ar user
>   (appropriate privileges presumed), and
>   .Dq assume to be
>   .Ar user
>   in some aspects, e.g. in respect to
>   .Ic file Ns
>   \(enexpansions of
>   .Ql %
>   etc.; also see
>   .Ev USER .
> 
Thanks  I will have a look at it soon

>  |(Note that I'm speaking of doctools as of latest git, not latest release)
> 
> As of last week right shortly after your mail i guess it was.
> 
>  |>|Heirloom in base is a win over groff because it has better \
>  |>|support for roff(7)
>  |>|better font handling etc.
>  |> 
>  |> The macros i use for myself don't work with n-t-r, too: once
>  |> i truly looked (a few months ago) i found that i would have to
>  |> rewrite all traps and other positioning in order to get that
>  |> right.
>  |
>  |Can you tell me more about the macros you do use and a sample \
>  |document so I can
>  |check and see if we can add support for it?
> 
> Well Carsten asked me this too last year and i've given him as
> much as i could.  But the macros are really rather simple layout
> via traps.  If i recall correctly the examples i've shown Carsten
> where letters.  I would have to rewrite the macros before i can
> make them public, but it is pretty "normal" troff stuff with
> traps and positioning, like e.g. the context-free following, for
> an example (i think i have sent Carsten some trap info back then?)
> 
>   .MACRO RECEIVER
>   .  S:BOOLIFY \\$1
>   .  ie \\n[S:#IS_BOOL] \{\
>   .     vs \\n(.su*\\*[RECEIVER_LINE_SPACING_SCALE_FACTOR]
>   .     nf
>   .     de S:RECEIVER_TRAP EOT
>   .        blm PARA
>   .        ds S:RECEIVER_DIVERSION_HOOK
>   .        sp 1v
>   \\*[RECEIVER_PREHOOK]\c
>   .EOT
>   .     di S:RECEIVER_DIVERSION
>   .     blm S:RECEIVER_TRAP
>   .  \}
>   .  el \{\
>   .     if d S:RECEIVER_DIVERSION_HOOK \{\
>   \\*[RECEIVER_POSTHOOK]\c
>   .        rm S:RECEIVER_DIVERSION_HOOK
>   .     \}
>   .     di
>   .     \" Calculate best position for address field and box out
>   .     if (\\n(dnu > \\n[#RECEIVER_HEIGHT]u) \{\
>   .        WARNING \
>   \\$0 address does not fit in address window! Growing window!!
>   .        nr #RECEIVER_HEIGHT \\n(dnu
>   .     \}
>   .     nr #1 (\\n[#RECEIVER_HEIGHT]u-\\n(dnu)/2u
>   .     sp |(\\n[#RECEIVER_START]u+\\n(#1u
>   .     rr #1
>   .     S:RECEIVER_DIVERSION
>   .     rm S:RECEIVER_DIVERSION
>   .     rm S:RECEIVER_TRAP
>   .     fi
>   .     vs
>   .  \}
>   ..
> 
> Pretty clean letter stuff like that it is.  (You asked for an
> example.)

Same I'll have a look :)

> 
>  |> Despite that you seem to do what you want to do anyway, n-t-r is
>  |> possibly a usable troff, if you go its way and deal with it you
>  |> may be able to gain a bit nicer output _faster_ and without
>  |> converting your beloved special fonts first, but in no way is
>  |> n-t-r a _replacement_ for groff.
>  |
>  |As I said you will be able to use groff from ports. I do not \
> 
> I will have my own one, then.  Enough work for getting old.  d^.^b
> 
>  |claim that n-t-r is
>  |a replacement for groff in general I propose it for a replacement \
>  |for groff in
>  |base.
>  |
>  |groff in base is stuck at 1.19.2 version while upstream is at 1.22.3 version
>  |which in particular has a couple of fixes for mdoc(7) format and a bit more.
> 
> The mdoc(7) macros are BSD licensed.  Nothing prevents anyone from
> taking those from GNU troff [master] branch and integrating them
> into their own roff version.  I did that for (the one that will
> be) mine.
> 
>  |Every user of groff will have huge benefit using newer groff versions: bug
>  |fixes, full functionnalities available etc.
> 
> The above macros originate from stuff written under FreeBSD 4.9
> and especially 5.3 and ran almost a decade there, very smoothly.
> Also i'm not so sure about that "huge" when i compare groff
> 1.19.2-574-gecbf4f1 (which is the last GPL2 commit) and 1.22.3.
> Also Carsten said that.  Until now i have indeed assumed it is
> purely rhetoric.
> 
> Out of interest: what do you mean when you say "full
> functionality"?  I see some improvements in the line breaking
> algorithm and support for PDF images, as well as a perl(1)-based
> PDF output device.  ?  Of course i want to have that, but for my
> personal German and English use the first isn't "deadly"
> necessary, PDF images i've never used (this is EPS then) and the
> last, well, time has to bring something.  Until then PS to PDF via
> Ghostscript has to do the trick.  It always did so for me.

By full functionnality yes I meant all this + X11 support + things like
groffpdf, all the roff2* commands etc.
> 
> (In fact i would possibly have been better off to start my own
> troff with a codebase from before the grohtml rewrite polluted
> even deepest parts of the troff engine itself instead of that last
> GPL2 commit.  And things like preconv(1) etc. i will be able to
> manage in a different way, too.  And MathML support for eqn(1) is
> possibly overkill if the normal HTML output is completely broken.)
> 
> Not to be misunderstood.  Yes, i dislike the decision to include
> n-t-r doctools in FreeBSD.  I took maintainership of a MUA that
> was written by the same person and the code was completely broken.
> (And everybody knew that but myself.)  Worse, it was terribly
> designed, and still is like this for the most part.  Even though
> the doctools came up years later, i guess the changes were made on
> least effort basis also here.  And how is that compatible with
> FreeBSD?

I do understand that you have been beaten by your experience. The goal here is
to have a working and maintainable roff toolchain, so far heirloom's seems to
do it. Of course it is not perfect, but from all documentation we do have in
base, it does a better job (speaking of fully roff documentation not manpages).

For all manpages that mandoc(1) does not handle, it makes a decent fallback
renderer and often gives better rendering that groff(1) if Interested I can give
you a couple of example where the rendering is better than the one from groff.

> (Yet surely parts of FreeBSD always smelled like rotten.  I don't
> know how often i got a hard locked burncd process, for example.)
> And of course i like that FreeBSD wants to keep a troff, NetBSD
> unfortunately seems to go a different route.  But what benefit can
> be something that doesn't work with existing stuff, but that needs
> to be addressed in a special way in order to get not the desired
> but the expected results?  That doesn't make sense to me.  But it
> definetely is not a win, but something different.  Maybe it is
> a win for those who search the latter, though.
> 
Of course feedback like yours are interesting and taken in account. I will make
sure to harass (in a friendly way :)) upstream to fix all bugs (if I cannot fix
them myself) reported by users.

I went the route of proposing this change because doctools is working properly
with all the existing stuff we do have in base! (minus a couple of pending
fixes) this mail was sent to actually get feedback (like yours) from users about
issue they might have with it so we can address them in time and finish
evaluating this tool.

I will create a branch soon to make the integration in base happening and
testable by more people, then send a call for testing so people can actually
test what we would provide to them.

Best regards,
Bapt

Received on Tue May 19 2015 - 15:21:39 UTC

This archive was generated by hypermail 2.4.0 : Wed May 19 2021 - 11:40:57 UTC