Re: RFC w.r.t. toggling debugging on/off for mountd via a signal

On Sun, May 19, 2019 at 02:47:10AM +0000, Rick Macklem wrote:
> Alan Somers wrote:
> >On Sat, May 18, 2019 at 7:59 PM Rick Macklem <rmacklem_at_uoguelph.ca> wrote:
> >>
> >> Hi,
> >>
> >> I've been working with Peter Errikson on a patch for mountd that adds a new option
> >> for incremental updating of exports. This seems to be helping a lot w.r.t. performance
> >> on an NFS server with lots (10000+) of exported file systems.
> >>
> >> I have debug syslog() calls in the code, which I/Peter think would be worth keeping
> >> in the production code in case someone runs into problems with this new option.
> >>
> >> As such, I'd like to have the code compiled in by default (not only if DEBUG is defined,
> >> as mountd.c has now). I also was thinking it would be nice if the daemon didn't need
> >> to be restarted to enable/disable the debugging output, since that breaks NFS
> >> mounting during the restart.
> >>
> >> So, I was thinking of having the debugging output toggled on/off via SIGUSR1.
> >>
> >> What do you think of this idea?
> >> Any other/better ways to do this?
> >> Also, would LOG_DAEMON and LOG_DEBUG sound like the correct facility and
> >> priority for theses syslog() calls?
> >>
> >> Thanks in advance for any comments, rick
> >
> >If the debug messages aren't so verbose that they'll slow down
> >syslogd, then you can just leave them enabled all the time.  syslogd
> >will filter them.  However, if they're super-verbose then SIGUSR1
> >sounds reasonable.  I can't think of another daemon with runtime
> >selectable logging verbosity like that.
> Yes, these are pretty chatty. 5-10 lines for each entry in an exports file.
> Multiply that times the number of entries. (Peter's servers are between 20000
> to 72000+ file systems. Not sure if he has multiple entries/file system.)
> 
> To give you a clue, without this patch, it can take 20sec->over 1min to reload them
> when mountd gets a SIGHUP.
> 
> It's just that the export file handling code is pretty convoluted, so I think the patch
> is ok, but I won't be too surprised if someone finds a problem.

Instead of toggling the debugging by SIGUSR1, you can make the daemon
to try to open some file by receiving the normal reload signal, e.g.
/var/run/mountd.debug. If the file is present, the debugging is enabled.
The advantage is that the user knows what is the debugging state, instead
of requiring to count the number of sent SIGUSR1's.

More, in traditions of malloc(3), /var/run/mountd.debug could be a symlink,
which is read and interpreted as some options controlling the debug.