Re: Race between cron and crontab

On Thursday, February 02, 2012 8:03:12 pm Doug Barton wrote:
> On 02/01/2012 04:42, John Baldwin wrote:
> > On Tuesday, January 31, 2012 9:23:12 pm Doug Barton wrote:
> >> On 01/31/2012 08:49, John Baldwin wrote:
> >>> A co-worker ran into a race between updating a cron tab via crontab(8) and 
> >>> cron(8) yesterday.  Specifically, cron(8) failed to notice that a crontab was 
> >>> updated.  The problem is that 1) by default our filesystems only use second 
> >>> granularity for timestamps and 2) cron only caches the seconds portion of a 
> >>> file's timestamp when checking for changes anyway.  This means that cron can 
> >>> miss updates to a spool directory if multiple updates to the directory are 
> >>> performed within a single second and cron wakes up to scan the spool directory 
> >>> within the same second and scans it before all of the updates are complete.
> >>>
> >>> Specifically, when replacing a crontab, crontab(8) first creates a temporary 
> >>> file in /var/cron/tabs and then uses a rename to install it followed by 
> >>> touching the spool directory to update its modification time.  However, the 
> >>> creation of the temporary file already changes the modification time of the 
> >>> directory, and cron may "miss" the rename if it scans the directory in between 
> >>> the creation of the temporary file and the rename.
> >>>
> >>> The "fix" I am planning to use locally is to simply force crontab(8) to sleep 
> >>> for a second before it touches the spool directory, thus ensuring that it the 
> >>> touch of the spool directory will use a later modification time than the 
> >>> creation of the temporary file.
> >>
> >> If you really want cron to have sub-second granularity I don't see how
> >> you could do it without using flags.
> >>
> >> crontab open	sets flag that it is editing a file
> >> crontab close	clears "editing" flag, sets "something changed" flag
> >> 		(if something actually changed of course)
> >>
> >> cron	checks existence of "something changed" flag, pulls the
> >> 	update if there is no "editing" flag, clears "changed" flag
> > 
> > I don't want it to have sub-second granularity,
> 
> Ok, I was interpolating, sorry if I misinterpreted your intentions.
> 
> > I just want to make
> > 'crontab -e' more reliable so that cron doesn't miss edits.  cron is
> > currently using the mod-time of the spool directory as the 'something
> > changed' flag (have you read the cron code?).
> 
> I understand the spool behavior from history/experience, and I am
> relying on your excellent summary for the details. :)
> 
> > The problem is that it
> > currently can set the 'something changed' flag non-atomically while it is
> > updating a crontab.
> 
> That much I understood from your post. My response to what it is I think
> you're trying to achieve is that it's not likely that you can achieve it
> by only using 1 flag, no matter what that 1 flag is. I may be wrong
> about that, but hopefully my suggestion gives you some other ideas to
> consider.

Eh, I think the mod-time on the directory is fine.  It will change anything
anything in that directory changes, so that is good enough, and it's ok to
lose the race in the direction of doing extra scans of the spool directory,
it just needs to avoid not doing enough scans.

> Meanwhile, I was thinking more about this and TMK cron doesn't actually
> *run* jobs with seconds granularity, only minutes, right?  If so then it
> seems that the only really important seconds to care about are :59 and
> :00. That would seem to present a solution that rather than having cron
> wake up every second to see if something has changed that it only do
> that at :59 (or however many seconds in advance of :00 that it needs,
> although if it's more than 1 I'll be surprised). That limits the race to
> someone who writes out a new crontab entry at the point during second
> :59 that is after cron wakes up to look but before :00. So that's not a
> perfect solution to your problem, but it should limit the race to a very
> narrow window without having to modify the code very much.

Hmm, did you see my original e-mail?  I have a one-line fix to crontab(8). :)
Do you think that patch is invalid?

Also, the race is not what you think it is: cron already only checks once a
minute, the problem is a crontab(8) replace happening concurrently with
cron's scan of the spool directory in such a way that cron only sees part of
the actions crontab(8) performs while updating a tab.

-- 
John Baldwin