Re: HEADS-UP: installworld on r299292 through r299317 will replace master.passwd, passwd, and group files

On Tue, May 10, 2016 at 10:22:04AM -0700, John Baldwin wrote:
> On Tuesday, May 10, 2016 05:12:28 PM Glen Barber wrote:
> > On Tue, May 10, 2016 at 10:04:47AM -0700, John Baldwin wrote:
> > > On Tuesday, May 10, 2016 06:32:29 AM Glen Barber wrote:
> > > > On Tue, May 10, 2016 at 08:25:22AM +0200, Thomas Zander wrote:
> > > > > On 10 May 2016 at 08:18, O. Hartmann <ohartman_at_zedat.fu-berlin.de> wrote:
> > > > > 
> > > > > > I haven't figured out so far how far this goes. Lucky for those having
> > > > > > recent /etc/ backups. A pity FreeBSD doens't backup this by default.
> > > > > 
> > > > > After having shot myself in the foot some time ago, "zfs snapshot" has
> > > > > become a part of my standard upgrade procedures :-)
> > > > > 
> > > > 
> > > > No argument that this is valuable, but we cannot rely on filesystem
> > > > specific solutions.  Similar topic came up a few days ago following
> > > > lunch.  It got me thinking of a better way to ensure this kind of thing
> > > > does not require home-grown foot protection from cannons.
> > > > 
> > > > It should be fairly trivial to automatically backup /etc (and related)
> > > > when 'distribution' is run, either intentionally or accidentally (or by
> > > > commit mistakes, such as this).
> > > 
> > > Saving the output of 'etcupdate diff' nightly might not be a bad first step.
> > > 
> > 
> > This is also a good way to alleviate such things, however I am unsure
> > how to handle cases where 'etcupdate' would inadvertently run into
> > a conflict.  This was my concern with implementing an "automatic"
> > etcupdate run in the runtime package.
> 
> I mean as part of the nightly jobs we could add one that stores
> 'etcupdate diff' in /var the same as we do with backups of the master.passwd,
> group, and aliases files in /var/backups.  You can then at least use that to
> reconstruct altered /etc files by applying the diffs.  This isn't meant to be
> an automated update run, but just saving a diff as part of the nightly jobs.
> 

To be clear, I do not disagree with the idea as a whole.  However,
I think we should considering making this part of the 'installworld' (or
a dependent step of installworld) so we don't need to rely on periodic
script execution.  (Consider cases where one may shut down their laptop
before going to sleep, and the job never runs).

> As far as what to do in runtime packages, presumably there isn't a single
> package with all of etc, but etc files can be split up (ppp.conf in the ppp
> package, etc.) and pkg needs to do its own 3-way merge of changes to conf
> files when upgrading.

As the state of things are now, /etc is not included in any package
(config files exempted), but pkg(8) does have merge ability now.  The
problem with the initial implementation of "packaging /etc" was that it
broke etcupdate(8) and presumably mergemaster(8), as the files were now
part of an 'install'-style target, not 'distribute'-style target.

But, I think you gave me an idea on how to handle this.  (I'll test what
I have in mind, but I'm not sure if it will continue to break the merge
tools again as a side effect.)

> (This would be nice for conf files for ports in
> /usr/local/etc as well.)  You still need to figure out how to handle
> conflicts, but if pkg manages /etc files as config files and does a 3-way
> merge of the old package and new package then that will serve to reimplement
> etcupdate as part of 'pkg upgrade'.

Merge conflicts where human intervention is required is why I am
hesitant to add an implicit 'etcupdate' invocation as a post-install
script, as it breaks automated, non-interactive upgrades.

> Having a 'pkg confdiff' or some such to
> output diffs made to conf files would be the equivalent of 'etcupdate diff'
> in that case (and would be nice as it would apply to conf files in ports as
> well).
> 

Agreed.

But, to be honest, I'd like to use etcupdate for this if it comes down
to it.  We use it in the cluster, and have never run into an issue
(until I introduced the change mentioned above, which removed things
like /etc/auto_master (part of the autofs package).

Glen