NetBSD mtree committed

I have committed NetBSD's mtree to the tree and am building and
installing it as nmtree.  I plan to replace our mtree with this version
after the following steps:

 - Add a WITH_NMTREE build option to install nmtree as mtree and the
   old mtree as omtree.
 - Switch the default to WITH_NMTREE.
 - Remove the old mtree.
 - Rename usr.sbin/nmtree to usr.sbin/mtree.

During this process I will also commit patches to switch makefs
to use NetBSD's mtree which will make the -F specfile option more
useful.

As a reminder we are doing this because, we will gain the -C option
which produces mtree files compatible with libarchive and makefs (one
line per file with full path) and the -N option which allows a stand
alone set of passwd and group files.  When mated with the -U and -M
options to NetBSD's install we will have most of the pieces require to
allow installworld to run as a user and then build images containing
proper permissions.

The new mtree does introduce some incompatibilities, but nearly all can
be overcome with small command line changes.  With one exception,
FreeBSD 9 compatible behavior can be restored by adding the "-F
freebsd9" to the command line.  In some cases new warnings are
generate to aid transition, but the output should otherwise be the same.
If you find cases where it is not, please let me know.

Known incompatibilities are:

 - The -u and -U (update) options do not update the modification time or
   set file flags unless the -t and -i options are passed respectively.
 - Because the -i option as already take, FreeBSD's option to indent
   4 spaces for each directory level is now -j.
 - The -d (directories only) option does not omit blank lines when
   entering a new directory or leaving one.  The -b option now enables
   this behavior independently.
 - The handling of the uname and group keywords when the uid or gid can
   not be resolved is changed.  In the new code, when the uname or group
   keyword is request and the name can not be found a uid or gid keyword
   is emitted instead.  Historically, mtree would report and error and
   exit unless the -w option was passed.  If that happened then a
   warning was printed and no keyword was emitted.  That resulted in
   potentially dangerous /set statements.  As a result I declined to
   implement this behavior.

   Here is an example of the dangerous \set statements:

   $ ls -l
   total 0
   -rwxr-xr-x  1 root   wheel  0 Dec 21 14:13 a
   -rwsr-xr-x  1 12345  wheel  0 Dec 21 14:13 b
   $ mtree -c -p . -n -k uname,mode -w > ../mtree.out
   mtree: Could not get uname for uid=12345
   $ cat ../mtree.out

   /set type=file uname=root mode=0755
   .               type=dir uname=brooks
       a          
       b           mode=04755
   ..

   $ mtree -f ../mtree.out -u -p .
   b:      user (0, 12345, modified)
   $ ll
   total 0
   -rwxr-xr-x  1 root  wheel  0 Dec 21 14:13 a
   -rwsr-xr-x  1 root  wheel  0 Dec 21 14:13 b
   $

I have heard some requests to MFC the new mtree code.  At this time I
have no concrete plans to do so.  If it were done then I would modify
the code to run with -F freebsd9 and would implement full uname/group
compatibility.

-- Brooks

----- Forwarded message from Brooks Davis <brooks_at_FreeBSD.org> -----

From: Brooks Davis <brooks_at_FreeBSD.org>
Date: Fri, 21 Dec 2012 21:00:01 +0000 (UTC)
To: src-committers_at_FreeBSD.org, svn-src-all_at_FreeBSD.org,
	svn-src-head_at_FreeBSD.org
Subject: svn commit: r244562 - in head: contrib/mknod contrib/mtree usr.sbin
	usr.sbin/nmtree

Author: brooks
Date: Fri Dec 21 21:00:00 2012
New Revision: 244562
URL: http://svnweb.freebsd.org/changeset/base/244562

Log:
  Add NetBSD's mtree to the tree and install it as nmtree as the first step
  towards replacing our mtree.
  
  Sponsored by:	DARPA, AFRL
  Thanks to:	cristos_at_NetBSD for reviewing and committing my patches
  		wiz_at_NetBSD for fixing typos in my patches

Added:
  head/contrib/mknod/
     - copied from r244548, vendor/NetBSD/mknod/dist/
  head/contrib/mtree/
     - copied from r244548, vendor/NetBSD/mtree/dist/
  head/usr.sbin/nmtree/
  head/usr.sbin/nmtree/Makefile   (contents, props changed)
Modified:
  head/usr.sbin/Makefile

Modified: head/usr.sbin/Makefile
==============================================================================
--- head/usr.sbin/Makefile	Fri Dec 21 20:50:47 2012	(r244561)
+++ head/usr.sbin/Makefile	Fri Dec 21 21:00:00 2012	(r244562)
_at__at_ -55,6 +55,7 _at__at_ SUBDIR=	adduser \
 	nfsdumpstate \
 	nfsrevoke \
 	nfsuserd \
+	nmtree \
 	nologin \
 	pc-sysinstall \
 	pciconf \

Added: head/usr.sbin/nmtree/Makefile
==============================================================================
--- /dev/null	00:00:00 1970	(empty, because file is newly added)
+++ head/usr.sbin/nmtree/Makefile	Fri Dec 21 21:00:00 2012	(r244562)
_at__at_ -0,0 +1,26 _at__at_
+#	$FreeBSD$
+
+.include <bsd.own.mk>
+
+.PATH: ${.CURDIR}/../../contrib/mtree
+
+PROG=		nmtree
+MAN=		nmtree.8
+SRCS=		compare.c crc.c create.c excludes.c getid.c misc.c mtree.c \
+		spec.c specspec.c verify.c
+LDADD+=		-lmd -lutil
+
+CFLAGS+=	-I${.CURDIR}/../../contrib/mknod
+.PATH:		${.CURDIR}/../../contrib/mknod
+SRCS+=		pack_dev.c
+
+CFLAGS+=	-I${.CURDIR}/../../lib/libnetbsd
+LIBNETBSDDIR=	${.OBJDIR}/../../lib/libnetbsd
+LIBNETBSD=	${LIBNETBSDDIR}/libnetbsd.a
+DPADD+=		{LIBNETBSD}
+LDADD+=		${LIBNETBSD}
+
+nmtree.8: mtree.8
+	cp ${.ALLSRC} ${.TARGET}
+
+.include <bsd.prog.mk>


----- End forwarded message -----