Re: Softupdates a mount option?

From: Stefan Eßer <se_at_FreeBSD.org>
Date: Wed, 26 May 2004 23:48:22 +0200
On 2004-05-26 21:15 +0200, Ivan Voras <ivoras_at_fer.hr> wrote:
> This has been really nagging me for a long time: Why aren't softupdates 
> made a mount option (like 'sync' and 'async')? Do I remember correctly that 
> it is done so in NetBSD (where it's called softdeps), so it's doable?

I prepared and posted patches to make soft updates a mount option several
years ago (June 2000, in fact). I'll just append the message I received
from Kirk McKusick to the end of this Mail. In case you are interested, 
I'll prepare a patch against a recent -current, tomorrow (the diff I add
to this mail is as of 4 years ago).

I'd be happy to commit my diffs (which I have kept in my system, all the
years) to -current, especially if NetBSD got that mount option 3 years ago.

Regards, STefan




>>From mckusick_at_mckusick.com Sat Jul  1 13:49:15 2000
Received: by hub.freebsd.org (Postfix)
	id CB4A437B9B7; Wed, 28 Jun 2000 13:51:08 -0700 (PDT)
Delivered-To: se_at_freebsd.org
Received: from beastie.mckusick.com (beastie.mckusick.com [209.31.233.184])
	by hub.freebsd.org (Postfix) with ESMTP
	id 8134937C1C5; Wed, 28 Jun 2000 13:51:01 -0700 (PDT)
	(envelope-from mckusick_at_mckusick.com)
Received: from beastie.mckusick.com (localhost [127.0.0.1])
	by beastie.mckusick.com (8.9.3/8.9.3) with ESMTP id NAA05776;
	Wed, 28 Jun 2000 13:51:01 -0700 (PDT)
	(envelope-from mckusick_at_beastie.mckusick.com)
Message-Id: <200006282051.NAA05776_at_beastie.mckusick.com>
To: arch_at_freebsd.org
Subject: Re: Soft updates mount change
Cc: Stefan Esser <se_at_freebsd.org>
Date: Wed, 28 Jun 2000 13:51:00 -0700
From: Kirk McKusick <mckusick_at_mckusick.com>
Content-Type: text
Status: RO
Content-Length: 12566
Lines: 301

After a long discussion (see below) with Stefan Esser, I have
agreed that it is reasonable to make soft updates a mount option
once we flip over to making them on by default. If you do not
want soft updates, you will add the option `nosoftdep' to your
fstab entry. That makes the extra options in /etc/fstab the
exception rather than the rule and generally does not require
naive administrators to know about them. It also avoids having
a flag day when all the entries in /etc/fstab need to be flipped
over as adding them now would do. I do not anticipate making 
them default on until they have had wider operational experience
from the new licensing and until I resolve the problems with
using them on nearly full filesystems (writes failing because
space has not yet been freed from recently deleted files).

	Kirk McKusick

=-=-=-=-=-=-=-=

Date: Wed, 28 Jun 2000 00:38:24 +0200
From: Stefan Esser <se_at_freebsd.org>
To: Kirk McKusick <mckusick_at_mckusick.com>
Cc: Stefan Esser <se_at_freebsd.org>, Stefan.Esser_at_o-tel-o.de
Subject: Re: Soft updates mount change
Reply-To: Stefan Esser <se_at_freebsd.org>

On 2000-06-24 12:00 -0700, Kirk McKusick <mckusick_at_mckusick.com> wrote:

Please grant me a last chance to state my reasons, why I think that 
soft-updates might become a mount option:

1) The pre-dominant reason to store the soft-updates status in the
   super-block is for FSCK.

2) File system update policy decisions are per file-system, not per 
   medium.

3) It should not be neccessary to have single-user console access to
   change the soft-updates enable bit. (Yes, I know about tunefs on
   a partition that is mounted R/O, but that will not work in all
   situations.)


Regarding 1:

There is a requirement, that fsck is able to determine whether 
the file-system was previously mounted with or with soft-updates.

This asks for a way to keep that information in a place where it
can't be deleted by accident, if the file system was not unmounted
in a clean state. Tunefs operates on unmounted file-systems, whether
clean or not. That means, that the SOFTDEP but in fs_flags can be 
modified either way on an unclean file system, as long as it is
not mounted.

The decision to use soft-updates should not share a bit with the
status flag that controls fsck's operation.


Regarding 2:

My policy to use or not use soft-updates depends on the file-system,
not the partition or medium (consider removable media). How about a
ZIP drive that is used with soft-updates in my system, but is written
to and disconnected without a clean unmount on a system, that does not
have soft-updates in the kernel (e.g. FreeBSD-2.2.8). The medium will 
require a fsck on my system, and fsck must assume it was last written
to with soft-update dependencies enforced, when in reality it was not.

This is a constructed case (I do not even own a ZIP drive), but meant 
to explain my point of view, that the use of soft-updates does not 
depend on the medium but rather on the file system.

My policy currently states, that soft-updates be used on all file
systems except "/" (since I had bad luck with /bin being only partly
populated after a "make installworld", which had required three times
the free space required by the new binaries).

I might have a policy, that requires /var to be written "sync", which
kind of conflicts with soft-updates.

I may have removable media that are moved between a number of systems
and mounted on different mount points, with changing access and usage
policies incl. whether async, sync or soft-updates are suitable for 
some particular use.

All these points aks for soft-updates to be configured per mount-point,
not per disk partition (i.e. *not* in the super-block).


Regarding 3:

I have a lot of systems in remote locations, some with only limited
access to the console. (For example, there may be a modem that is 
switched off, unless activated for the duration of a remote access.)

If I want to enable soft-updates on tens of machines, I'd rather only
distribute a configuration file, than connect to each machine, boot 
into single-user mode, enable soft-updates on each file-system with 
tunefs, and then let the system go back to multi-user.

There are other scenarios, where a system may receive a completely new
kernel, without an administrator present, who knows how to use tunefs.
(Think about PicoBSD: It should be possible to have soft-updates enabled
or disabled by a configuration file delivered with the boot floppy.)

The natural place for persistent per file-system configuration is either
the super-block, where disklabel or newfs store immutable parameters, 
that can't be modified without loosing all data on a partition.

The other place where such information is kept is /etc/fstab, which 
holds the usage, security and update policy decisions of a system in
per records per mount-point. (Even if the first column in fstab is the 
partition name, I consider fstab to specify what to mount on a *mount
point* and how to do it. but I admit that this is just my personal POV ;-)

It is of course possible the have /etc/rc (or one of the rc.* sub-tasks)
modify soft-updates before the "mount -a". But that would be more complex 
than my proposed solution.

> Your above proposal would work, though that is not how NetBSD
> implemented it. I feel that it is a lot of extra mechanism for very
> little gain. Administrators generally make a one-time decision to
> run soft updates on a filesystem. It is not the sort of thing that
> they want to change on a regular basis. It is possible to run tunefs

Agreed. But there are multiple places where this information may be 
stored. One is to hide it in the super-block, the other to expose it
in /etc/fstab. Since mount reports the soft-updates state, I think the
POLA asks for mount to be able to choose between softdep/nosoftdep.

> on a filesystem that is mounted read-only, so it no more difficult
> to use tunefs than it is to make it a mount-time option (i.e., they
> still have to down-grade to read-only, set the option, then upgrade).

This is not easily possible for some file-systems (like /var), where 
some processes keep files open for writing at all times and may fail,
if the file system mode is forced to R/O.

> Finally, I expect that soft updates will eventually just be defaulted
> to `on' when a filesystem is built, and in a few rare instances an
> administrator will want to turn it off. I do not want to have an
> option that needs to be added to nearly every fstab entry to get
> the default behavior.

I fully agree that the default should not have to be added to each 
line in fstab. But then it could still default to on in the kernel,
and a "nosoftdep" could disable it for selected file systems. But I
see your point. 

Hmmm, I still do not think that the command character of FS_DOSOFTDEP 
and the reporting character (was previously mounted with soft-updates) 
of the same bit are a good design choice. The two functions should be
separated. One way to do this is by  making soft-updates a mount option
as I propose, the other to have two flag bits in the super-block, one
enable soft-updates, the other to report to fsck whether it was enabled.

>                       Plus it is just one more bit of trivia that
> new system administrators need to learn to make their systems run
> well. The more of those details that need not be learned because
> they just do the right thing, the better.

But the tunefs method is anything but obvious. Well, if the next version 
of "sysinstall" offers to enable soft-updates in the partition editor, 
it will not be missed as easily as currently ...

I think that soft-updates should be on by default, except possibly on the
root file-system (which may happen to be too small: at least 15MB of free 
space are required for a "make installworld" or part of the programs in 
/bin and /sbin will be lost). There should of course not be different
defaults for / and other file-systems, since this would violate the POLA.
I'm still not convinced. I think that soft-updates should be enabled by
default, and some people may want to disable it on the root fs ...



I am writing this message on a system, which is modified to make softdep a 
mount option. This required a total of some 10 additional lines (incl. 
comments) to be added to ffs_vfsops.c, plus a few lines that changed from 
fs_flags to mnt_flag. A total of 3 lines had to be added to the mount program 
(two to enable/disable softdep, one to print its state for "mount -p").

Patches are appended below, just to show that this is a very minor change.
Please consider adding these patches at least at the time, when soft-updates
defaults to "on" ...

Best regards, STefan

*****************************************************************************
Patches to make softdep a "mount" option:

-> Initial mount (no update mount, file system currently not mounted):

The following patches make ffs_mount() perform a soft-updates mount
(which sets the FS_DOSOFTDEP bit in fs->fs_flags, if successful), if
all of the following conditions are true:

1) the MNT_SOFTDEP flag is set in mp->mnt_flag
2) soft-updates is compiled into the kernel
3) the file system is unmounted or mounted read-only

The FS_DOSOFTDEP bit is cleared, if the mount succeeds without 
MNT_SOFTDEP being set. This can be caused by either of:

1) a successful mount without the "-o softdep" option (or the 
   nosoftdep option, if softdep is to become the default).
2) a mount on a system without soft-updates support



-> Update mount (typically: read-only -> read-write):

If this is an update mount and the file-system is not clean, then the 
soft-updates enable status is "locked" (which is an even stronger 
condition than the requirement, that the file system be unmounted 
for "tunefs -n enable", which doesn't prevent changing an un-clean
file system and mis-directing a subsequent fsck run ...).

(If the file system is mounted R/O, the situation is identical to
the "Initial mount" case above.)


--- /sys/ufs/ffs/ffs_softdep.c	2000/06/22 00:29:53	1.69
+++ /sys/ufs/ffs/ffs_softdep.c	2000/06/27 18:22:49
_at__at_ -962,5 +962,5 _at__at_
 
 	mp->mnt_flag &= ~MNT_ASYNC;
-	mp->mnt_flag |= MNT_SOFTDEP;
+	fs->fs_flags |= FS_DOSOFTDEP;
 	/*
 	 * When doing soft updates, the counters in the
--- /sys/ufs/ffs/ffs_softdep_stub.c	2000/06/16 13:00:33	1.11
+++ /sys/ufs/ffs/ffs_softdep_stub.c	2000/06/27 18:15:44
_at__at_ -72,4 +72,5 _at__at_
 	struct ucred *cred;
 {
+	mp->mnt_flag &= ~MNT_SOFTDEP;
 
 	return (0);
--- /sys/ufs/ffs/ffs_vfsops.c	2000/06/04 04:50:36	1.122
+++ /sys/ufs/ffs/ffs_vfsops.c	2000/06/27 20:33:37
_at__at_ -196,4 +196,12 _at__at_
 		err = 0;
 		ronly = fs->fs_ronly;	/* MNT_RELOAD might change this */
+		if (ronly == 0) {
+			/* preserve current softdep status, unless mounted read-only */
+			if (fs->fs_flags & FS_DOSOFTDEP) {
+				mp->mnt_flag |= MNT_SOFTDEP;
+			} else {
+				mp->mnt_flag &= ~MNT_SOFTDEP;
+			}
+		}
 		if (ronly == 0 && (mp->mnt_flag & MNT_RDONLY)) {
 			flags = WRITECLOSE;
_at__at_ -244,5 +252,5 _at__at_
 
 			/* check to see if we need to start softdep */
-			if (fs->fs_flags & FS_DOSOFTDEP) {
+			if (mp->mnt_flag & MNT_SOFTDEP) {
 				err = softdep_mount(devvp, mp, fs, p->p_ucred);
 				if (err)
_at__at_ -658,4 +666,7 _at__at_
 		goto out;
 	}
+	/* clear softdep flag in superblock, if not a softdep mount */
+	if (mp->mnt_flag & MNT_SOFTDEP == 0)
+		fs->fs_flags &= ~FS_DOSOFTDEP;
 	ump = malloc(sizeof *ump, M_UFSMNT, M_WAITOK);
 	bzero((caddr_t)ump, sizeof *ump);
_at__at_ -746,5 +757,5 _at__at_
 		fs->fs_maxfilesize = maxfilesize;		/* XXX */
 	if (ronly == 0) {
-		if ((fs->fs_flags & FS_DOSOFTDEP) &&
+		if ((mp->mnt_flag & MNT_SOFTDEP) &&
 		    (error = softdep_mount(devvp, mp, fs, cred)) != 0) {
 			free(base, M_UFSMNT);


--- /usr/src/sbin/mount/mntopts.h	1999/10/09 11:54:07	1.16
+++ /usr/src/sbin/mount/mntopts.h	2000/06/27 19:06:19
_at__at_ -57,4 +57,5 _at__at_
 #define MOPT_NOCLUSTERW		{ "clusterw",	1, MNT_NOCLUSTERW, 0 }
 #define MOPT_SUIDDIR		{ "suiddir",	0, MNT_SUIDDIR, 0 }
+#define MOPT_SOFTDEP		{ "softdep",	0, MNT_SOFTDEP, 0 }
 
 /* Control flags. */
--- /usr/src/sbin/mount/mount.c	2000/01/15 14:28:13	1.39
+++ /usr/src/sbin/mount/mount.c	2000/06/27 20:14:39
_at__at_ -741,4 +741,5 _at__at_
 	if (flags & MNT_NOSYMFOLLOW)	res = catopt(res, "nosymfollow");
 	if (flags & MNT_SUIDDIR)	res = catopt(res, "suiddir");
+	if (flags & MNT_SOFTDEP)	res = catopt(res, "softdep");
 
 	return res;
--- /usr/src/sbin/mount/mount_ufs.c	1999/08/28 00:13:27	1.16
+++ /usr/src/sbin/mount/mount_ufs.c	2000/06/27 17:26:50
_at__at_ -67,4 +67,5 _at__at_
 	MOPT_SYNC,
 	MOPT_UPDATE,
+	MOPT_SOFTDEP,
 	{ NULL }
 };
Received on Wed May 26 2004 - 12:48:59 UTC

This archive was generated by hypermail 2.4.0 : Wed May 19 2021 - 11:37:55 UTC