Re: GEOM hangover disables NFS

From: Doug Rabson <dfr_at_rabson.org>
Date: Fri, 14 Nov 2008 10:36:51 +0000
On 14 Nov 2008, at 10:05, Ivan Voras wrote:

> Steve Kargl wrote:
>> On three nodes in my cluster (nodes n17, n18, and n19), I had
>> GEOM use /dev/ad4s1e for tests with gmirror and ggated/ggatec.
>> I found that GEOM was insufficient for my needs and decided
>> to return the 3 partitions to NFS-exported partitions.  It seems
>> that once GEOM touches a partition, the partition can no longer
>> be used by NFS.
>>
>> I'll illustrute the problem with n17:/dev/ad4s1e.  In what follows,
>> n10 is the master node.  Both n10 and n17 have brand new worlds
>> and kernels from about 45 minutes ago.
>>
>> n10:kargl[203] ssh n17
>> n17:kargl[201] df
>> Filesystem        1M-blocks  Used  Avail Capacity  Mounted on
>> /dev/ad4s1a             247   104    123    46%    /
>> devfs                     0     0      0   100%    /dev
>> /dev/ad4s1e          222780     0 204958     0%    /data
>> /dev/ad4s1d            3962   182   3463     5%    /usr
>> n10:/home            193947 92855  85576    52%    /home
>> n10:/usr/local        19832 10494   7750    58%    /usr/local
>>
>> n17:kargl[202] tail -1 /etc/exports
>> /data   -alldirs        node10 node21
>>
>> The above is after a 'newfs -U /dev/ad4s1e' and a reboot.
>>
>> n10:root[244] ls / | grep -E ^n
>> n11/ n12/ n13/ n14/ n15/ n16/ n17/ n18/ n19/ n20/ n21/
>>
>> n10:root[245] mount_nfs -o tcp n17:/data /n17
>> n10:root[246] mount -v | grep n17
>> n17:/data on /n17 (nfs, fsid 0eff000303000000)
>> n10:root[247] ls /n17
>> ls: /n17: Input/output error
>> n10:root[248] ls / | grep -E ^n
>> ls: n17: Input/output error
>> n11/ n12/ n13/ n14/ n15/ n16/ n18/ n19/ n20/ n21/
>>
>> n10:root[251] umount /n17
>> n10:root[252] ls / | grep -E ^n
>> n11/ n12/ n13/ n14/ n15/ n16/ n17/ n18/ n19/ n20/ n21/
>>
>> So, how does one exorcise GEOM from /dev/ad4s1e?
>
> Your message is very confusing. Here's some information that could  
> help you:
>
> * "GEOM" is just a name for "system of handling disk-like devices in
> FreeBSD". If you "remove GEOM" (which would probably be hard to do and
> would involve heavily modifying the kernel source) you will have no
> access to your disk drives and other similar devices.
>
> * Using ggated/ggatec you can either:
>   - mount the file system exactly once as rw and none as ro (doesn't
> matter if the file system is on local node or exported, the total is
> important)
>   - mount the file system as ro any number of times and none as rw.
> (In other words, only one machine can have the file system mounted for
> reading and writing. Once a file system has been mounted as rw, no  
> other
> machines can do anything with it. This has nothing to do with NFS as  
> NFS
> operates on top of a mounted file system and inherits its rules.)
>
> * You can remove gmirror metadata on devices with "gmirror clean".
> ggated and ggatec don't store metadata on devices.

This was a bug in the NFS code which is now fixed. The bug had nothing  
to do with GEOM.
Received on Fri Nov 14 2008 - 09:36:53 UTC

This archive was generated by hypermail 2.4.0 : Wed May 19 2021 - 11:39:37 UTC