Re: ffs_clusteralloc: allocated out of group on 5TB fs (5.3-Stable)

On Mon, Dec 06, 2004 at 10:56:34AM -0800, Doug White wrote:
> On Sat, 4 Dec 2004, Henry Whincup wrote:
> 
> > On a machine with a 5TB array I created a single partition via gpt and a
> > filesystem using the defaults from newfs.
> >
> > All seemed well until I tried to fill the partition with data.  With "only"
> > 870GB or so used utilities started reporting the disk as full. Shortly (a
> > few hours) after the machine panic'd.

I, too, have a 5tb box (two 2.5tb 3ware 9xxx raid5's and gstripe), and
see the issue fairly quickly. One of my users has a 50gb tarball of his
homedir, and about 20g into untarring, it panics with the message. 5.3p1

> 
> Ooh, fun. I wish I had a 5TB array to play with :)
> 
> > During this time I had been trying to copy file via nfs and rsync onto the
> > array.  After several crashes I tried a simple "dd if=/dev/zero of=./file"
> > and after writing ~33GB the machine rebooted (the panic details below are
> > from this reboot)

I wasn't able to get dd to crash it, even with 100g out file.

> >
> > Details follow:
> 
> [..]
> > Backtrace:
> > #0  0xc060ce82 in doadump ()
> > #1  0xc060d47b in boot ()
> > #2  0xc060d7a1 in panic ()
> > #3  0xc073445e in ffs_clusteralloc ()
> > #4  0xc0733680 in ffs_hashalloc ()
> > #5  0xc07329ea in ffs_reallocblks_ufs2 ()
> > #6  0xc073205a in ffs_reallocblks ()
> > #7  0xc0657e7c in cluster_write ()
> > #8  0xc074c350 in ffs_write ()
> > #9  0xc066bf10 in vn_write ()
> > #10 0xc062d624 in dofilewrite ()
> > #11 0xc062d4d1 in write ()
> > #12 0xc07bb07f in syscall ()
> > #13 0xc07a8a0f in Xint0x80_syscall ()
> >
> > Kernel is the default SMP kernel and sources are from 28th Nov 2004.
> >
> > I assume a proper debug kernel with symbols would be more helpful, but I
> > just wanted to know if this was a known bug - or something I have done
> > wrong, first.
> > Other than that is there anything else I can provide to help find this bug?
> 
> Very large FFS filesystems have not been extensively tested. Maybe you can
> help :)

dang

> 
> You should be able to compile GENERIC with
> 
> makeoptions	"DEBUG=-g"
> 
> in the config file and that will leave a kernel.debug in the compile
> directory that you can use.
> 
> Once you get that loaded up the output of 'bt full' would be nice to have.
> I suspect an integer overflow in dtog(), which is a macro.
> 
> 1545         if (dtog(fs, bno) != cg)
> 1546                 panic("ffs_clusteralloc: allocated out of group");

a couple weeks ago, when I found the issue, I tried dumping that line
using a printf and got

dtog(0xc4896800, -1894026712) != 25518

(I think I used %x, %d, %d)

I've no idea what those #'s mean, and couldn't find any documentation on
'dtog' (something to group?)

I have a kernel with the DEBUG=-g option, (and config -g), can't get the
dang thing to dump, though

> 
> -- 
> Doug White                    |  FreeBSD: The Power to Serve
> dwhite_at_gumbysoft.com          |  www.FreeBSD.org
> _______________________________________________
> freebsd-current_at_freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-current
> To unsubscribe, send any mail to "freebsd-current-unsubscribe_at_freebsd.org"
> 

-- 
        -Erik <erik_at_smluc.org> [http://math.smsu.edu/~erik]

The opinions expressed by me are not necessarily opinions. In all probability,
they are random rambling, and to be ignored. Failure to ignore may result in
severe boredom or confusion. Shake well before opening. Keep Refrigerated.