Re: panic: ffs_blkfree: freeing free block

Hey Kostik,

2007/10/18, Kostik Belousov <kostikbel_at_gmail.com>:
>
> On Wed, Oct 17, 2007 at 07:05:59AM -0500, Eric Anderson wrote:
> > Kostik Belousov wrote:
> > >On Tue, Oct 16, 2007 at 12:20:59PM +0200, Arjan van Leeuwen wrote:
> > >>2007/10/16, Eric Anderson <anderson_at_freebsd.org>:
> > >>>Pieter de Goeje wrote:
> > >>>>On Sunday 14 October 2007, Eric Anderson wrote:
> > >>>>>Pieter de Goeje wrote:
> > >>>>>>On Saturday 13 October 2007, Rolf Witt wrote:
> > >>>>>>>Backtrace from crash tonight (after portsnap cron):
> > >>>>>><snip>
> > >>>>>>
> > >>>>>>># uname -a
> > >>>>>>>FreeBSD peanuts.homenet.local 7.0-PRERELEASE FreeBSD
> 7.0-PRERELEASE
> > >>>>>>>#121: Thu Oct 11 16:29:05 CEST 2007
> > >>>>>>>rowi_at_peanuts.homenet.local:/usr/obj/usr/src/sys/PEANUTS  i386
> > >>>>>>>
> > >>>>>>>Sources from Builddate.
> > >>>>>>>
> > >>>>>>>Rowi
> > >>>>>>Eric sent me this patch:
> > >>>>>>http://www.googlebit.com/freebsd/patches/ffs_softdep.c-patch
> > >>>>>>which seems to be working great so far. I am still testing it, but
> I
> > >>>>>>think it fixed the problem.
> > >>>>>>To apply, cd /usr/src; patch < /path/to/patch and then rebuild the
> > >>>>>>kernel.
> > >>>>>>
> > >>>>>>Cheers,
> > >>>>>>Pieter de Goeje
> > >>>>>It doesn't actually 'fix' the problem, but I think it helps
> identify
> > >>>it.
> > >>>>>I'm not 100% certain if this is the right fix our not, but so far
> > >>>>>feedback has been good when running with this patch.
> > >>>>>
> > >>>>>Can somebody confirm that this patch is ok?
> > >>>>>
> > >>>>>Eric
> > >>>>Can you elaborate on what this patch exactly does / what the problem
> is?
> > >>>>
> > >>>>Pieter
> > >>>
> > >>>I think the problem is that blocks are being put in the worklist
> twice,
> > >>>but I'm not certain why yet.  The patch reduces the chance of this
> > >>>happening by more often removing the block from the worklist instead
> of
> > >>>leaving it on.  I think actually the patch is hiding the real issue
> more
> > >>>than anything, which is why I said it isn't fixing the problem, but
> > >>>because of the reports I've seen, I think we're sniffing in the right
> > >>>area.  I'll be looking more into this soon, when I get another few
> > >>>minutes of free time and my debugger..
> > >>I actually seem to be getting the panic more often with the patch (i.e
> .
> > >>twice in just 30 minutes with the patch, about twice per day without
> the
> > >>patch), while doing the same things as usual. The panic usually
> happens
> > >>when
> > >>I'm compiling or linking. Is that strange?
> > >
> > >Patch is wrong. Actually, it does put the dirrem to the proccessing
> twice
> > >when xp != NULL.
> >
> > Yes, exactly why I said 'it doesn't fix the problem'.  In fact, I
> > actually did not post it to the list, and didn't intend for it to be
> > posted.  I was hoping for feedback for further examination.
> >
> > Kostik - do you have any ideas on what is going on here?  Did you see
> > the message in another thread that looked similar (it was regarding
> > gjournal)?
>
> No, I have no idea. BTW, some further information from the people that
> experience the problem could be helpful. For instance,
> UFS1/UFS2

UFS2, filesystem created with FreeBSD 6-STABLE

size of the problematic fs/% of the space used

 /dev/ad4s1g 90613368 22619950 60744350 27% /home

are quotas enabled in kernel/active for the fs ?

No to both.

are softupdates on ?

Yes. Turning them off seems to fix the problem (fingers crossed - I only
turned it off this morning, didn't have a panic yet)

what block number is reported as being freed twice ? is it always the same ?
> what is the full fsck diagnostic after reboot ?

I'm in X when it happens, so I can't see the message. It doesn't say which
block is being freed on reboot.
I haven't saved the fsck output either. I can turn softupdates back on
tonight and reproduce the panic to get a full fsck log, if you want me to.

The range of questions is so wide because I try to somewhat localize the
> search field down from "anywhere".
>
>
np :)

Arjan