Re: BETA4: Panic and can't cleanup filesystem

On 17 Sep, B. Bonev wrote:
> When trying to unmount my flash drive with force (#umount -f /flash) machine
> hardlocked.
> After restart, when fsck-ing in background, computer panicked.
> Machine is SMP 2x500MHz PIII with Asus P2B-DS motherboard
> cvsuped and updated to RELENG_6 (BETA4) on 14 september with SHED_ULE and
> PREEMPTION
> panic is written on hand:
> 
> panic: handle_written_inodeblock: live inodedep
> cpuid=0
> KDB: enter: panic
> [thread pid3 tid 100033]
> stopped at 	kdb_enter+0x2b:nop
> db> where
> Tracing pid 3 tid 100033 td 0xc15d17d0
> kdb_enter(c062b964) at kdb_enter+0x2b
> panic(c063a716,0,2,0,0) at panic+0x127
> handle_written_inodeblock(c1809a00,cbc1a738) at
> handle_written_inodeblock+0x533
> softdep_disk_write_complete(cbc1a738) at softdep_disk_write_complete+0xb6
> bufdone(cbc1a738) at bufdone+0x160
> g_vfs_done(c1825ad4) at g_vfs_done+0x8a
> biodone(c1825ad4,d42e8cc4,0,c062632c,1ea) at biodone+0x57
> g_io_shedule_up(c15d17d0) at g_io_shedule_up+0xb5
> g_up_procbody(0,d42e8d38,0,c04be5c0,0) at g_up_procbody+0x5a
> fork_exit(c04be5c0,0,d42e8d38 at fork_exit+0xa0
> fork_trampoline() at fork_trampoline+0x8
> --- trap 0x1,eip=0,esp=0xd42e8d6c,ebp=0 ---
> db>show alllocks
> Process 3 (g_up) thread 0xc15d17d0 (100033)
> exclusive sleep mutex SoftdepLock r=0 (0xc06dbda0) locked _at_
> /usr/src/sys/ufs/ffs/ffs_softdep.c:4075
> exclusive sleep mutex g_xup r=0 (0xd42e8cc4) locked _at_
> /usr/src/sys/geom/geom_io.c:490
> Process 31 (irq20:acpi0) thread 0xc15cd320 (100021)
> exclusive sleep mutex acpica subsystem lock r=0 (0xc15c0280) locked _at_
> /usr/src/sys/modules/acpi/../../../dev/acpica/Osd/Osdynch.c:361
> db> panic
> Dump...

That looks like this problem:

<http://people.freebsd.org/~pho/stress/log/cons162.html>

which I'm pretty sure was on the TODO list for 6.0-RELEASE
<http://www.freebsd.org/releases/6.0R/todo.html> as a showstopper bug. I
don't see it there now, so it might have been fixed in the last few
days.


> after restart, system stoped and entered in single mode with inconsistency
> in /var
> after fsck, there was message:
> 
> BAD SUPERBLOCK: VALUES IN SUPERBLOCK DISAGREE WITH THOSE IN FIRST ALTERNATE
> 
> then search for SUPERBLOCK failed,
> and fsck don't have option -b...

But fsck_ufs does, and it looks like you can pass this with the fsck -T
option.  Try something like:
	fsck -Tufs,-b ...


> tried to mount all filesystems and got another panic:
> # /var: bad dir ino 2 at offset 0: mangled entry
> panic: ufs_dirbad: bad dir
> cpuid=0
> KDB: enter: panic
> [thread pid 133 tid 100063]
> stoped at	kdb_enter+0x2b: nop
> db>where
> Tracing pid 133 tid 100063 td 0xc18554000
> kdb_enter(c062b964) at kdb_enter+0x2b
> panic(c063b64c,c1829800,da96c904,c059eb66,c18866b4) at panic+0x127
> ufs_dirbad(c18866b4,0,c063b606,c1854000,0) at ufs_dirbad+0x3a
> ufs_lookup(da96c92c) at ufs_lookup+0x36a
> VOP_CACHEDLOOKUP_APV(c066fd80,da96c92c) at VOP_CACHEDLOOKUP_APV+0x87
> lookup(da96cbcc,c0515064,0,c1854000,c06bc7c0) at lookup+0x3d6
> namei(da96cbcc,c062cf75,267,c1854000,da96ca68) at namei+0x35a
> vn_open_cred(da96cbcc,da96cccc,0,c15c9a80,3) at vn_open_cred+0x277
> vn_open(da96cbcc,da96cccc,0,3,c062cf75) at vn_open+0x1e
> kern_open(c185400,28065e40,0,1,0) at kern_open+0xb6
> open(c185400,da96cd04,3,1,296) at open+0x1a
> syscall(3b,3b,3b,28070000,80486b1) at syscall+0x27f
> xint0x80_syscall() at xint0x80_syscall+0x1f
> --- syscall(5,FreeBSD ELF32,open),
> eip=0x28054e67,esp=0xbfbfebcc,ebp=0xbfbfec78 ---
> 
> With every restart, system going in single mode. What can i do to clean /var
> filesystem and to debug these panics? Any help are welcome...
>