Re: Panic in ZFS: solaris assert: dn->dn_datablkshift != 0

On Sat, Sep 07, 2013 at 02:35:45PM +0200, Jeremie Le Hen wrote:
> Hi,
> 
> I have the following panic every time I do a zfs receive on a given
> dataset.
> 
> For the background, I synchronize a zfs dataset every couple of minutes
> using zfs send/receive.
> 
> I think I recently got a panic (I'm running mav_at_'s geom direct dispatch
> patch) which probably happen at a bad time and left the snapshot/data in
> an inconsistent state.  Now, whenever my cron job runs, it triggers the
> panic.
> 
> The process that triggers the panic is:
>     zfs receive -F data/jail/caravan
> 
> Probably relevant, on boot, I have the following message:
>     Solaris: WARNING: can't open objset for data/jail/caravan/%recv
> 
> 
> I have a core around if needed to debug.  I will not try to repair the
> snapshot/dataset during this weekend, to get a chance to test a patch.
> Afterward I will have to start this job again.
> 
> 
> panic: solaris assert: dn->dn_datablkshift != 0, file: /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/dmu_tx.c, line: 638
> cpuid = 1
> KDB: stack backtrace:
> db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame
> 0xfffffe00e62401a0
> kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe00e6240250
> vpanic() at vpanic+0x126/frame 0xfffffe00e6240290
> panic() at panic+0x43/frame 0xfffffe00e62402f0
> assfail() at assfail+0x22/frame 0xfffffe00e6240300
> dmu_tx_hold_free() at dmu_tx_hold_free+0x167/frame 0xfffffe00e62403e0
> dmu_free_long_range() at dmu_free_long_range+0x1f5/frame
> 0xfffffe00e6240450
> dmu_free_long_object() at dmu_free_long_object+0x1f/frame
> 0xfffffe00e6240480
> dmu_recv_stream() at dmu_recv_stream+0x86e/frame 0xfffffe00e62406b0
> zfs_ioc_recv() at zfs_ioc_recv+0x96c/frame 0xfffffe00e6240920
> zfsdev_ioctl() at zfsdev_ioctl+0x54a/frame 0xfffffe00e62409c0
> devfs_ioctl_f() at devfs_ioctl_f+0xf0/frame 0xfffffe00e6240a20
> kern_ioctl() at kern_ioctl+0x2ca/frame 0xfffffe00e6240a90
> sys_ioctl() at sys_ioctl+0x11f/frame 0xfffffe00e6240ae0
> amd64_syscall() at amd64_syscall+0x265/frame 0xfffffe00e6240bf0
> Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe00e6240bf0
> --- syscall (54, FreeBSD ELF64, sys_ioctl), rip = 0x8019ddf1a, rsp =
> 0x7fffffff5c08, rbp = 0x7fffffff5c90 ---

I rolled back my kernel arbitrarily in the past (2013/08/01).  The panic
doesn't happen any more.

I will try to narrow this down by dichotomy but that will be more
efficient if someone has a rough idea wherefrom the problem appeared.

-- 
Jeremie Le Hen

Scientists say the world is made up of Protons, Neutrons and Electrons.
They forgot to mention Morons.