Re: reproducible panic with mount_smbfs

On Sun, Nov 02, 2008 at 06:53:25PM +0100, Attilio Rao wrote:
> 2008/11/2, Yuri Pankov <yuri.pankov_at_gmail.com>:
> > On Sun, Nov 02, 2008 at 05:17:18PM +0100, Attilio Rao wrote:
> >  > 2008/11/2, Attilio Rao <attilio_at_freebsd.org>:
> >  > > 2008/11/2, Yuri Pankov <yuri.pankov_at_gmail.com>:
> >  > >
> >  > > > Hi,
> >  > >  >
> >  > >  >  Trying to mount nonexistent smb share with mount_smbfs leads to
> >  > >  >  following panic:
> >  > >  >
> >  > >  >  # mount_smbfs //yuri_at_lifebane/blahblah /mnt
> >  > >  >
> >  > >  >  Unread portion of the kernel message buffer:
> >  > >  >  smb_co_lock: recursive lock for object 1
> >  > >  >  panic: Lock (lockmgr) smb_vc not locked _at_
> >  > >  >  /usr/src/sys/modules/smbfs/../../netsmb/smb_conn.c:329.
> >  > >  >  cpuid = 0
> >  > >  >  KDB: stack backtrace:
> >  > >  >  db_trace_self_wrapper() at db_trace_self_wrapper+0x2a
> >  > >  >  panic() at panic+0x182
> >  > >  >  witness_assert() at witness_assert+0x21a
> >  > >  >  __lockmgr_args() at __lockmgr_args+0x17a
> >  > >  >  smb_co_put() at smb_co_put+0x76
> >  > >  >  smb_sm_lookup() at smb_sm_lookup+0xfe
> >  > >  >  smb_usr_lookup() at smb_usr_lookup+0xcd
> >  > >  >  nsmb_dev_ioctl() at nsmb_dev_ioctl+0x1f6
> >  > >  >  giant_ioctl() at giant_ioctl+0x75
> >  > >  >  devfs_ioctl_f() at devfs_ioctl_f+0x76
> >  > >  >  kern_ioctl() at kern_ioctl+0x92
> >  > >  >  ioctl() at ioctl+0xfd
> >  > >  >  syscall() at syscall+0x1bf
> >  > >  >  Xfast_syscall() at Xfast_syscall+0xab
> >  > >  >  --- syscall (54, FreeBSD ELF64, ioctl), rip = 0x800939aec, rsp =
> >  > >  >  0x7fffffffe038, rbp = 0x7fffffffe450 ---
> >  > >  >  Uptime: 6m46s
> >  > >  >  Physical memory: 2032 MB
> >  > >
> >  > >
> >  > > So, what is happening here is that smb_co_lock() is AFU.
> >  > >  Infact looking at the code:
> >  > >  int
> >  > >  smb_co_lock(struct smb_connobj *cp, int flags, struct thread *td)
> >  > >  {
> >  > >  ...
> >  > >         if (smb_co_lockstatus(cp, td) == LK_EXCLUSIVE &&
> >  > >             (flags & LK_CANRECURSE) == 0) {
> >  > >                 SMBERROR("recursive lock for object %d\n", cp->co_level);
> >  > >                 return 0;
> >  > >         }
> >  > >  ...
> >  >
> >  > Yuri,
> >  > could you please test this fix:
> >  > http://www.freebsd.org/~attilio/netsmb.diff
> >  >
> >  > and report if it works?
> >  > You could get a KASSERT running but this is expected as I want to
> >  > identify on the callers who passes a malformed request and fix it.
> >  >
> >  > Thanks,
> >  > Attilio
> >  >
> >  >
> >  > --
> >  > Peace can only be achieved by understanding - A. Einstein
> >
> >
> > Thanks, Attilio.
> >
> >  With this patch system doesn't panic anymore with nonexistent share
> >  names (though I had to comment out smb_co_lockstatus prototype and
> >  function to get rid of -Werror complaints). Still getting a LOR:
> >
> >  netsmb_dev: loaded
> >  lock order reversal:
> >   1st 0xffffff0021644008 smb_vc (smb_vc) _at_
> >  /usr/src/sys/modules/smbfs/../../netsmb/smb_conn.c:332
> >   2nd 0xffffffff81037368 smbsm (smbsm) _at_
> >  /usr/src/sys/modules/smbfs/../../netsmb/smb_conn.c:348
> >
> > KDB: stack backtrace:
> >  db_trace_self_wrapper() at db_trace_self_wrapper+0x2a
> >
> > _witness_debugger() at _witness_debugger+0x2e
> >  witness_checkorder() at witness_checkorder+0x81e
> >  __lockmgr_args() at __lockmgr_args+0xc2a
> >  smb_co_lock() at smb_co_lock+0x38
> >  smb_co_gone() at smb_co_gone+0x38
> >
> > smb_sm_lookup() at smb_sm_lookup+0xfe
> >  smb_usr_lookup() at smb_usr_lookup+0xcd
> >  nsmb_dev_ioctl() at nsmb_dev_ioctl+0x1f6
> >  giant_ioctl() at giant_ioctl+0x75
> >  devfs_ioctl_f() at devfs_ioctl_f+0x76
> >  kern_ioctl() at kern_ioctl+0x92
> >  ioctl() at ioctl+0xfd
> >  syscall() at syscall+0x1bf
> >  Xfast_syscall() at Xfast_syscall+0xab
> >  --- syscall (54, FreeBSD ELF64, ioctl), rip = 0x800939aec, rsp =
> >
> > 0x7fffffffe048, rbp = 0x7fffffffe460 ---
> 
> I've updated the patch in order to fix smb_co_lockstatus problem.
> Could you please stress test it while I investigate the LOR problem?

Not sure what do you mean by "stress test". I've tried mounting several
different shares and copied ~100Gb from them, hope this should suffice.

> Are you running with INVARIANTS?

Yes.

> 
> Thanks,
> Attilio
> 
> 
> -- 
> Peace can only be achieved by understanding - A. Einstein


Thanks,
Yuri