Re: Any success stories for HAST + ZFS?

On Fri, Mar 25, 2011 at 12:55 AM, Pawel Jakub Dawidek <pjd_at_freebsd.org> wrote:
> On Thu, Mar 24, 2011 at 01:36:32PM -0700, Freddie Cash wrote:
>> I've tried with FreeBSD 8.2-RELEASE, 8-STABLE, 8-STABLE w/ZFSv28
>> patches, and 9-CURRENT (after the ZFSv28 commit).  Things work well
>> until I start hastd.  Then either the system locks up, or hastd causes
>> a kernel panic, or hastd dumps core.
>
> The minimum amount of information (as always) would be backtrace from
> the kernel and also hastd backtrace when it coredumps. There is really
> decent logging in hast, so I'm also sure it does log something
> interesting on primary or secondary. Another useful thing would be to
> turn on debugging in hast (single -d option for hastd).
>
> The best you can do is to give me the simplest and quickest procedure to
> reproduce the issue, eg. configure two hast resources, put ZFS mirror on
> top, start rsync /usr/src to the file system on top of hast and switch
> roles. The simpler the better.

FreeBSD 8-STABLE r219754 with the ZFSv28 patches applied.

hast.conf:
resource disk-a1 {
    local /dev/label/disk-a1

    on omegadrive {
        remote tcp4://10.20.0.102
    }

    on alphadrive {
        remote tcp4://10.20.0.101
    }
}

resource disk-a2 {
    local /dev/label/disk-a2

    on omegadrive {
        remote tcp4://10.20.0.102
    }

    on alphadrive {
        remote tcp4://10.20.0.101
    }
}

Following will crash hastd:
service hastd onestart
hastctl create disk-a1
hastctl create disk-a2
hastctl role primary all

hastd backtrace is here:
http://www.sd73.bc.ca/downloads/crash/hast-backtrace.png

I'll try running it with -d to see if there's anything interesting there.

Sure, running it with -d and -F, output to a log file, everything
works well using 2 disks.

Hrm, running it with all 24 disks, I can't make it crash now.
However, I did change the kernel hz from 100 to 1000.  I'll see if I
can switch it back to 100 and try the tests again using -dF.

The backtrace listed above is with kern.hz=100.

-- 
Freddie Cash
fjwcash_at_gmail.com