Re: Automated submission of kernel panic reports: sysutils/panicmail

On Monday, November 04, 2013 4:29:27 pm Colin Percival wrote:
> On 11/04/13 04:49, Alfred Perlstein wrote:
> > Colin, have you had a few minutes to check out the crash reporting facilities in
> > FreeNAS?
> 
> Yes.
> 
> > The reason I ask is that:
> > 
> > 1) we would like to share code.
> > 2) we have this running for a few months now and have a huge corpus of information.
> > 3) we are building a nice UI (screenshots attached) over it, we have a couple of
> > thousands of lines of code we can share for this.
> 
> Once I have a useful number of panics collected, I was hoping to take the best
> pieces from FreeNAS's processing, from the SoC project, and from the processing
> I've been doing of automatic panic reports from EC2 instances.
> 
> > We send a minimal set of information: kernel stack trace, ddb buffer and
> > hardware.  Just enough to get some very, very handy stuff.
> 
> I'm currently sending the dump header and what I get from kgdb 'bt'.  If I find
> that I'm missing something important, I can always add it to a new version of
> the panicmail port. ;-)

One of my previous employers maintained a database of panics and I added ways
to recognize "known panics" and tag them.  I ended up relying a lot on stack
trace details from specific OS versions to mark a panic as an instance of a
specific bug.  Also, you may have very different stack traces even on the same
build version for a single bug.  In the case of my employer we had a
constrained set of kernel configs and specific build versions to work with.
It might be harder to correctly match panics in the wild what with patched
trees and random kernel configs.

-- 
John Baldwin