Re: Runaway intr, not flash related

on 19/08/2010 20:30 Doug Barton said the following:
> On 08/19/2010 08:24, Andriy Gapon wrote:
>> I am sorry, but I don't see anything dramatically wrong here. So
>> "swi4: clock" uses 5.76% of WCPU, is that such a big deal to be
>> called "runaway intr"?
> 
> That's the symptom.

OK, I see.

Perhaps you will find this message (and its ancestor thread) interesting:
http://lists.freebsd.org/pipermail/freebsd-hackers/2008-February/023447.html
I believe that your issue is different, but perhaps that stuff will inspire you to
use ktr(4) and schedgraph to properly debug this issue.  I strongly believe that
you have some sort of a scheduling issue and ktr seems to be the way to
investigate it.

Perhaps, you can first try the following dtrace script.
It should give a better view of what statclock sees, but I am not sure if that
information will be sufficient.
/********************************************************/
fbt::statclock:entry
/curthread->td_oncpu == 0/
{

	_at_stacks0[stack()] = count();
	counts0++;
}

fbt::statclock:entry
/curthread->td_oncpu == 1/
{

	_at_stacks1[stack()] = count();
	counts1++;
}

fbt::statclock:entry
{

	_at_stacks[pid, tid, stack()] = count();
	counts++;
}

END
{
	printf("\n");
	printf("***** CPU 0:\n");
	normalize(_at_stacks0, counts0 / 100);
	trunc(_at_stacks0, 5);
	printa("%k%_at_u\n\n", _at_stacks0);

	printf("\n\n");
	printf("***** CPU 1:\n");
	normalize(_at_stacks1, counts1 / 100);
	trunc(_at_stacks1, 5);
	printa("%k%_at_u\n\n", _at_stacks1);

	printf("\n\n");
	printf("***** Top Processes:\n");
	normalize(_at_stacks, counts / 200);
	trunc(_at_stacks, 20);
	printa(_at_stacks);
}
/********************************************************/
You would run this script when the problem hits, few seconds should be sufficient.
You may want to play with values in trunc() calls, you may also want to filter
gathered statistics (using conditions in /.../) by pid/tid if you spot anything
interesting unusual.

-- 
Andriy Gapon