Re: sysctl -a is slow

Lawrence Stewart wrote:
> On 09/21/10 02:21, David Xu wrote:
>   
>> jhell wrote:
>>     
>>> On Mon, 20 Sep 2010 10:26, David Xu wrote:
>>> In Message-Id: <4C976F14.8000408_at_freebsd.org>
>>>
>>>       
>>>> jhell wrote:
>>>>         
>>>>> On 09/19/2010 09:28, David Xu wrote:
>>>>>           
>>>>>> just typed sysctl -a on keyboard, and found it is slow, sometimes
>>>>>> it has been stuck for a few seconds, further studied,I found it is
>>>>>> stucked at sysctl kern.geom:
>>>>>>
>>>>>> %/usr/bin/time sysctl -a kern.geom
>>>>>> kern.geom.collectstats: 1
>>>>>> kern.geom.debugflags: 0
>>>>>> kern.geom.label.debug: 0
>>>>>> kern.geom.label.ext2fs.enable: 1
>>>>>> kern.geom.label.iso9660.enable: 1
>>>>>> kern.geom.label.msdosfs.enable: 1
>>>>>> kern.geom.label.ntfs.enable: 1
>>>>>> kern.geom.label.reiserfs.enable: 1
>>>>>> kern.geom.label.ufs.enable: 1
>>>>>> kern.geom.label.ufsid.enable: 1
>>>>>> kern.geom.label.gptid.enable: 1
>>>>>> kern.geom.label.gpt.enable: 1
>>>>>>        2.01 real         0.00 user         0.00 sys
>>>>>>
>>>>>> it seems it needs more than 2 seconds to complete.
>>>>>>
>>>>>>             
>>>>> A ktrace(1) and a kdump(1) of the resulting ktrace.out file would
>>>>> probably help here along with uname -a. Ive seen this happen once
>>>>> before
>>>>> but do not recall what caused it.
>>>>>
>>>>>
>>>>> Regards & good luck,
>>>>>
>>>>>           
>>>> Result is dumped here.
>>>> http://people.freebsd.org/~davidxu/sysctl_slow.txt
>>>> I think the culprit is sysctl kern.geom.confdot,
>>>> which does not appear in normal output, until I check the kdump result.
>>>> I tried five times, and it was blocked three times.
>>>>
>>>>         
>>> Inspecting the output of sysctl -b kern.geom.confdot will give you
>>> what you currently have configured in the system as disks and what not
>>> through geom. If this seems to be bailing at that point, which is an
>>> opaque MIB/OID which doesn't come up other than when you use the "-o"
>>> switch to sysctl(1) then could you check your labels for your disks
>>> for any weird characters in the labels ?
>>>
>>> ( sysctl -bo kern.geom )
>>>
>>> Also does this have the same effect when run in a xterm, cons25
>>> terminal ?
>>>
>>> And same for the above but with the C, *_COUNTRY.UTF-8 or your normal
>>> locale ?
>>>
>>> ( env LANG=C sysctl kern.geom )
>>>
>>> Looking at the output from mine there are quite a few unprintable
>>> characters present. Maybe these are having an impact with one of your
>>> labels.
>>>
>>>
>>>       
>> I redirect all output to a disk file, and it still needs 1 second to
>> complete, this machine is dual-core pentium E5500, faster than previous
>> one which is a dual-core AMD 5000+ machine, the 5000+ needs 2
>> seconds to complete.
>>
>> $/usr/bin/time sysctl -b kern.geom.confdot > sysctl_geom_confdot.txt
>>         1.00 real         0.00 user         0.00 sys
>>
>> the file is here:
>> http://people.freebsd.org/~davidxu/sysctl_geom_confdot.txt
>>     
>
> As an extra data point, running "/usr/bin/time sysctl -b
> kern.geom.confdot" repeatedly on my amd64 8.1-STABLE desktop varies
> between 0s and 2s. It reports 0 majority of the time but every 5 or so
> runs it'll stall for 1 or 2 seconds. So the problem isn't isolated to head.
>
> Cheers,
> Lawrence
>   

I happened to set kern.sched.preempt_thresh=200, so the kernel is more 
aggressive
than default on thread preemption. it is easier than default to 
reproduce the
problem,  my desktop machine is idle, but  it still stalls 1 or 2 
seconds on the sysctl.