Re: file descriptor leak in 5.2-RC

From: Oliver Brandmueller <ob_at_e-Gitt.NET>
Date: Thu, 25 Dec 2003 22:58:38 +0100
Hello again.

On Thu, Dec 25, 2003 at 09:46:26PM +0100, Oliver Brandmueller wrote:
> - No Linuxulator.
> - Tried different Threading libs.
> - I do not have any unusual exits (since upgrading the RAM to a 
>   resonable high size).

Forgot something: I have no /etc/libmap.conf - I installed a libmap.conf 
that changed libc_r to libkse during one test (it was correctly parsed 
and used as I saw with clamd). I saw no difference.

> I saw in high traffic times about 6000-8000 kern.openfiles after 10-12 
> minutes of filtering, that sums up to about 10 per second. I will count 
> the number of mails fed through the system next time I try and will see 
> if I can see any relation (same number, twice or unrelated). I can do 
> the next test within the next few hours.

# sysctl kern.openfiles
kern.openfiles: 3473

# fgrep -c 'R=amavis' /var/log/exim/mainlog
1715

# fgrep -c 'H=localhost' /var/log/exim/mainlog
3463

Here is the interesting point!

fstat shows for sh and init (with out itself) 10 file
descriptors.

3463 + 10 = 3473

This means for every localhost connect related to amavisd we have one 
leaking file descriptor!

amavisd listens on localhost port 10024 and feeds everything back to
exim on port 10025 in this setup. Since exim gets connects on other
ports also and the number is exactly equal to the number of connects to
and from amavisd, it's very unlikely that it's related to exim. Since 
amavisd feeds every mail to clamd (via socket) only once and seems to 
use SpamAssassins libs directly, it's very likely in the way, amavisd 
handles the connections to/from exim.

amavisd (perl script) uses - at least for outgoing SMTP - standard perl
modules, in this Net::SMTP which it self includes "use Socket 1.3" and
also "use IO::Socket".

Most likely I'd say that perl 5.6.1 plus something in FBSD 5 together 
produce this leaking of file descriptors. I will take some time and 
consider updating to perl5.8 to see if it still happens.

The problem seems - as far as I can tell - not related to the threading 
then. On one side I used different rhreading libs, on the other I think 
perl 5.6.1 does not use threading at all and I could not find anything 
that looks like that in amavisd (in fact it spawns simply a whole bunch 
of daemons).

Since Michal Mertl detected the problem together with Mozilla as he 
wrote, it seems not to be related to perl only - which would be bad 
enough, but more a general problem somewhere.


- Oliver


-- 
| Oliver Brandmueller | Offenbacher Str. 1  | Germany       D-14197 Berlin |
| Fon +49-172-3130856 | Fax +49-172-3145027 | WWW:   http://the.addict.de/ |
|               Ich bin das Internet. Sowahr ich Gott helfe.               |
| Eine gewerbliche Nutzung aller enthaltenen Adressen ist nicht gestattet! |
Received on Thu Dec 25 2003 - 12:58:41 UTC

This archive was generated by hypermail 2.4.0 : Wed May 19 2021 - 11:37:35 UTC