Re: performance of jailed processes

On Tue, 30 Mar 2004, Dag-Erling Smørgrav wrote:

> Robert Watson <rwatson_at_freebsd.org> writes:
> > - DNS -- I know you mentioned it, but I'd check anyway.  Especially if
> >   resolv.conf has bad DNS servers in it in the jails, etc.  You might try
> >   writing a trivial gethostbyname() test app and timing it in and out of
> >   the jail.  Also look at the reverse lookup done by the MySQL server.
> >   The impact of the source IP address might be particularly interesting.
> 
> Packet traces already show that there is no delay between query and
> reply, the reply just takes a long time to transmit. 

Somewhat more painful suggestion, but could you generate ktraces against a
mysql client doing the query inside and out of jail, then using whatever
flag sets relative timestamps on kdump, diff the two and see where the
substantial differences begin?

> > - It would be interesting to know if applications outside the jail bound
> >   to various IP addresses see performance differences depending on the IP
> >   used.  We have hashed IP address lookup, but there are some operations
> >   in the stack that require walking the list of addresses, etc.  If the
> >   non-jailed software always uses the first address because they're all in
> >   the same subnet, that might conceivably make a difference.  Taking jail
> >   out of the picture in some basic micro-benchmarks might help here also. 
> 
> Non-jailed software always uses the first IP address, which is in its
> own subnet.  The jails draw from a pool of ~1000 IP addresses on the
> same interface, but in a different subnet.  The jail I've been testing
> in is about a quarter of the way down the list. 
> 
> > Can you identify any micro-benchmarks rather than macro-benchmarks that
> > reflect a significant difference?
> 
> haven't had much luck with that...  fetch, for instance, doesn't seem
> to suffer, but with mysql the difference is dramatic:
> 
> (outside jail)
> 1 row in set (0.01 sec)
> 
> (inside jail)
> 1 row in set (13.20 sec)
> 
> note that 13 seconds is far too short for a DNS issue, and that the time
> reported is measured *after* login (i.e. after any DNS lookup) 

13 seconds is too long for most of the potential things I have in mind...

Robert N M Watson             FreeBSD Core Team, TrustedBSD Projects
robert_at_fledge.watson.org      Senior Research Scientist, McAfee Research