Re: ufs multilabel performance (fwd)

On Apr 15, 2012, at 12:30 PM, O. Hartmann wrote:

> Am 04/15/12 15:59, schrieb Richard Kojedzinszky:
>> Thank you for the reply.
>> 
>> Unfortunately, dont know why, but on my xen virtualised environment,
>> fbsd amd64 domU performs much slower, not only 30 times. Without
>> multilabel, file creation speed is around 2500/s, but with multilabels
>> enabled, it is only 15/s (!). so it is more than 100 times slower.
>> 
>> And anyway freebsd is known to be fast as well, as functional. The power
>> to serve. :)
>> 
>> But in my environment, 15/s file creation is very-very slow. The
>> hardware is a q6700 cpu with 4G ram, 2x1T sata disks in raid1, the host
>> runs linux. I think with this hw the mentioned speed is really slow.
>> 
>> Regards,
>> 
>> 
>> Kojedzinszky Richard
>> Euronet Magyarorszag Informatikai Zrt.
>> 
>> On Sun, 15 Apr 2012, O. Hartmann wrote:
>> 
>>> Date: Sun, 15 Apr 2012 13:20:23 +0200
>>> From: O. Hartmann <ohartman_at_zedat.fu-berlin.de>
>>> To: Richard Kojedzinszky <krichy_at_tvnetwork.hu>
>>> Cc: freebsd-security_at_freebsd.org
>>> Subject: Re: ufs multilabel performance (fwd)
>>> 
>>> Am 04/14/12 21:37, schrieb Richard Kojedzinszky:
>>>> Dear list,
>>>> 
>>>> Although it is not only security-related question, I did not get any
>>>> answer from freebsd-performance. The original question is below.
>>>> 
>>>> Can someone give some advice?
>>>> 
>>>> Thanks in advance,
>>>> 
>>>> 
>>>> Kojedzinszky Richard
>>>> Euronet Magyarorszag Informatikai Zrt.
>>>> 
>>>> ---------- Forwarded message ----------
>>>> Date: Thu, 10 Nov 2011 06:16:57 +0100 (CET)
>>>> From: Richard Kojedzinszky <krichy_at_tvnetwork.hu>
>>>> To: freebsd-performance_at_freebsd.org
>>>> Subject: ufs multilabel performance
>>>> 
>>>> Dear List,
>>>> 
>>>> I've noticed that when I enable multilabel on an fs, a file creation
>>>> gets around 20-30 times slower than without multilabel set.
>>>> 
>>>> This one-liner can be used to test the differences:
>>>> $ truss -D perl -e 'open(F, ">$_.file") for 1 .. 1000'
>>> 
>>> Same here, creating files seems to be 10 - 30 times slower with
>>> multilabels as it is without.
>>> 
>>> But as several posts and discussions reflects, FreeBSD isn't supposed to
>>> be fast although it is claimed that writing is the major than reading;
>>> FBSD should serve functionality.
>>>> 
>>>> And one can see that the open call takes much more when multilabel is
>>>> set on an fs. It seems that only file creation needs that many time,
>>>> when a file exists it is opened much faster.
>>>> 
>>>> Could someone acknowledge this, and have some suggestions how to make it
>>>> faster?
>>>> 
>>>> Regards,
>>>> 
>>>> 
>>>> Kojedzinszky Richard
>>>> TvNetWork Nyrt.
>>>> E-mail: krichy (at) tvnetwork [dot] hu
>>>> PGP: 0x54B2BF0C8F59B1B7
>>>>  Fingerprint = F6D4 3FFE AF03 CACF 0DCB  46A1 54B2 BF0C 8F59 B1B7
> 
> At the moment, I'm troubled with a nasty kernel bug on all FreeBSD 10
> boxes I have spare to test.
> 
> I just tried to reproduce your observation and as far as I can go with
> my experience, I can confirm that by using your perl script.
> 
> I'd like to test this again with a small C program.
> 
> I can only test the issue (test is too far optimistic, it's simply a
> reproduction of your observation) on FreeBSD 10, the only remaining
> FreeBSD server at our department is running FBSD 9-STABLE/amd64 and "in
> production", so changing multilabel support is a bit harsh at the moment.
> 
> 
> Sorry about crossposting, but I think this belongs more to CURRENT and
> PERFORMANCE than SECURITY.

My suggestion is completely take perl out of the equation because the way you're invoking it above uses stdio and a few other things that add unnecessary overhead.

Try the attached C program/bourne shell snippet instead.

Cheers,
-Garrett

#!/bin/sh

set -e

tmp=$(mktemp -d tmp.XXXXXX)
trap "cd /; rm -Rf $tmp" EXIT
cd $tmp

cat > test_open.c <<EOF

#include <fcntl.h>
#include <stdio.h>
#include <unistd.h>

int
main(void)
{
        char buf[20];
        int i;

        for (i = 0; i < 1000; i++) {
                sprintf(buf, "%d", i);
                close(open(buf, O_WRONLY|O_CREAT, 0600));
        }
        return (0);
}
EOF

gcc -o test_open test_open.c
time ./test_open