Hi Guys, I am in the same boat as you guys are. I am using pf from 7.0-RELEASE FreeBSD 7.0-RELEASE Hardware Specs are IBM x3655 Interfaces: em0 and em1 (intel 1G cards) FW Setup: As Bridge I get very high load on CPU whenever em1 taskq and em0 taskq reaches it peaks. I noticed this when traffic goes to about more than 17MB/s or I can see drops as well on em1(external interface) when packets go over 14K packets/s. Which is a bit slow for this new hardware. But the other thing we setup with PF is Altq. Basically We have queues on both the em0(internal) and em1(externaL) interfaces. May this is another bottleneck? I was just wondering on how you remedied this problem. I am badly in need of solutions. Our internet has been crappy for the past week because when the CPU load goes high it drops heaps of packets. Best Regads, Mark Max Laier wrote: > > On Sunday 27 January 2008, Stefan Lambrev wrote: >> Greetings, >> >> Max Laier wrote: >> >> -cut- >> >> >> Well I think the interesting lines from this experiment are: >> >> max total wait_total count avg >> >> wait_avg cnt_hold cnt_lock name >> >> 39 25328476 70950955 9015860 2 7 >> >> 5854948 6309848 /usr/src/sys/contrib/pf/net/pf.c:6729 (sleep >> >> mutex:pf task mtx) >> >> 936935 10645209 350 50 212904 7 >> >> 110 47 /usr/src/sys/contrib/pf/net/pf.c:980 (sleep >> >> mutex:pf task mtx) >> > >> > Yeah, those two mostly are the culprit, but a quick fix is not really >> > available. You can try to "set timeout interval" to something bigger >> > (e.g. 60 seconds) which will decrease the average hold time of the >> > second lock instance at the cost of increased peak memory usage. >> >> I'll try and this. At least memory doesn't seems to be a problem :) >> >> > I have the ideas how to fix this, but it will take much much more >> > time than I currently have for FreeBSD :-\ In general this requires >> > a bottom up redesign of pf locking and some data structures involved >> > in the state tree handling. >> > >> > The first(=main) lock instance is also far from optimal (i.e. pf is a >> > congestion point in the bridge forwarding path). For this I have >> > also a plan how to make at least state table lookups run in parallel >> > to some extend, but again the lack of free time to spend coding >> > prevents me from doing it at the moment :-\ >> >> Well, now we know where the issue is. The same problem seems to affect >> synproxy state btw. >> Can I expect better performance with IPFW's dynamic rules? > > Not significantly better, I'd predict. IPFW's dynamic rules are also > protected by a single mutex leading to similar congestion problems as pf. > There should be a measureable constant improvement as IPFW does much less > sanity checks. i.e. better performance at the expense of less security. > It really depends on your needs which is better suited for your setup. > >> I wonder how one can protect himself on gigabit network and service >> more then 500pps. >> For example in my test lab I see incoming ~400k packets per second, but >> if I activate PF, >> I see only 130-140k packets per second. Is this expected behavior, if >> PF cannot handle so many packets? > > As you can see from the hwpmc trace starting this thread, we don't spend > that much time in pf. The culprit is the pf task mutext, which forces > serialization in pf congesting the whole forward path. Under different > circumstances pf can handle more pps. > >> The missing 250k+ are not listed as discarded or other errors, which is >> weird. > > As you slow down the forwarding protocols like TCP will automatically slow > down. Unless you have UDP bombs blasting at your network this is quite > usual behavior. > > -- > /"\ Best regards, | mlaier_at_freebsd.org > \ / Max Laier | ICQ #67774661 > X http://pf4freebsd.love2party.net/ | mlaier_at_EFnet > / \ ASCII Ribbon Campaign | Against HTML Mail and News > > > -- View this message in context: http://www.nabble.com/FreeBSD-7%2C-bridge%2C-PF-and-syn-flood-%3D-very-bad-performance-tp15093449p17873869.html Sent from the freebsd-current mailing list archive at Nabble.com.Received on Mon Jun 16 2008 - 20:04:26 UTC
This archive was generated by hypermail 2.4.0 : Wed May 19 2021 - 11:39:32 UTC