Re: FreeBSD, IPFW and the SIP/VoIP NAT problem

On Tue, 26 Sep 2017 11:27:05 +0200
Guido Falsi <madpilot_at_FreeBSD.org> wrote:

> On 09/26/2017 10:35, O. Hartmann wrote:
> > Hello,
> > 
> > trying to build a FreeBSD based router/PBX (Asterisk 13) appliance, I ran
> > into several problems. My questions might have a "noobish" character, so my
> > apology, my experiences with IPFW are not as thorough as they should be.
> > 
> > Before I'll got into medias res, aquestion about Pine64/AARCH64:
> > 
> > - port net/asterisk13 is supposed to build only on armv6, is aarch64 about
> >   coming soon also supported?   
> 
> I'm maintaining the asterisk ports. At present I don't have any ARM64
> hardware to test it on, but I plan to create an ARM64 jail in poudriere
> so I can try to make it at least build there.

Hello Guido,

I posted by accident the question again to this list as I introduced a typo
when sending it also to the IPFW list. I'm sorry.

I already tried to build net/asterisk13 on my AARCH64 jail, but since I'd like
to have the databases/postgresql96-client aboard and this specific port fails
to build, I gave up - it is, by the way, the only port (pgsql) as far as I
know that fails in my poudriere cross compiling jail. I did not get further,
but I saw that it is supposed to build only for amd64 and armv6.
 
> 
> In such a case would you be willing to test port changes on the hardware
> to actually check it runs?

I'd like to if the efforts are not to much time consuming - I do not have a
working Pine64 image anymore, but I have a Pine64 with 2GB RAM at hand. Months
ago I started playing with cross compiling world/kernel for AARCH64, but I'm
not familiar with crochet and preparing the SD image - but willing to do. But
beware: my home box preparing the cross cimpilation is not the fastest!
 
> 
> > - would a Pine64 running CURRENT (12) be sufficient as a PBX platform
> > (assumed having 2 GB of RAM)?  
> 
> That very much depends on the kind of load you are expecting.
> 
> Asterisk can process a lot of calls. Especially  if it can avoid being
> in the media path.

For the moment, the ARM based PBX should perform SoHo tasks - three, up to ten
lines at maximum. 

> 
> On the other hand if you plan doing a lot of audio transcoding or some
> video transcoding, load can get up quite fast. Compressed codecs like
> the "simple" G729 will make your load grow relatively fast even with
> audio transcoding. Transcoding also lowers call quality so it should
> anyway be avoided as much as possible.

Good to know. But the PBX is more like an experiment for "at home's PBX" and,
if applicable, later for some fellows working scientifically in field and in
need for some small and neat equipement. Video message/streaming is not so much
the focus on the first attempts, but if possible, welcome. if not: not so
tragic.

> 
> Also load can go up if you're doing many disk operations. Monitoring and
> saving audio for a bunch of calls can be quite heavy on disk resources
> AND could require additional transcoding.

There are Linux fellows running Asterisk 13 on raspberry Pi3 very successfully
and this little box has only 1GB RAM as far as I know. Why should FreeBSD fail?

> 
> > 
> > My main concern is about IPFW (we do not use PF for some reasons, I have to
> > stay with IPFW).
> > 
> > I'm a customer of two ITSPs and my SoHo network is behind NAT and not yet
> > IPv6. The FreeBSD system acting as a router is supposed to have a jail soon
> > containing the Asterisk 13 IP PBX (at the moment running on the main
> > system). To provide access to the VoIP infrastructure inside/behind the
> > router/NAT system, the in-kernel NAT facility of FreeBSD is forwarding the
> > relevant UPD/TCP ports for VoIP to its destination network, and here I have
> > a problem to solve.
> > 
> > While it is sumple and easy to forward 5060/udp, 5070/tcp and other ports,
> > it is a mess and pain in the arse to forward a whole range, say 11000/udp -
> > 35000/udp, which is a range one of my providers is sending RTP on. A second
> > provider uses another range for RTP, starting at 5000/udp. So, the logical
> > consequence would be a union set up UDP range to forward, which exapnds then
> > form 5000/udp to 45000/udp - which is much more a pain ...  
> 
> The asterisk project has some suggestions on this here [1]

I know those references as I'm with the problem now for a couple of months ...

> 
> RTP with NAT+FW is a pain. I'm not aware of any IPFW tools able to
> actively inspect SIP packets (which could also be encrypted if using
> SIPS, so there would be no clean way to inspect them).
> 
> Depending on your phone providers you could use a stun and/or turn
> server by enabling the ICE protocol [2], which are all technologies
> supported by asterisk, but require support from your provider. Another
> option is symmetric RTP, which is a trick by creating symmetric port
> numbers connections which sometimes can trick firewalls properly configured.
> 
> You setup also forces you to keep asterisk in the media path
> (direct_media=no in peer configuration)

I know. It is configured that way and it works well with one of my providers,
which has "ingres" calls via 10000/udp through 30000/udp. By simply doing a
forwarding of these portranges in the IPFW rules for the NAT section, my
firewall is open on that range! And this open range grows larger with the
second provider, which has another range on which its RTP communications is
attempting to establish ingress calls. 

> 
> Unluckily none of these technologies is 100% bulletproof.
> 
> RTP is not made to play well with NAT, so the professional solution is
> to spare an IP and redirect almost all ports to the SIP/RTP box. Also
> it's much better to do this with static rules, to avoid load problems on
> the FW (see later).
> 
> 
> You can sidestep the whole issue by running a proxy on your firewall
> machine, if you have control of it. There are a few in the ports tree.
> 
> Take a look at:
> 
> net/kamailio - It is really a SIP proxy, but can parse SIP/SDP packets
> and modify their content on the fly, allowing you to play neat tricks.
> Requires some knowledge of the protocol and work though. (maybe you can
> get him to punch holes in the firewall, but I have not checked if it's
> possible)
> 
> net/rtpproxy: this is more specific and maybe your best bet. Beware of
> the load of proxying RTP in userland though.

Yes, I'm aware of the problemacy regarding NAT and RTP. The problem is the
pinholing of IPFW. 


> 
> 
> > 
> > One of the most disturbing and well known problems is that due to the
> > stateful firewall the RTP session very often is half duplex - it seems one
> > direction  
> 
> Depending on how many simultaneous SIP calls you plan to manage keep an
> eye on your firewall too. each call will create at least 3 states, one
> for SIP and one for each leg of the RTP stream, so it can pile up quite
> fast.
> 
> > of the RTP connection doesn't make it through IPFW/NAT. As often I search
> > the net, I always get informed this is a typical problem and solutions are
> > provided by so called ALGs - since SIP protocol's SDP indicates within the  
> 
> This would require coding it in IPFW, and the load on the firewall could
> be significant.
> 
> It could be done in userland maybe, leveraging divert(4) and having a
> daemon listening there and doing the extra work, but this would be quite
> expensive. Depending on your call volume the load could be too much for
> your firewall.

I thought this specific "divert" could be handled by some ng_XXX thingies of
the netgraph() suite?

As far as I know: egress calls open the IPFW in a stateful manner for the
desired UDP ports (two of them if RTCP is not used, otherwise four, in and out)
and if the prerequisite symmetric rtp is enforced and respected, the "other
side" should send back RTP via the already opened ports.

More complicated is the inbound/ingress direction, since the IPFW doesn't know
about the desired UDP ports for RTP (or TCP, doesn't matter here), so somehow
my(!) PBX has to open the IPFW some ways - and I thought this could be done by
an inspection of SDP. I might be wrong here, since I havn't studied the
protocol in deep. 

Linux folks talk about a "tracking of communications" - if I recall right. It
sounded to me as they have in IPtables some facility doing exactly this
inspecting stuff.

> 
> > payload of the packets on which UDP ports both ends wish to establish their
> > RTP session, it would be "easy" to pinhole the IPFW on exactly those ports
> > for a theoretical large number of sessions, if IPFW could "divert" those
> > packets to an instance inspecting SDP (or whatever is used for the RTP port
> > indication, I'm new to that, sorry for the terminology) and then pinholing
> > the NAT/IPFW for exactly this purpose without the forwarding mess. I came
> > along netgraph() while searching for hints and hooks, but it seems a
> > complete Linux domain, when it somes to appliances like VoIP/IP PBX.
> > 
> > Either, the problem is that trivial on FreeBSD, so no further mentioning is
> > necessary (which would explain the vast emptyness of explanations, hints and
> > so on) or FreeBSD is a complete wasteland on this subject - which I also
> > suspect, since pfSense and OPNsense must have come along with such problems
> > and I simply do not know or recognise the software used for those
> > purposes.  
> 
> I'm not aware of any silver bullets in those products for this.
> 
> > 
> > So, if someone enlightened in this matter stumbles over my question and
> > could delegate me onto the right way (ports, ng_XXX netgraph ficilities to
> > look at, some ipfw techniques relevant to the problem apart from the stupid
> > simple forwarding large ranges of ports) - I'd appreciate this and  
> 
> Maybe you could also get fancy with netflow, and that could have a lower
> load but you'd have to create the IP parsing code yourself.
> 
> Keep in mind that SIP, being used only for signaling, has a relatively
> low cost on being manipulated(obviously it depends on the load of calls
> being managed but it gets hundreds of calls starting and stopping every
> few seconds to create an heavy SIP traffic).
> 
> RTP on the other hand consists on a continuous UDP packet flow, and in
> VoIP it means two for each call, which count directly on the PPS
> (Packets per second) load of you networking equipment, performing
> manipulation on those imposes a constant, heavy per connection load on
> the machines performing it, so you should try to use static FW rules and
> perform no further processing on it unless you have very low call
> volume(let's say that more that 10 through firewall performing extra
> manipulation on RTP streams is starting to be non trivial) or
> appropriate hardware.

All correct and I'd like to go with static rules as far as I can go (on the
other hand, I do not know how IPFW is going to be forced into using dynamic
rulesets on this specific matter ...).

But I think the trivial concern here is the worst case in the RTP/NAT scenario:
how to pinhole (or punchhole?) the IPFW? As I said above, one provider
expects/send RTP from 10000/udp-30000/udp. It is easy to tell asterisk to use
10000-30000 via config rtp.conf. Again: as I understand outgoing connections,
IPFW is opened "statefully" on the desired port and with symmetric RTP, media
data flow on both ends on the same ports. But how is the incoming/ingress case
handled? By forwarding the SIP signalling (and this means, the IPF is
inherently opened for this port), incoming call request contains the desired
RTP ports. In the stateful firewall case, asterisk now should try to contact
the other side exactly via those both ports. But one of those ports is supposed
to be the receiving port, so the asterisk could never establish a stable
outgoing connection - and here is the problem, how to open IPFW (without
ALG/inspecting) - or any stateful firewall? Either my understanding is
completely bullshit or ...
> 
> 
> Hope this all helps.
> 
> 
> [1]
> https://wiki.asterisk.org/wiki/display/AST/Configuring+res_pjsip+to+work+through+NAT
> 
> [2]
> https://wiki.asterisk.org/wiki/display/AST/Interactive+Connectivity+Establishment+%28ICE%29+in+Asterisk
> 


Thank you very much.

Kind regards,

Oliver