ppp with dynamic IPs and ipfw "me"

Hello,

I've a strange effect here when using "ppp -ddial" with my
ISP and my firewall rules.

Sometimes, I get disconnected with my DSL router and ppp
reconnects me in background. This is correct so far, but
I also get a new (dynamic) IP and it seems that ipfw
does not forget my old IP.

I noticed this behavior with the rules which restrict
the communication to the ntpd-port 123.

The rules look like this:
allow udp from xx.xx.xx.xx 123 to me dst-port 123
allow udp from me 123 to xx.xx.xx.xx dst-port 123

After I get a new IP, the logs show that the communication
from "me" to xx.xx.xx.xx is blocked.

I'm on DSL here (using PPPoE and NAT).

Is this a bug or should I update the rules myself?
If yes, what is the best way to update the rules?

Martin