When running pf firewalls and loadbalancers with relayd, some metrics
are critical. One thing that might not be obvious when looking at pf is
that the maximum number of sessions a firewall can handle is fixed, as
evidenced from looking at the output of pfctl -si
Status: Enabled for 10 days 00:39:47 Debug: err State Table Total Rate current entries 6 searches 7970148 9.2/s inserts 60227 0.1/s removals 60221 0.1/s Counters match 67984 0.1/s bad-offset 0 0.0/s fragment 1 0.0/s short 167 0.0/s normalize 0 0.0/s memory 0 0.0/s bad-timestamp 0 0.0/s congestion 0 0.0/s ip-option 0 0.0/s proto-cksum 0 0.0/s state-mismatch 34 0.0/s state-insert 0 0.0/s state-limit 0 0.0/s src-limit 0 0.0/s synproxy 0 0.0/s
Now this can be compared to the limit values you have set which can be
queried through pfctl -sm
states hard limit 200000 src-nodes hard limit 10000 frags hard limit 5000 tables hard limit 1000 table-entries hard limit 200000
In this particular example, the limit won’t be reached for a while!
By default pf has a very low state limit to cope with machines with very
small amounts of RAM, OpenBSD still runs on VAX where 32m of RAM is a
lot. On a default install only 10k states are allowed, for a production
firewall this is likely to be too small really soon. To raise this limit
use the set limit { states }. My advice would be to start at
10000 and to raise appropriately if need be.
Beware that once the state limit is reached, no new states will be allowed to keep old ones functioning leading to a difficult debugging situation.
As a bonus, here is a simple github gist which implements statistics collection for collectd, allowing you to graph state consumption and alert when the limit is nearly reached.