== A few small notes about OpenBSD PF (as of 4.4 and 5.1)

Suppose that you read the _pf.conf_ manpage (in OpenBSD 4.4 or 5.1) and
stumble across the following:

> - _max-src-conn <number>_: Limits the maximum number of simultaneous
>   TCP connections which have completed the 3-way handshake that
>   a single host can make.

Great, you say, this is just what you need to make sure that bad people
are not holding too many connections to your web server open at once. So
you write a PF rule more or less like this:

.pn prewrap on
>  table <BRUTES> persist
>  block quick log on $EXT_IF proto tcp from <BRUTES> to any port 80
>  pass in quick on $EXT_IF proto tcp from any to any port 80 \
>       keep state \
>       (max-src-conn 20, overload <BRUTES> flush)

Shortly after you activate this rule you may discover an
ever-increasing number of web crawler IPs listed in your _BRUTES_ table,
which will probably surprise you. What is going on is that the OpenBSD
manpage is misleading you.
_max-src-conn_ does not limit the number of concurrent TCP connections.
It limits *the number of state table entries* for TCP connections that
have been fully established. If you examine the state tables as a web
crawler is walking your site, you will discover any number of entries
sitting around in ((FIN_WAIT_2)). These connections are thoroughly
closed but, guess what, they count against _max-src-conn_ until they
expire completely.

An extremely technical reading of the wording of the _pf.conf_ manpage
might lead you to a claim that this is allowed by the manpage (if you
say that a TCP connection still exists in _``FIN_WAIT_2''_), but at the
least I think this is going to surprise almost everyone. It also renders
this _max-src-conn_ rule useless in limiting the number of concurrent
real TCP connections. Given that states linger in ((FIN_WAIT_2)) for
on the order of a minute or more, there is no feasible setting for
_max-src-conn_ that will allow a crawler to make one or two requests a
second without getting blocked while also giving you a useful concurrent
connections limit.

(This almost certainly applies to _max-src-states_ too, but at least
that is explicitly documented in terms of state table entries.)

But wait, the fun isn't done yet. You decide that you really need to
limit the number of concurrent real TCP connections. You don't really
care if stray out of sequence packets from fully closed connections
get rejected by the firewall (they'd only get rejected by the host
anyways), so the obvious solution is to set a very fast timeout for
those lingering ((FIN_WAIT_2)) states. You read the fine _pf.conf_
manpage again and spot some timeout settings (which can be either global
or per-state-creating-rule):

> - _tcp.closed_: The state after one endpoint sends an RST.
> - _tcp.finwait_: The state after both FINs have been exchanged and
>   connection is closed. [...]

There is no pleasant way to put this: the _pf.conf_ manpage is lying to
you. Setting _tcp.finwait_ to a very low value will do exactly nothing
to help you; you need to set _tcp.closed_. The state timeouts are
actually:

|_. _tcp.closed_	| Both sides in ((FIN_WAIT_2)) or ((TIME_WAIT)).
| _tcp.finwait_		| Both sides in ((CLOSING)), or one side ((CLOSING))
			  and the other side has progressed a bit further.
| _tcp.closing_		| One but not both sides in ((CLOSING)), ie a _FIN_
			  has been sent.
| _tcp.established_	| Both sides ((ESTABLISHED)).
| _tcp.opening_		| At least one side not ((ESTABLISHED)) yet.

(All of this is expressed in terms of what '_pfctl -ss_' will print
as the states. There are a few intermediate transient states that may
show up which I am eliding because my head hurts. See the logic in
_sys/net/pf.c_ and the list of states in ((sys/netinet/tcp_fsm.h)) if
you really care.)

The manpage is partly technically correct in that after an _RST_ is
sent, PF puts the state into ((TIME_WAIT)) and _tcp.closed_ applies.
This is also the only time that a state winds up in ((TIME_WAIT)).

(I have verified this behavior on OpenBSD 4.4. I have not verified
the behavior on OpenBSD 5.1 but the _sys/net/pf.c_ code involved is
basically the same and reads just the same as the 4.4 version; in fact
my table above is generated by reading the 5.1 _pf.c_ source code (and
my manpage quotes are from the 5.1 manpages). I have not looked at 5.2
source or manpages.)