2024-02-22
A recent abrupt change in Internet SSH brute force attacks against us
It's general wisdom in the sysadmin community that if you expose a SSH port to the Internet, people will show up to poke at it, and by 'people' I mean 'attackers that are probably mostly automated'. For several years, the pattern to this that I've noticed was an apparent combination of two activities. There was a constant background pitter-patter of various IPs each making a probe once a minute or less (but for tens of minutes or longer), and then periodic bursts where a single IP would be more active, sometimes significantly so.
(Although I can't be sure, I think the rate of both the background probes and the periodic bursts was significantly up compared to how it was a couple of years ago. Unfortunately making direct comparisons is a bit difficult due to Grafana Loki issues.)
Then there came this past Tuesday, and I noticed something that I reported on the Fediverse:
This is my system administrator's "what is wrong" face when Internet ssh authentication probes against our systems seem to have fallen off a cliff, as reported by system logs. We shouldn't be seeing only two in the last hour.
(The nose dive seems to have started at 6:30 am Eastern and hit 'basically nothing' by 9:30 am.)
After looking at this longer, the pattern I'm now seeing on our systems is basically that the background low-volume probes seem to have gone away. Every so often some attacker will fire up a serious bulk probe, making (for example) 400 attempts over a half an hour (often for a random assortment of nonexistent logins); rarely there will be a burst where a dozen IPs will each make an attempt or two and then stop (there's some signs that a lot of the IPs are Tor exit nodes). But for a lot of the time, there's nothing. We can go an hour or three with absolutely no probes at all, which never used to happen; previously a typical baseline rate of probes was around a hundred an hour.
Since the higher-rate SSH probes get through fine, this doesn't seem to be anything in our firewalls or local configurations (I initially wondered about things like a change in logging that came in with an Ubuntu package update). Instead it seems to be a change in attacker behavior, and since it took about two hours to take full effect on Tuesday morning, I wonder if it was something getting progressively shut down or reoriented.