Wandering Thoughts archives


You should keep your system logs for longer than you probably are

One simple thing you can do to improve your life is to make your machines keep their logs for longer than they currently do. Most systems ship with relatively short log retention defaults that basically date from the days when systems had what are now very small disks and sysadmins got really grumpy about logs eating up lots of scarce disk space. Those days are over now for most systems; for example our new servers come with 500 GB HDs as the default. A 500 GB disk will hold really quite a lot of logs. SSDs change this a bit, but even small SSDs these days are in the 64 to 80 GB range and you usually have to work hard to get a system install to use more than a few GB. Even on SSDs we wind up with tens of GB free.

(Of course this goes well with having a central syslog server, because usually you can easily give the central syslog server a lot of disk space to store lots of logs. This isn't true in big environments where you have a lot of log traffic in the aggregate, but most sysadmins are not in such environments.)

The core reason to keep logs for a relatively long time is that you don't always find out about things that you want to look into right away. The longer you keep logs for, the further you can look back into history to see things. The obvious case where this is really important is if you ever experience a system compromise or security problem that you didn't detect immediately. But you can also be looking back to see how frequent something is, or even doing long term historical analysis on how things have changed over time.

Having said that, there are some concerns involved if you're thinking of doing this. We're lucky enough to be in a situation without real concerns about information sensitivity and anti-retention policies. For information sensitivity, we don't have any really sensitive logs that we have to closely safeguard and we consider all of our machines about as secure as each other.

(Of course I am a big fan of not logging sensitive information you don't need.)

Once you've made the decision to keep your logs for a relatively long time, there are of course a bunch of things you can do to improve the situation even more. The obvious ones are centralizing your logs and setting up a long-term archival system for them so that if need be you can go back really extended periods of time. If you do periodic archival system backups, for example, you can make sure that your log storage is captured in the backups and that you keep enough logs to cover at least the full time interval between those archival backups.

(This elaborates on a tweet of mine.)

sysadmin/KeepLogsLonger written at 22:10:24; Add Comment

TLS versions in connections to my spam-catching sinkhole SMTP server

I've written before about TLS usage on our real inbound mail gateway and the general TLS breakdown on my sinkhole SMTP server. My sinkhole server didn't initially log the TLS version used, but after Heartbleed hit I changed that because it was now interesting information, and here's a preliminary report. Note that, unlike our main mail gateway, this is all spam sending attempts.

The basic version breakdown is that out of 588 connections that negotiated TLS since I added this logging, 102 used SSL v3, 151 used TLS v1.0, only seven used TLS v1.1, and 328 used TLS v1.2. I looked through the seven by hand and there's no particular pattern. Those connections resulted in only 153 actual message submissions, and the breakdown there is 29 SSL v3, 57 TLS v1.0, 2 TLS v1.1, and 65 with TLS v1.2.

I looked through the email received using SSL v3, and almost all of it is basic advanced fee fraud or phishing spam. The first interesting exception is spam from gallopade.nmsrv.com, which is a name that has been trying to spam me for quite a long time. The other interesting exception is genuine email from Twitter's 'please confirm your new account' system and that apparently comes from an account creation initiated by some sort of spammer. Both messages received using TLS v1.1 were also advanced fee fraud emails, although these seem to come from real ISP systems instead of random and probably ancient mail servers around the Internet.

I did a random spot sampling of the messages received with TLS v1.0 and couldn't spot any particular pattern in the mailers involved; connecting to people's SMTP ports turned up Exim, Sendmail, and 'Microsoft ESMTP MAIL Service' aka Microsoft Mail Server before I got bored. Microsoft was clearly the most popular, followed by Exim (although the Exim versions in SMTP greeting banners varied). Interestingly, both GMail and Yahoo used TLS v1.0 at least once.

The prevalence of SSL v3 and some other things in my sinkhole connections goes along with what I've long thought about the machines exploited to send phish and advance fee fraud, which is that many of them are basically neglected. Old, neglected machines are quite likely to be running old software that only supports old versions of TLS.

spam/SinkholeTLSUsage2014-11 written at 03:04:42; Add Comment

Page tools: See As Normal.
Login: Password:
Atom Syndication: Recent Pages, Recent Comments.

This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.