The effects of a moderate Hacker News link to here

November 22, 2014

A few days ago my entry on Intel screwing up their DC S3500 SSDs was posted to Hacker News here and rose moderately highly up the rankings, although I don't think it made the front page (I saw it on the second page at one point). Fulfilling an old promise, here's a report of what the resulting traffic volume looked like.

First, some crude numbers from this Monday onwards for HTTP requests for Wandering Thoughts, excluding Atom feed requests. As a simple measurement of how many new people visited, I've counted unique IPs fetching my CSS file. So the numbers:

(day) (that entry) (other pages) (CSS fetches)
November 17 0 5041 453
November 18 18255 6178 13585
November 19 17112 10141 11940
November 20 908 6341 876
November 21 228 4811 530

(Some amount of my regular traffic is robots and some of it is from regular visitors who already have my CSS file cached and don't re-fetch it.)

Right away I can say that it looks like people spilled over from the directly linked entry to other parts of Wandering Thoughts. The logs suggest that this mostly went to the blog's main page and my entry on our OmniOS fileservers, which was linked to in the entry (much less traffic went to my entry explaining why 4K disks can be a problem for ZFS). Traffic for the immediately preceding and following entries also went up, pretty much as I'd expect, but this is nowhere near all of the extra traffic so people clearly did explore around Wandering Thoughts to some extent.

Per-day request breakdowns are less interesting for load than per minute or even per second breakdowns. At peak activity, I was seeing six to nine requests for the entry per second and I hit 150 requests for it a minute (for only one minute). The activity peak came very shortly after I started getting any significant volume of hits; things start heating up around 18:24 on the 18th, go over 100 views a minute at 18:40, peak at 19:03, and then by 20:00 or so I'm back down to 50 a minute. Unfortunately I don't have latency figures for DWiki so I don't know for sure how well it responded while under this load.

(Total page views on the blog go higher than this but track the same activity curve. CSpace as a whole was over 100 requests a minute by 18:39 and peaked at 167 requests at 19:05.)

The most surprising thing to me is the amount of extra traffic to things other than that entry on the 19th. Before this happened I would have (and did) predict a much more concentrated load profile, with almost all of the traffic going to the directly linked entry. This is certainly the initial pattern on the 18th, but then something clearly changed.

(I was surprised by the total amount of traffic and how many people seem to have visited but that's just on a personal basis where it's surprising for so many people to be interested in looking at something I've written.)

This set of stats may well still leave people with questions. If so, let me know and I'll see if I can answer them. Right now I've stared at enough Apache logs for one day and I've run out of things to say, so I'm stopping this entry here.

Sidebar: HTTP Referers

HTTP Referers for that entry over the 18th to the 20th are kind of interesting. There were 17,508 requests with an empty Referer, 13,908 from the HTTPS Hacker News front page, 592 from a redirector of some sort, 314 from the link in this HN repeater tweet, and then we're down to a longer tail (including reddit's /r/sysadmin, where it was also posted). The Referers feature a bunch of various alternate interfaces and apps for Hacker News and so on ( was surprisingly popular). Note that there were basically no Referers from any Hacker News page except the front page, despite that as far as I know the story never made it to the front page. I don't have an explanation for this.

Written on 22 November 2014.
« Lisp and data structures: one reason it hasn't attracted me
I'm happier ignoring the world of spam and anti-spam »

Page tools: View Source, Add Comment.
Login: Password:
Atom Syndication: Recent Comments.

Last modified: Sat Nov 22 00:46:51 2014
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.