The HTTP Referer header is fading away (at least as a useful thing)
Referer header on
requests is famously misspelled (it should be
Referrer), and also
famously not liked because of privacy and security concerns.
The privacy and security concerns are especially strong with external
('cross-origin') Referers, which is also the ones that many people
find most useful because they tell you where visitors to your pages
are coming from and let you find places where people have linked
to you or are mentioning you.
I've been reading my Referer logs for essentially as long as Wandering Thoughts has existed, and over the years (and especially lately) it's become clear to me that the Referer header is fading away. Fewer requests have Referer headers, and many of the values that are there aren't all that useful (at least to me). Some of this is the general issue of social media and web applications, where most everything from a place like Twitter either has 'https://twitter.com/' (if the person came from web twitter) or nothing (if they came from a Twitter client). Others seem to be specific choices made by sites. For example, a lot of search engines now arrange things so that the Referer you see is their main URL and doesn't have any information on what the person searched for that led them to your pages.
(Probably an increasing number of people are also using browser extensions that block or spoof Referer, although I don't know if this is common.)
Referer is clearly going to fade away more in the future. This
effort started with the
which gave web server operators a simple way to mostly control the
outbound Referer from any links on their web pages (without having
to touch the HTML). Now the browsers are in the process of moving
to a stricter default behavior, called 'strict-origin-when-cross-origin';
this sends only the origin (the website), omitting the path and the
query string. A switch to this default would make almost all websites
behave the way that Twitter and some search engines do (although
for different reasons).
In theory web sites could set a
Referrer-Policy to revert back
to the current state of affairs once the browser default changes.
In practice most websites will never change the default (and many
of the ones that do might make it stricter, likely going to
'same-origin'). And someday browsers will probably ratchet privacy
and security one step further, so that by default they only send
Referer headers to the same site.
When that happens, I'll definitely miss the old days when I could see where my visitors were coming from. Discovering various articles and people who've linked to my entries has definitely made me happy and sometimes given me valuable information and feedback. But the arrow of time on the web points to more privacy, and there's not much we can do about that; the bad actors vastly outnumber the good ones.