the If-Modified-Since header, validated against the Last-Modified of the reply.

Strictly speaking, based on the Mozilla pages you linked, it's validated against the "Last-Modified date of the distant resource", which need not ever have been sent by the server.

First, it's common for syndication feed readers and other things to treat the 'If-Modified-Since' value they provide as a timestamp, not as an opaque string that echoes back your previous Last-Modified.

Well, sure—it's explicitly documented as a timestamp: "The If-Modified-Since request HTTP header makes the request conditional: the server sends back the requested resource, with a 200 status, only if it has been last modified after the given date." It says the date required to be in "GMT", has explicit instructions for parsing and generating it, and notes the risk of the clients and servers having different clocks (a risk that's significantly decreased in the 25 years since publication).

you can change to comparing your Last-Modified value against people's If-Modified-Since as a timestamp

Where would one get the idea it should be treated as an opaque string, and compared by strict equality? RFC 2616 says the server "may" do that, but I think you're interpreting the word "match" too literally; 14.25 is quite explicit that it should be compared as a "less-than"—although that's only a "should", and it's also stated that clients should use the exact date sent by the server, if known.

So, it's explicitly valid, and possibly even useful, to use arbitrary timestamps that were never sent by the server. Maybe I just want to see "what's Chris posted in the last week?", not knowing the exact last time I checked (like if I'm setting up a feed-reader on a new machine and don't want hundreds of old posts appearing). For that matter, it seems valid for clients to "make up" ETag values. Hypothetically, if I told some stateless downloader to save URL X to path Y, where a file already exists, it could hash that file in various ways to generate plausible ETag values and potentially avoid needless re-downloading. I don't, however, see much benefit to a client sending an ETag and timestamp together.

The pragmatic problem with treating the If-Modified-Since value as a timestamp and doing time comparisons with it against your own timestamp is that if you do so, your entire web server environment needs to make absolutely sure that the timestamp of a URL can never go backward, and it's my view that this is very hard to guarantee in practice. If the URL timestamp every goes backward, you will incorrectly tell people that they have the current, correct version of a resource when they don't (unless they gave you an If-None-Match too so you can check a better validator).

(I should have linked to my past entry on this problem in this entry, I just forgot.)

Note that making up a recent If-Modified-Since will not normally limit how many syndication feed entries or the like you get. It's not RFC-compliant to serve you different replies based on the specific values of your If-Modified-Since, so any server that does it would be going well out of its way to guess what a HTTP client means by a random If-Modified-Since. If you make up an If-Modified-Since for a fresh client, either you get nothing (a 304 Not Modified) or you get the full current version of the resource, however big and old it is.

As for ETag (well, If-None-Match), it's not useful for clients to try to make up a value on their own because it is explicitly an opaque server-generated value. Unless you know extremely specific information about how the server generates its ETag values and the server does so only from information that you have access to, you cannot duplicate the server's ETag calculations. If you calculate a hypothetical ETag and it matches the server, there's absolutely no guarantee that you actually have the same content, and in general it's much more likely that it doesn't.

(For example, Apache generates ETag values for static files from stat() information for their inodes. Clients do not know, for example, the inode number of a server side file, so even in the best case you can't duplicate this calculation.)

I agree with you about sending both If-Modified-Since and If-None-Match; if I had my way, everyone would use only If-None-Match. However, tons of actual clients send both (for example, most syndication feed readers that support ETags also send If-Modified-Since). Perhaps part of it has to do with HTTP caches, but I haven't looked into this.

You are correct that I was misremembering how RSS worked, which invalidates my example. Thus, the actual question I could ask is "did Chris post anything within the last week?", which seems quite a bit less useful.

it's not useful for clients to try to make up a[n ETag] value on their own because it is explicitly an opaque server-generated value. Unless you know extremely specific information about how the server generates its ETag values

The servers etc. are almost all open-source, right? And it appears the client can pass multiple ETag values, with only one having to match. So, while I'm not actually suggesting clients do this, it might be an interesting thing to experiment with.

If you calculate a hypothetical ETag and it matches the server, there's absolutely no guarantee that you actually have the same content, and in general it's much more likely that it doesn't.

I'm not following here. If I were to pass an MD5 tag of the existing data (again, just hypothetically), and a SHA-1 and SHA-256 tag, I don't see why those would have any meaningful chance of matching the actual ETag, if the data were different. Unlike the timestamps, these have to match exactly (even for a "weak" match, the server has to recognize the exact string).

If the URL timestamp every goes backward, you will incorrectly tell people that they have the current, correct version of a resource when they don't (unless they gave you an If-None-Match too so you can check a better validator).

Okay, thanks for linking your previous page. I can kind of see your point; but, also, it'll get me on a bit of a rant about the primitive state of URL management. Before I start that, I'll just note that if you don't think you can manage Last-Modified dates, I reckon it's better to omit the header entirely; maybe your instrumentation will support or refute this idea.

The W3C wrote a document long ago (as you may be able to tell from the images being in GIF, and highly aliased—it was 1998) called "Cool URIs don't change". Unfortunately, it seems that they never actually wrote any software to help people manage that, and nor did anyone else; at least nothing that became popular. In my view, the simplistic file-system-based web-serving should've been stopped long ago, except maybe for "toy" sites. Instead, there should be a database—in 2024, I'm thinking maybe something git-based—relating URLs to data hashes (which could double as ETags; weak ETags would require extra tracking). Discontinuing or changing any URL should be a deliberate choice, something very difficult to do by accident; and records should be kept of all such changes.

By contrast, today's "state of the art" seems to be to accept that any URL is bound to break after a few years, and to have the 404 page apologize, mention some recent "re-organization", and maybe link to Google.

Suppose -- half hypothetically :-P -- I'm writing Yet Another Feed Reader. Can I in fact depend on the server providing an ETag? What should I do if there is none? Fall back on timestamps? That's already more complexity than I can tolerate.

I don't think you can count on the server providing an ETag, so as a good feed reader you have to be willing to use the Last-Modified instead. If I was writing a feed reader that was going to do its best against arbitrary and potentially screwed up feed sources, I would send both If-Modified-Since and If-None-Match headers if the server had provided both, because who knows, it could have broken ETag handling but working Last-Modified.

Unfortunately these are the breaks of writing a good feed reader.