Why our Grafana URLs always require HTTP Basic Authentication
As part of our new metrics and monitoring setup, we have a Grafana server for our dashboards that sits behind an Apache reverse proxy. The Apache server also acts as a reverse proxy for several other things, all of which live behind the same website under different URLs.
People here would like to be able to directly access our Grafana dashboards from the outside world without having to bring up a VPN or the like. We're not comfortable with exposing Grafana or our dashboards to the unrestricted Internet, so that external access needs to be limited and authenticated. As usual, we've used our standard approach of Apache HTTP Basic Authentication, restricting the list of users to system staff.
Now, having to authenticate all of the time to see dashboards is annoying, so it would be nice to offer basic anonymous access to Grafana for people who are on our inside networks (and Grafana itself supports anonymous access). Apache can support this in combination with HTTP Basic Authentication; you just use a RequireAny block. Here's an example:
<Location ...> AuthType Basic [...] <RequireAny> Require ip 127.0.0.0/8 Require ip 18.104.22.168/24 [...] Require valid-user </RequireAny> </Location>
People outside the listed networks will be forced to use Basic Auth; people on them get anonymous access.
It's also useful for system staff to have accounts in Grafana itself, because having a Grafana account means you can build your own dashboards and maybe even edit our existing ones (or share your dashboards with other staff members and edit them and so on). Grafana supports a number of ways of doing this, including local in-Grafana accounts with separate passwords, LDAP authentication, and HTTP Basic Authentication. For obvious reasons, we don't want people to have to obtain and manage separate Grafana accounts (it would be a pain in the rear for everyone). Since we're already using HTTP Basic Authentication to control some access to Grafana, reusing that for Grafana accounts makes a lot of sense; for instance, if you're accessing the server from the outside, it means that you don't have to first authenticate to Apache and then log in to Grafana if you want non-anonymous access.
But this hypothetical setup leaves us with a problem: how do you log in to Grafana when you're on our inside networks, where you won't be required to use HTTP Basic Authentication? It would be a terrible experience if you could only use your Grafana account if you weren't at work.
Before I set the server up and started experimenting, what I was
hoping was that HTTP Basic Authentication was treated somewhat like
cookies, in that once a browser was challenged to authenticate, it
would then send the relevant
Authorization header on all further
accesses to the entire website. There are other areas of our web
server that always require HTTP Basic Authentication, even from our
internal networks, so if Basic Auth worked like cookies, you could
go to one of them to force Basic Auth on, then go to a Grafana URL
and the browser would automatically send an
and Apache would pass it to Grafana and Grafana would have you
logged in to your account.
Unfortunately browsers do not treat HTTP Basic Authentication this
way, which is not really surprising since RFC 7617 recommends a
different approach in section 2.2. What RFC 7617
recommends and what I believe browsers do is that HTTP Basic
Authentication is scoped to a URL path on the server. Browsers will
only preemptively send the
Authorization header to things in the
same directory or under it; they won't send it to other, unrelated
(If a browser gets a '401 Unauthorized' reply that asks for a realm that the browser knows the authorization for, it will automatically retry with that authorization. But then you're requiring HTTP Basic Authentication in general.)
The simplest, least hacky way out of this for us is to give up on the idea of anonymous access to Grafana, so that's what we've done. And that is why access to our Grafana URLs always requires HTTP Basic Authentication, however somewhat inconvenient and annoying it is. We have to always require HTTP Basic Authentication so that people can have and readily use frictionless Grafana accounts.
(As I mentioned in my entry on why we like Apache HTTP Basic Authentication, we're not willing to trust the authentication of requests from the Internet to Grafana itself. There are too many things that could go wrong even if Grafana was using, say, a LDAP backend. Fundamentally Grafana is not security software; it's written by people for whom security and authentication is secondary to dashboards and graphs.)
Sidebar: The theoretical hack around this
In theory, if browsers behave as RFC 7617 suggests, we can get
around this with a hack. The most straightforward way is to have a
web page at the root of the web server that we've specifically
configured to require HTTP Basic Authentication; call this page
/login.html. When you visit this page and get challenged, in
theory your browser will decide that the scope of the authentication
is the entire web server and thus send the
on all further requests to the server, including to Grafana URLs.
However I'm not sure this actually works in all common browsers (I haven't tested it) and it feels like a fragile and hard to explain thing. 'Go to this unrelated URL to log in to your Grafana account' just sounds wrong. 'You always have to use HTTP Basic Authentication' is at least a lot simpler.