Chris's Wiki :: blog/sysadmin/SSLChasingCertChains

Chasing SSL certificate chains to build a chain file

April 14, 2014

Supposes that you have some shiny new SSL certificates for some reason. These new certificates need a chain of intermediate certificates in order to work with everything, but for some reason you don't have the right set. In ideal circumstances you'll be able to easily find the right intermediate certificates on your SSL CA's website and won't need the rest of this entry.

Okay, let's assume that your SSL CA's website is an unhelpful swamp pit. Fortunately all is not lost, because these days at least some SSL certificates come with the information needed to find the intermediate certificates. First we need to dump out our certificate, following my OpenSSL basics:

openssl x509 -text -noout -in WHAT.crt

This will print out a bunch of information. If you're in luck (or possibly always), down at the bottom there will be a 'Authority Information Access' section with a 'CA Issuers - URI' bit. That is the URL of the next certificate up the chain, so we fetch it:

wget <SOME-URL>.crt

(In case it's not obvious: for this purpose you don't have to worry if this URL is being fetched over HTTP instead of HTTPS. Either your certificate is signed by this public key or it isn't.)

Generally or perhaps always this will not be a plain text file like your certificate is, but instead a binary blob. The plain text format is called PEM; your fetched binary blob of a certificate is probably in the binary DER encoding. To convert from DER to PEM we do:

openssl x509 -inform DER -in <WGOT-FILE>.crt -outform PEM -out intermediate-01.crt

Now you can inspect intermediate-01.crt in the same to see if it needs a further intermediate certificate; if it does, iterate this process. When you have a suitable collection of PEM format intermediate certificates, simply concatenate them together in order (from the first you fetched to the last, per here) to create your chain file.

PS: The Qualys SSL Server Test is a good way to see how correct your certificate chain is. If it reports that it had to download any certificates, your chain of intermediate certificates is not complete. Similarly it may report that some entries in your chain are not necessary, although in practice this rarely hurts.

Sidebar: Browsers and certificate chains

As you might guess, some but not all browsers appear to use this embedded intermediate certificate URL to automatically fetch any necessary intermediate certificates during certificate validation (as mentioned eg here). Relatedly, browsers will probably not tell you about unnecessary intermediate certificates they received from your website. The upshot of this can be a HTTPS website that works in some browsers but fails in others, and in the failing browser it may appear that you sent no additional certificates as part of a certificate chain. Always test with a tool that will tell you the low-level details.

(Doing otherwise can cause a great deal of head scratching and frustration. Don't ask how I came to know this.)

Comments on this page:

By dozzie at 2014-04-15 07:13:37:

@cks:

Generally or perhaps always this will not be a plain text file like your certificate is, [...]

Actually, I wouldn't call PEM a plain text. It's text, yes, but it's base64-encoded DER, the binary form (hence it's not plain). But this is just a minor issue with terminology.

About checking whether you have complete certificate chain or not, you can do it in more automatic manner:
cat intermediate-ca.cert.pem root-ca.cert.pem > all-ca.cert.pem
openssl verify -CAfile all-ca.cert.pem -CApath /dev/nonexistent/ your.example.org.cert.pem
Order of CA certs shouldn't matter for openssl verify (but it could in production; check the documentation of service the certificate is for). Providing both -CAfile and -CApath is important here, because otherwise openssl could use default CA path(s) for verification (it could happen that intermediate CA is already in that default CA path and that would make the result inaccurate).

Out of the openssl verify command you should get one of the following:

Everything OK, the cert chain is complete (and includes root CA, which is somewhat unnecessary for production use):
your.example.org.cert.pem: OK
CA certificate that signed your certificate is missing (note the distinguished name of the certificate and depth in error message):
your.example.org.cert.pem: /OU=.../CN=your.example.org
error 20 at 0 depth lookup:unable to get local issuer certificate
CA certificate that signed one of the intermediate CAs is missing (again, note the distinguished name of the certificate and depth in error message):
your.example.org.cert.pem: /O=CA Company/CN=CA Company Intermediate Cert
error 2 at 1 depth lookup:unable to get issuer certificate

By David Magda at 2014-04-16 22:53:55:

One important thing to note about chains: if you're using Java's keystore, and trying to import things with the keytool utility, you need to import the root and intermediary certs first before being able to import your own cert.

If the reply [to the CSR request] is a single X.509 certificate, keytool attempts to establish a trust chain, starting at the certificate reply and ending at a self-signed certificate (belonging to a root CA). The certificate reply and the hierarchy of certificates used to authenticate the certificate reply form the new certificate chain of alias. If a trust chain cannot be established, the certificate reply is not imported. In this case, keytool does not print out the certificate and prompt the user to verify it, because it is very hard (if not impossible) for a user to determine the authenticity of the certificate reply.

http://docs.oracle.com/javase/7/docs/technotes/tools/solaris/keytool.html

We were having trouble importing a plain CA-signed PEM file (the private key was already in the store), and then tried the PKCS#7 file which worked. The error message from trying the PEM file were not very useful and so we had some confusion as to why it wasn't working.

If you do a search, you'll find most instructions showing importing of the CA's public keys, and it's never explicitly stated that this is mandatory.

By Rich Moore at 2014-04-19 17:52:26:

If you want to reproduce missing intermediates then using Firefox, close the browser, then delete cert8.db from your profile directory. Then go immediately to your problematic site. This will allow you to see the problem.

FF caches intermediate sites and uses them to correct the problem, so you won't see it in normal usage. IE uses a similar mechanism but fetches the intermediates via windows update.

Written on 14 April 2014.

«	My reactions to Python's warnings module

Where I feel that btrfs went wrong