It all started with Ryan Zezeski's question on Twitter:

Twitter friends: I want to better manage my SSH keys. E.g. different pairs for different things. Looking for good resources. Links please.

I have opinions on this (of course) but it turns out that I've never actually written them all down for various reasons, including that some of them feel obvious to me by now. So this is my shot at writing up what I see as good practices for OpenSSH keypairs. This is definitely not the absolutely best and most secure practice for various reasons, but I consider it a good starting point (but see my general cautions about this).

There are some basic and essentially universal things to start with. Use multiple SSH keypairs, with at least different keypairs for different services; there is absolutely no reason that your Github keypair should be the keypair that you use for logging in places, and often you should have different keypairs for logging in to different places. The fundamental mechanic for doing this is a .ssh/config with IdentityFile directives inside Host stanzas; here is a simple example.

(My personal preference is to have different keypairs for each separate machine I'll ssh from, but this could get hard to manage in a hurry if you need a lot of keypairs to start with. Consider doing this only for keypairs that give you broad access or access to relatively dangerous things.)

Encrypt all of your keys. Exactly what password(s) you should use are a tradeoff between security and convenience, but simply encrypting all keys stops or slows down many attacks. For instance, the recent OpenSSH issue would only have exposed (some of) your encrypted keys, which are hopefully relatively hard to crack.

Whenever possible, restrict where your keys are accepted from. This is a straightforward way to limit the damage of a key compromise at the cost of some potential inconvenience if you suddenly need to access systems from an abnormal (network) location. In addition, if you have some unencrypted keys because you need some automated or unattended scripts, consider restricting what these keys can do on the server by using a 'command=' setting in their .ssh/authorized_keys line; an example where we do this is here (see also this). You probably also want to set various no-* options, especially disabling port forwarding.

At this point we're out of truly universal things, as the path splits depending on whether you will access all of your keys via ssh-agent or whether at least some of them will be handled only by ssh (with passphrase challenges every time you use them). There is no single right answer (and covering the issues needs another entry), but for now I'll assume that you'll access all keys via ssh-agent. In this case you'll definitely want to read the discussion of what identities get offered to a remote server and use IdentitiesOnly to limit this.

If you need to ssh to hosts that are only reachable via intermediate hosts, do not forward ssh-agent to the intermediate hosts. Instead, use ProxyCommand to reach through the intermediates. This is sometimes called SSH jump hosts and there are plenty of guides on how to do it. Note that modern versions of OpenSSH have a -W argument for ssh that makes this easy to set up (you no longer need things like netcat on the jumphost).

(There are some cases that need ssh agent forwarding, but plain 'I have to go through A to get to B' is not one of them.)

With lots of keys loaded, your ssh-agent is an extremely large basket of eggs. There are several things you can do here to reduce the potential damage of an attacker gaining access to its full authentication power, although all of them come with convenience tradeoffs:

• Set infrequently used or dangerous keys so that you'll have to confirm it before they can be used, by loading them with ssh-add's -c 'have ssh-agent ask for confirmation' argument.

• Treat some keys basically like temporary sudo privileges by loading them into ssh-agent with a timeout via ssh-add's -t argument. This will force you to reload the key periodically, much as you have to sudo and then re-sudo periodically.

• Arrange to wipe all keys from ssh-agent when you suspend, screenlock, or otherwise clearly leave your machine; my setup for this is covered here.

  (This is good practice in general, but it becomes really important when access to ssh-agent is basically the keys to all the kingdoms.)

You'll probably want to script some of these things to make them more convenient; you might have an 'add-key HOST' command or the like that runs ssh-add on the right key with the right -c or -t parameters. Such scripts will make your life a lot easier and thus make you less likely to throw up your hands and add everything to ssh-agent in permanent, unrestricted form.

(Also, check your ssh_config manpage to see if you have support for AddKeysToAgent. This can be used to create various sorts of convenient 'add to ssh-agent on first use' setups. This is not yet in any released version as far as I know but will probably be in OpenSSH 7.2.)

PS: You probably also want to set HashKnownHosts to yes. I feel conflicted about this, but it's hard to argue that it doesn't increase security and most people won't have my annoyances with it.

PPS: My personal views on SSH key types are that you should use ED25519 keys when possible and otherwise RSA keys (I use 4096 bits just because). Avoid DSA and ECDSA keys; the only time you should generate one is if you have to connect to a device that only supports DSA (and then the key should be specific to the device).