If you use Amanda to back up your machines (as we do), and you have just added some Ubuntu 18.04 LTS machines to your fleet and installed the usual amanda-client Ubuntu package to get the necessary client programs, you may some day fire up amrecover on one of them to restore some of those backups. Well, to attempt to restore those backups:

# amrecover -s <server> -t <server> -C <s_config>
AMRECOVER Version 3.5.1. Contacting server on <server> ...
[request failed: amrecover: error [exec /usr/lib/amanda/ambind: No such file or directory]]

Our Amanda servers are running Ubuntu 16.04 LTS, with Amanda 3.3.6. Given this error message (and also the fact that amrecover generally takes several seconds to produce it), we concluded that the 3.5.1 amrecover now requires the Amanda server to have this new ambind program (which only appeared in 3.5). This seemed about par for the course for Ubuntu in 18.04, given issues like libreadline6.

This turns out not to be the case (to my disgusted surprise). Despite how the error message looks, it's the Amanda client (the 18.04 machine) that needs ambind, not the server; amrecover itself is trying to directly execute ambind and failing because indeed ambind's not there. The reason that it's not there is that Ubuntu put ambind into the amanda-server package instead of either amanda-client (which would be appropriate if it's only needed by amrecover) or amanda-common (if it's also needed by Amanda server programs). You probably haven't installed the amanda-server package on your Amanda client machines because, really, why would you?

The good news is that this is easily fixed. Just install amanda-server as well as amanda-client on all of your Ubuntu 18.04 Amanda clients, and everything should be fine. As far as I can tell, installing the server package doesn't do anything dangerous like enable services; it just adds some more programs and manpages.

This packaging issue appears to be inherited from Debian, where the current 'buster (testing)' packages of 3.5.1 also put ambind in the amanda-server package. However, Debian testing is the rolling 'latest development state' of Debian, not shipping as an official LTS release the way Ubuntu 18.04 is.

PS: This is a terrible error message from amrecover, especially under the circumstances. If your program talks to a server, you should always make it completely unambiguous about when you're reporting a local error compared to when you're just relaying an error from the server. If there is any chance of confusion in your error messages, you're doing it wrong.

Sidebar: How I worked this out (mostly grim determination and flailing)

We thought we had a workaround in the form of hacking up the Ubuntu 16.04 Amanda 3.3.6 packages and installing them on 18.04, but then we started to run into various troublesome issues and I decided to see if there was some way of turning off this 'invoke ambind on the server' behavior with an Amanda configuration setting or amrecover command line option. So I went off to look at just what was happening on the Amanda server.

I started by looking at the Amanda server logs. Well, trying to look at them, because there was absolutely nothing being logged about this (which is unusual, the Amanda server stuff is usually quite verbose). My next step was to get out the big hammer and run 'strace -f -e trace=file -o /tmp/st-out -p <xinetd's PID>' on the Amanda server while I invoked amrecover on the client. This too was completely empty, so I spent a while wondering if there was some security setting that was making the strace not work.

Interspersed with trying to trace the server's actions I was also reading through the Amanda source code to try to follow the control flow that sent a message from the client to invoke ambind on the server. The problem was that I couldn't really find anything that looked like this; the only use of ambind I could see seemed entirely inside one file, not the back and forth exchange to the Amanda server stuff that I'd expect. However, I could find something that looked a lot like the 'error [exec ...]' error message that was ultimately being printed out.

All of this led me to run strace on amrecover itself, and lo and behold there was the smoking gun:

20360 execve("/usr/lib/amanda/ambind", ["/usr/lib/amanda/ambind", "5"], 0x7ffe980502e8 /* 113 vars */) = -1 ENOENT (No such file or directory)

Then it was just a matter of using packages.ubuntu.com to verify that ambind was in the amanda-server package and some testing to verify that installing it on an 18.04 test machine appeared to make amrecover happy with life.