== I may be wrong about my simple answer being the right one

In [[a recent entry BashNoInterpreterMsg]] I wrote about how I had
misinterpreted an error message from bash about a script failing,
and I also mentioned in passing that if I had paid attention to
the structure of the error message I would have known that I was
wrong.  I take that back. Detailed investigation has now left me more
confused than I was before and less confidant of what exactly my
co-worker's problem was (and absolutely sure that paying attention to
the structure of the error message does not really help). The problem
is related to [[bash being too smart for its own good in error messages
../unix/BashSuperintelligentExec]]; because of bash's smartness but not
huge smartness, we cannot tell what the actual error is.

As a reminder, here's bash's error message:

> _bash: /a/local/script: /bin/sh: bad interpreter: No such file or
> directory_

You would think that this means that _/bin/sh_ is not present; after
all, it is the straightforward interpretation of the error, plus bash
has actually [[gone out of its way ../unix/BashSuperintelligentExec]] to
give you a more detailed error message. Unfortunately, that is the wrong
interpretation of the error message. What bash is really reporting is two
separate facts:

* /bin/sh is the listed interpreter for /a/local/script
* when bash attempted to _exec()_ the script, the kernel told it _ENOENT_,
  'No such file or directory'.

~~Bash does not mean that /bin/sh is missing~~; it never bothers to
check that (and arguably it can't do so reliably). This matters because
as we saw in [[my previous entry ../unix/BashSuperintelligentExec]], the
kernel will also report _ENOENT_ if the ELF interpreter for a binary is
missing. Now, you guessed it, if your script has a _#!_ line that points
to a binary which has a missing ELF interpreter:

> _bash: /tmp/exmpl: /tmp/a.out: bad interpreter: No such file or directory_

(_/tmp/a.out_ exists and is nominally executable, but I binary edited it
to have a nonexistent ELF interpreter.)

So in my co-worker's case, we can't definitively conclude that _/bin/sh_
was temporarily missing. All we know is that for some reason the
_exec()_ returned _ENOENT_, and that there are at least *two* potential
reasons for it. A _/bin/sh_ symlink being missing is still probably the
most likely explanation, but on a system that's under unusual stresses
things start getting rather uncertain here.

(I am far from certain that I could predict all of the reasons that the
Linux kernel would return _ENOENT_ on _exec()_ without actually tracing
the kernel code. And even then I'm not sure, since there's a lot of deep
bits involved and thus a lot of code to really understand.)