== A problem with _gnome-terminal_ in Fedora 21, and tracking it down


Today I discovered that Fedora 21 subtly broke some part of my environment
to the extent that _gnome-terminal_ refuses to start. More than that, it
refuses to start with a completely obscure error message:

.pn prewrap on

 ; gnome-terminal
 Error constructing proxy for org.gnome.Terminal:/org/gnome/Terminal/Factory0: Error calling StartServiceByName for org.gnome.Terminal: GDBus.Error:org.freedesktop.DBus.Error.Spawn.ChildExited: Process org.gnome.Terminal exited with status 8

If you're here searching for the cause of this error message, let
me translate it: what it really means is that your session's
_dbus-daemon_ could not start _/usr/libexec/gnome-terminal-server_
when gnome-terminal asked it to. In many cases, it may be because
your system's environment has not initialized (($LC_CTYPE)) or
(($LANG)) to some UTF-8 locale at the time that your session was
being set up (even if one of these environment variables gets set
later, by the time you're running gnome-terminal). In the modern
world, increasing amount of Gnome bits absolutely insist on being
in a UTF-8 locale and fail hard if they aren't.

Some of you may be going 'what?' here. What you suspect is correct;
the modern Gnome 3 'gnome-terminal' program is basically a cover
script rather than an actual terminal emulator. Instead of opening
up a terminal window itself, it exists to talk over DBus to a master
gnome-terminal-server process (which will theoretically get started
on demand). It is the g-t-s process that is the actual terminal
emulator, creates the windows, starts the shells, and all. And
yes, one process handles *all* of your gnome-terminal windows; if
that process ever hits a bug (perhaps because of something happening
in one window) and dies, all of them die. Let's hope g-t-s doesn't
have any serious bugs.

To find the cause of this issue, well, if I'm being honest a bunch of
this was found with an Internet search of the error message. This didn't
turn up my exact problem but it did turn up people reporting locale
problems and also a mention of _gnome-terminal-server_, which I hadn't
known about before. For actual testing and verification I did several
things:

* first I used _strace_ on _gnome-terminal_ itself, which told me
  nothing useful.

* I discovered that starting _gnome-terminal-server_ by hand before
  running _gnome-terminal_ made everything work.

* I used _dbus-monitor --session_ to watch DBus messages when I tried
  to start _gnome-terminal_. This didn't really tell me anything that
  I couldn't have seen from the error message, but it did verify that
  there was really a DBus message being sent.

* I found the _dbus-daemon_ process that was handling my session DBus
  and used '_strace -f -p ..._' on it while I ran _gnome-terminal_.
  This eventually wound up with it starting _gnome-terminal-server_
  and g-t-s exiting after writing a message to standard error.
  Unfortunately the default _strace_ settings truncated the message,
  so I reran _strace_ while adding '_-e write=2_' to completely dump
  all messages to standard error. This got me the helpful error
  message from g-t-s:
	((Non UTF-8 locale (ANSI_X3.4-1968) is not supported!))

  (If you're wondering if _dbus-daemon_ sends standard error
  from either itself or processes that it starts to somewhere
  useful, ha ha no, sorry, we're all out of luck. As far as I
  can tell it specifically sends standard error to _/dev/null_.)

* I dumped the environment of the _dbus-daemon_ process with
  '_tr '\0' '\n' </proc/<PID>/environ | less_' and inspected what
  environment variables it had set. This showed that it had been
  started without my usual (($LC_CTYPE)) setting ([[cf LocaleQuest]]).

With this in hand I could manually reproduce the problem by trying
to start gnome-terminal-server with (($LC_CTYPE)) unset, and then I
could fix up [[my X startup scripts EncryptedSSHKeyMigration]] to
set (($LC_CTYPE)) before they ran _dbus-launch_.

(This entry is already long enough so I am going to skip my usual rant
about Gnome and especially Gnome 3 making problems like this very
difficult for even experienced system administrators to debug because
there are now so many opaque moving parts to even running Gnome programs
standalone, much less in a full Gnome environment. How is anyone normal
supposed to debug this when _gnome-terminal_ can't even be bothered to
give you a useful error summary in addition to the detailed error report
from DBus?)