XDG is what was once called the X Desktop Group and is now freedesktop.org. Part of their work is to create various cross-desktop and desktop agnostic standards and tools for how to do things like determine what program should handle a particular MIME type. This includes special 'scheme' MIME types, like x-scheme-handler/ftp, which means 'whatever should handle FTP' (technically this is 'FTP urls'). The XDG tool for mapping MIME types to programs (actually .desktop files) is xdg-mime. Like all of the XDG tools, xdg-mime uses a collection of environment variables, which will normally have the default values covered in the XDG base directory specification.

In theory there are two sorts of data that xdg-mime looks at. The first, found in files called <desktop>-mimeapps.list and mimeapps.list, is supposed to be a set of specifically chosen applications (whether configured by the user or the system). The second is a general cache of MIME type mapping information based on what program can handle what MIME types (as listed in each program's .desktop file); these are found in files called defaults.list and mimeinfo.cache. In practice, a system mimeapps.list file may have just been slapped together based on what the distribution thought was a good idea, and it may not correspond to what you want (or even what you have installed).

(Not all Linux distributions ship a system mimeapps.list. Fedora does; Ubuntu doesn't.)

Xdg-mime searches for mimeapps.list in any of $XDG_CONFIG_HOME, $XDG_CONFIG_DIRS, and the 'applications' subdirectories of $XDG_DATA_HOME and $XDG_DATA_DIRS. It searches for the other two files only in the applications subdirectories of $XDG_DATA_HOME and $XDG_DATA_DIRS.

Now, we get to an important consequence of this. $XDG_DATA_DIRS is normally two directories, /usr/local/share and /usr/share, each of which has MIME type information only for itself, and they are checked in order. The normal result of this is that things installed into /usr/local steal MIME types from things installed in /usr because xdg-mime will check the /usr/local/share/applications files first.

(I discovered this when I installed Google Chrome on my laptop and it promptly stole URL handling from Firefox, which I did not want, because the RPM put bits into /usr/local instead of /usr. Actually finding the files that controlled this was remarkably hard.)

Normally, nothing automatically generates or updates the system mimeapps.list; on Fedora, it's part of the shared-mime-info RPM (and kde-settings for the kde-mimeapps.list version). The mimeinfo.cache files are maintained by update-desktop-database, which will normally be automatically run on package installs, removals, and probably upgrades.

Now, xdg-mime does not give you an actual program to run. What it does is give you the name of a .desktop file, eg firefox.desktop. In order to use this to run a program, you must look through the right magic places to find the .desktop file and then parse it to determine the command line to run. Probably you don't want to do this yourself, but I don't know what your alternative is; as far as I know, there is no XDG tool script to say 'run this .desktop command with the following arguments'.

(Note that the .desktop file is not necessarily in the same directory as the MIME mapping file that caused it to be the handler application. For example, your $HOME/.local/share/applications might just have various MIME type overrides you've set for what system application should handle what.)

This explanation is somewhat dependent on what exact version of the XDG tools and scripts you have installed. It's also not necessarily totally complete, because I am reading through undocumented shell scripts here and I've left a few things out. See the Arch wiki entry on default applications for much more information.

PS: If you're trying to track down why xdg-mime is giving you some strange result, set $XDG_UTILS_DEBUG_LEVEL to at least '2'. This will tell you just what files it looked at when, although I don't think it ever reports what directories it looked at but didn't find any files in.