Your C compiler's optimizer can make your bad programs compile

July 14, 2016

Every so often I learn something by having something brought to my awareness. Today's version is that one surprising side effect of optimization in C compilers can be to make your program compile. The story starts with John Regehr's tweets:

@johnregehr: tonight I am annoyed by: a program that both gcc and clang compile at -O2 and neither compiles at -O0

@johnregehr: easier to arrange than you might think

void bar(void);
static int x;
int main(void) { if (x) bar(); }

(With the Fedora 23 versions of gcc 5.3.1 and clang 3.7.0, this will fail to compile at -O0 but compile at -O1 or higher.)

I had to think about this for a bit before the penny dropped and I realized where the problem was and why it happens this way. John Regehr means that this is the whole program (there are no other files), so bar() is undefined. However, x is always 0; it's initialized to 0 by the rules of C, it's not visible outside this file since it's static, and there's nothing here that creates a path to change it. With optimization, the compiler can thus turn main() into:

int main(void) { if (0) bar(); }

This makes the call to bar() unreachable, so the compiler removes it. This leaves the program with no references to bar(), so it can be linked even though bar() is not defined anywhere. Without optimization, an attempt to call bar() remains in the compiled code (even though it will never be reached in execution) and then linking fails with an error about 'undefined reference to `bar''.

(The optimized code that both gcc and clang generate boil down to 'int main(void) { return 0; }', as you'd expect. You can explore the actual code through the very handy GCC/clang compiler explorer.)

As reported on Twitter by Jed Davis, this can apparently happen accidentally in real code (in this case Firefox, with a C++ variant).

Written on 14 July 2016.
« Our central web server, Apache, and slow downloads
Sudo and changes in security expectations (and user behaviors) »

Page tools: View Source, Add Comment.
Login: Password:
Atom Syndication: Recent Comments.

Last modified: Thu Jul 14 00:15:00 2016
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.