Blog Archives

Debugging programs with a external symbol file

As explained in one of my previous posts, objcopy utility can be used to store the debugging information generated by gcc compilation in an external file. This is very useful when you want to debug a program using a core file generated by the stripped binary.

Imagine that you’re a developer in a company which distributes program binaries which are stripped before being delivered to the end users (you probably don’t need too much imagination for this). Of course, that evil company doesn’t distribute the source code of the program.

One of your users experiences a problem with your program. You tell him to run:

ulimit -c unlimited

This will tell the system that the size allowed for the core files generated is unlimited. These core files are generated when some unhandled signals are received in the program, like SIGSEGV when a segmentation fault occurs, and include a complete dump of the program’s memory, program stack and such when the crash occured.

If you don’t get a core file after a segmentation fault and you already executed the ulimit command, it may be due to different reasons, like these:

  • Kernel limitation (there’s a specific maximum size for the core files)
  • Available disk space (you don’t have enough disk space to store your core file).

If you got a complete valid core file after a program crash, you may want to debug it with the GNU Debugger gdb. Of course, you don’t want the user of the program to know the internals of the code (remember that the company is evil and doesn’t distribute the source code), so in order to debug the core file generated by the user’s execution, you need your symbol file for that specific compilation of the program.


gdb ./myprogram corefile

Once you have started the GNU debugger with the previous command, you are able to investigate what happened in the program abort. If you don’t have debugging symbols in the binary, you will get a backtrace equivalent to this one:

Core was generated by `./myprogram'.
Program terminated with signal 11, Segmentation fault.
[New process 8986]
#0 0x080483dd in myfunction ()
(gdb) bt
#0 0x080483dd in myfunction ()
#1 0x080483d1 in myfunction ()
#2 0x080483d1 in myfunction ()
#3 0x080483d1 in myfunction ()
#4 0x080483d1 in myfunction ()
#5 0x080483d1 in myfunction ()
#6 0x080483d1 in myfunction ()
#7 0x080483d1 in myfunction ()
#8 0x080483d1 in myfunction ()
#9 0x080483d1 in myfunction ()
#10 0x080483d1 in myfunction ()
#11 0x0804840e in main ()

Not really useful isn’t it? That’s the maximum information your user will know about the problem.

But you are the developer and when you compiled the program, and before you redistributed it, you stripped the binary and stored the debugging information in an external symbol file. So you just need to tell gdb where that file is:

(gdb) symbol-file myprogram.debug
Reading symbols from /home/drehbahn/myprogram.debug...done.

And magic happens. Now you can get a pretty backtrace of the program stack:

(gdb) bt
#0 0x080483dd in myfunction (value=10) at myprogram.c:26
#1 0x080483d1 in myfunction (value=9) at myprogram.c:20
#2 0x080483d1 in myfunction (value=8) at myprogram.c:20
#3 0x080483d1 in myfunction (value=7) at myprogram.c:20
#4 0x080483d1 in myfunction (value=6) at myprogram.c:20
#5 0x080483d1 in myfunction (value=5) at myprogram.c:20
#6 0x080483d1 in myfunction (value=4) at myprogram.c:20
#7 0x080483d1 in myfunction (value=3) at myprogram.c:20
#8 0x080483d1 in myfunction (value=2) at myprogram.c:20
#9 0x080483d1 in myfunction (value=1) at myprogram.c:20
#10 0x080483d1 in myfunction (value=0) at myprogram.c:20
#11 0x0804840e in main () at myprogram.c:33

And you can step into the first frame to see the real details of the problem which caused the segfault:

(gdb) fr 0
#0 0x080483dd in myfunction (value=10) at myprogram.c:26
26 char character = *ptr; /* oops */
(gdb) list
21 }
22 else
23 {
24 /* Create nice segfault... */
25 char *ptr = NULL;
26 char character = *ptr; /* oops */
27 printf("Did I really arrive here?\n");
28 }
29 }
30

So, yes, it’s easy to debug a program with a separate symbol file created with objcopy.

And even better, you don’t need to be an evil company which doesn’t publish the source code of their apps to use this approach. Stripping binaries before shipping the programs is quite common in the world of Free Software, as size of the binaries really matters. The difference is that in this case you can always download not only the original source code that was used to generate the binary, but also the specific symbol files for direct debugging with the GNU Debugger.

[References]

  1. GDB, The GNU Project Debugger
  2. GDB Files
  3. GNU Binutils

[See also]

  1. Make your programs lose weight: Stripping binaries

Make your programs lose weight: stripping binaries

`objcopy‘  is one of the utilities included in the GNU Binutils package. This tool allows copying binary files while transforming them in the process.

One of the typical purposes of modifying binary files after they have been compiled is to separate the debugging symbols and sections and the real executable. If you compiled you binary with debugging information (option ‘-g’ in gcc), the size of the binary will grow a lot due to the extra symbols, sections and human-readable stuff stored within the binary. That information is really useful when you are debugging your app, anyway, so you don’t really want to completely get rid of it.

So, we can use objcopy to move all the debugging information from the binary to another file, so that:

  • Binary is smaller
  • Binary does not include any ‘human-readable’ information of the program
  • Debugging information can be kept by the developers, so that if any ‘core’ file is received for that specific binary, debugging with GDB is possible.

[Step 1] Copy the binary to another file, keeping only the debugging information. This will create the symbol file for the binary, and will be kept by the developers of the program for debugging purposes:

objcopy --only-keep-debug $BINARY_PATH $DEBUG_INFO_FILE

[Step 2] Once you have created the symbol file, you can now remove all the debugging information from the binary itself:

objcopy --strip-debug $BINARY_PATH

[Step 3] After you have created both files, an extra optional option is to add a debug-link to the binary, to specify which is the exact file containing the debug information for that binary:

objcopy --add-gnu-debuglink=$DEBUG_INFO_FILE $BINARY_PATH

Please note that this third step is optional, as we can use specific GDB commands when debugging a core file to specify in which file the debug information is located.

[References]

  1. http://sourceware.org/binutils
  2. http://www.gnu.org/software/binutils
Follow

Get every new post delivered to your Inbox.

Join 36 other followers