Why ––as–needed doesn’t work as expected for your libraries on your autotools project

Introduction to ––as–needed
Quoting from the Gentoo Linux documentation for ––as–needed [1]:

The –as-needed flag is passed to the GNU linker (GNU ld). The flag tells the linker to link in the produced binary only the libraries containing symbols actually used by the binary itself. This binary can be either a final executable or another library.

In a brief, this means that if you pass unneeded libraries to link to when linking your program, the linker will skip those dependencies when that flag is used.
Example, without ––as–needed
Imagine an example where you want to have a “libb.so” shared library, which needs glib-2.0. You’re lazy, and you don’t want to check whether that library will need gthread-2.0, gio-2.0 or gmodule-2.0, so you just compile and link assuming they are used:

$> gcc `pkg-config --cflags glib-2.0 gthread-2.0 gio-2.0 gmodule-2.0` -c src/libb/b.c -fPIC -DPIC -o b.o
$> gcc -shared b.o `pkg-config --libs glib-2.0 gthread-2.0 gio-2.0 gmodule-2.0` -o libb.so

Where “b.c” just contains:

#include "glib.h"
test_b (int number)
  g_debug ("Hello world from libb: %d", number);

If you now show all unused direct dependencies of libb.so, with ldd, you get all unneeded ones:

$> ldd -u -r libb.so
Unused direct dependencies:

Example, with ––as–needed
Lets compile and link now with the ––as–needed linker flag (note that you need to tell gcc to pass the option to the linker, so you’ll need to use “–Wl,––as–needed“):

$> gcc `pkg-config --cflags glib-2.0 gthread-2.0 gio-2.0 gmodule-2.0` -c src/libb/b.c -fPIC -DPIC -o b.o
$> gcc -shared -Wl,--as-needed b.o `pkg-config --libs glib-2.0 gthread-2.0 gio-2.0 gmodule-2.0` -o libb.so

And check the list of unused dependencies:

$> ldd -u -r libb.so
Unused direct dependencies:

So it really seems that the ––as–needed linker option worked here and cleaned up all unneeded dependencies, nice!

But wait! Be careful with this statement in the GNU Linker manpage [2]:

This option affects ELF DT_NEEDED tags for dynamic libraries mentioned on the command line after the ––as–needed option.

You should thus, consider ––as–needed not as a global option to the GNU Linker, but as a specific option for specific dependencies. You could for example apply the option only to some of the libraries you’re linking to:

$> gcc -shared b.o -pthread -Wl,--export-dynamic -lgio-2.0 -lgobject-2.0 -Wl,--as-needed -lgthread-2.0 -lgmodule-2.0 -lrt -lglib-2.0 -o libb.so
$> ldd -u -r libb.so
Unused direct dependencies:

Example, with autotools
You probably don’t compile and link your libraries using plain gcc, and instead use some GNU automagic (autoconf, automake, libtool…). If that is the case, and following the previous example, you would probably use PKG_CHECK_MODULES in your configure.ac:

                  [glib-2.0 gthread-2.0 gmodule-2.0 gio-2.0])

And you’ll probably create your shared library using libtool, so in your Makefile.am you will have:

lib_LTLIBRARIES = lib/libb.la
lib_libb_la_SOURCES = src/libb/b.c
lib_libb_la_CFLAGS = $(GLIB_CFLAGS)
lib_libb_la_LIBADD = $(GLIB_LIBS)

If you compile this autotools project without any additional option, you will end up having “ldd -u -r” report the same unused dependencies as before (gio, gobject, gthread…)

Example, with autotools and ––as–needed
Once you have your autotools project ready, you can try to enable the ––as–needed linker flag passing it in LDFLAGS during configure:

$> ./configure LDFLAGS="-Wl,--as-needed"

This should pass LDFLAGS during the linking operation while running `make’ … and it does pass it, but in a way where it doesn’t have any effect… libtool will call gcc like this:

libtool: link: gcc -shared .libs/lib_libb_la-b.o /usr/lib/libgio-2.0.so /usr/lib/libgobject-2.0.so /usr/lib/libgmodule-2.0.so /usr/lib/libgthread-2.0.so -lrt /usr/lib/libglib-2.0.so -pthread -Wl,--as-needed -pthread -Wl,--export-dynamic -pthread -Wl,-soname -Wl,libb.so.0 -o lib/.libs/libb.so.0.0.0

As you can see, ––as–needed is given after all libraries to link to, which is like doing nothing. If you check for unused dependencies in the generated shared library, you will get:

$> ldd -u -r lib/.libs/libb.so
Unused direct dependencies:

Now, what is the correct way then to enable ––as–needed in this project? Said previously that this option shouldn’t be considered global, and thus passing it in LDFLAGS when running configure doesn’t seem to be a good choice.

This issue is an already known bug in GNU Libtool [4], and while some projects patch their ltmain.sh [5], there are already efforts to push a patch upstream [6].

What about ––as–needed when linking programs?
This is a whole different story. Assume we also have a “a.c” program which uses “libb.so”, and which also has all previous glib, gio and friends as dependencies (but doesn’t use them):

bin_PROGRAMS = bin/a
bin_a_SOURCES = src/a/a.c
bin_a_CPPFLAGS = -I$(top_srcdir)/src/libb
bin_a_LDADD = lib/libb.la $(GLIB_LIBS)

We do again try to pass the linker option in LDFLAGS during configure:

$> ./configure LDFLAGS="-Wl,--as-needed"

We check how libtool passes the option when linking:

libtool: link: gcc -pthread -I/usr/include/glib-2.0 -I/usr/lib/glib-2.0/include -g -O2 -Wl,--as-needed -o bin/.libs/a bin_a-a.o -pthread -Wl,--export-dynamic lib/.libs/libb.so /usr/lib/libgio-2.0.so /usr/lib/libgobject-2.0.so /usr/lib/libgmodule-2.0.so /usr/lib/libgthread-2.0.so -lrt /usr/lib/libglib-2.0.so -pthread

In this case, the option is passed before all linking dependencies, so ––as–needed should take effect. We check with ldd the unused dependencies in the generated binary program:

$> ldd -u -r bin/.libs/a
Unused direct dependencies:

So, even if we couldn’t tell the linker to remove all unused dependencies from the generated shared libraries, the removal takes place in case of generated programs.

You could arguably say that you don’t need to remove unneeded dependencies on shared libraries that you compile, as the programs linking with your library may really need them. In this example, if a program links to our “libb.so” and the program needs gio libraries, if we didn’t remove the unneeded dependencies in “libb.so”, the program would satisfy the gio dependency directly from “libb.so”. But that is definitely not the way to go, if the program needs gio because it uses some of its symbols, it should explicitly link to it, not assume that “libb.so” will force the dependency.

[1] Gentoo Linux documentation for ––as–needed
[2] ld(1) GNU Linker
[3] Full autotools-based example
[4] GNU libtool bug report for –as-needed failing with libraries
[5] Meego libcontentaction patch and Meego Tracker patch
[6] Debian bug #347650

Posted on February 16, 2011, in Development and tagged , , , , . Bookmark the permalink. 4 Comments.

  1. Thanks for this one, it bugged me for quite a while why it didn’t work (not that I looked too much into it until now).

  2. Hi,

    Can you explain why does pthread.so is listed as not used even when –as-needed is specified?

  3. this are strange, pthread not used when -as-needed but its necesary, well some tips may help

    firts, the option was specified after
    second: the include header are only the glib.h

  4. A workaround I’ve found for this: pass the -l as -Xlinker params as well for libtool!
    For example:
    Instead of:
    mylib_la_LDFLAGS = -Xlinker –as-needed -l -l …
    Do this:
    mylib_la_LDFLAGS = -Xlinker –as-needed -Xlinker -l -Xlinker -l …

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: