Bug 3097 - [PENDING] dependent library elimination causing link problems with -zdefs
[PENDING] dependent library elimination causing link problems with -zdefs
Status: RESOLVED FIXED
Product: pkg-config
Classification: Unclassified
Component: src
unspecified
x86 (IA32) Linux (All)
: high normal
Assigned To: Tollef Fog Heen
:
: 3278 (view as bug list)
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2005-04-22 04:05 UTC by James Henstridge
Modified: 2005-10-06 15:14 UTC (History)
1 user (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description James Henstridge 2005-04-22 04:05:10 UTC
The dependent library removal code causes problems when compiling libraries with
the "-z defs" parameter (libtool's -no-undefined translates to this on some
architectures).

It seems that some OOo hackers have run into this problem:
  http://thread.gmane.org/gmane.comp.gnome.ximian.openoffice/617

You should be able to reproduce this bug with the following steps:
  $ cat foo.c
  #include <gtk/gtk.h>

  GtkWidget *
  foo(void)
  {
      g_message("foo called");
      return gtk_button_new_with_label ("foo");
  }
  $ gcc -fPIC -c foo.c -o foo.o `pkg-config --cflags gtk+-2.0`
  $ gcc -shared -Wl,-zdefs -o libfoo.so foo.o `pkg-config --libs gtk+-2.0`
  foo.o(.text+0x1b): In function `foo':
  : undefined reference to `g_log'
  collect2: ld returned 1 exit status

The -no-undefined/-zdefs flag tells the linker that you're listing every library
that is directly used by the particular library you're linking.  The linker can
use this information to produce slightly better code, since it knows which
library each symbol belongs to.

Some packages assumed that pkg-config would expand the library list for them,
which was a valid assumption for all versions up to 0.15.


What was the rationale behind this change in behaviour?  From what I can tell,
it will also cause problems for static linking, since no existing packages know
that they'd need to use the "pkg-config --static" option in order to get a
complete list, like they used to.
Comment 1 James Henstridge 2005-04-22 04:18:18 UTC
From the sound of it, this problem has mainly been showing its face on Solaris.
 While GCC/Linux supports -zdefs, libtool doesn't actually do anything with its
-no-undefined flag on the platform (on Solaris -no-undefined becomes -zdefs).
Comment 2 James Henstridge 2005-04-22 05:14:40 UTC
Just looked at the closed bugs, and realise this is fairly similar to bug 3060,
which includes your rationale (which I don't fully agree with).

With dependent libraries, there are generally two types of dependencies:
"public" and "private" dependencies.

With public library dependencies the dependency is exposed in the API, and
applications/libraries will generally use API from both libraries.  An example
of this sort of dependency is "gtk+" and "glib".  In this case, omitting the
library dependency is not going to protect against API changes in the dependent
library.

With a private library dependency, the other library is basically an
implementation detail, and applications/libraries couldn't care less.  In this
case, omitting the dependent library on the link line is probably a good thing
for all the reasons you mentioned.  Examples of this type of dependency include
most libraries that use libm.

The way pkg-config worked til 0.16 basically assumed that library dependencies
were public.  From 0.16 onward, it seems to be somewhere in between -- the
cflags of the dependent library are passed so the app can find the include files
but the libraries aren't linked to.  This seems to confuse things a bit.

If the aim is to reduce the number of libraries apps link to, then it would
probably have been better to extend the .pc file format to let people list
private dependencies.

With respect to howl (which you used as an example), it isn't part of the Gnome
development platform.  No library depends on it (as a public or private
dependency) -- only apps that actually use it.
Comment 3 James Henstridge 2005-04-22 07:17:36 UTC
Another note: if the problem that you're trying to solve is to not link against
unused shared libraries, then you'd probably get a better result using the
linker's --as-needed argument.  With the example program I gave, I get the
following:

  $ gcc -shared -Wl,-zdefs -Wl,--as-needed -o libfoo.so foo.o \
        `pkg-config --libs --static gtk+-2.0`
  $ objdump -x libfoo.so | grep NEEDED
    NEEDED      libgtk-x11-2.0.so.0
    NEEDED      libglib-2.0.so.0
    NEEDED      libc.so.6

The libraries that aren't actually used directly by libfoo.so don't generate
DT_NEEDED.  It is wrong to say that the "libglib-2.0.so.0" dependency is
superfluous, since it encodes the fact that libfoo.so depends on the ABI of
"libglib-2.0.so.0" and will likely break if you replace it with an incompatible
version.
Comment 4 Tollef Fog Heen 2005-05-13 02:38:22 UTC
*** Bug 3278 has been marked as a duplicate of this bug. ***
Comment 5 Tollef Fog Heen 2005-05-21 02:41:04 UTC
A fix for this in line with our discussion in Sydney has been commited to CVS.  Can you verify 
that you're happy with the solution?
Comment 6 James Henstridge 2005-05-23 04:05:31 UTC
I'm not sure the change you checked in fully solves the problem.  This is a
summary of the change from my reading of the change log and new tests:
 1. adds a Libs.private line that can be used in a .pc file
 2. `pkg-config --libs foo` will print the results of "Libs" (plus required
    deps)
 3. `pkg-config --static --libs foo` will include the "Libs.private" flags as
    well as "Libs".

While this works for private dependencies that don't have .pc files of their own
(where you'd usually include those flags directly in your own .pc file), it
doesn't handle the case where the dependency does have a .pc file (which will
hopefully become more popular in the future).

To handle this case, what you want is a "Requires.private" line which would do
the following:
 1. act as if it were appended to "Requires" when performing "pkg-config 
    --exists"
 2. ignored when calculating the result for "pkg-config --cflags"
 3. ignored when calculating the result for "pkg-config --libs", except if
    --static is set, in which case act as if it were appended to "Requires".

An example use case for this is Cairo, which links with libpng, but doesn't
#include <png.h> in any of it's public headers.  Currently it has "libpng12" in
its "Requires" line, which could easily be moved into a "Requires.private" line
here, because:
 a. we don't need libpng12's cflags because the headers aren't used.
 b. the libpng12 libs will only need to be explicitly mentioned for static
    linking.
 c. libpng12 is an optional dependency of cairo, so any program depending on
    libpng's headers being available just because they asked for cairo are
    broken :)
Comment 7 James Henstridge 2005-06-12 23:48:16 UTC
Created attachment 2884 [details]
Download button

Here's a first go at implementing what I described above.  I haven't done
extensive testing to see how it interacts with other parts of the code, but it
doesn't break any of the tests, and the following doesn't print anything:
  for file in ~/prefix/lib/pkgconfig/*.pc; do
    pc=`basename $file .pc`
    if [ "`/usr/bin/pkg-config --cflags $pc`" != "`./pkg-config --cflags $pc`"
]; then
      echo $pc cflags
    fi
    if [ "`/usr/bin/pkg-config --libs $pc`" != "`./pkg-config --libs $pc`" ];
then
      echo $pc libs
    fi
  done

I also added a test of --cflags/--libs with and without --static for a package
with Requires and Requires.private.
Comment 8 Tollef Fog Heen 2005-06-27 14:00:06 UTC
pkg-config 0.18 has now been released with a fix for this bug.
Comment 9 Mikhail Zabaluev 2005-10-07 08:14:19 UTC
(In reply to comment #6)
> To handle this case, what you want is a "Requires.private" line which would do
> the following:
>  1. act as if it were appended to "Requires" when performing "pkg-config 
>     --exists"

This solves the problem only halfway. For example, cairo now lists libpng12 in
Requires.private, and every configure script that tests for cairo now fails if
libpng12.pc is not present, even though libpng is not needed for all-dynamic
linking. Maybe --exists should also omit Requires.private unless --static flag
has been specified?
Should I reopen this entry or file a new one to discuss this issue?