Bug 7054

The only ABI/API guarantees provided by poppler are for the glib and qt
bindings.  We can not guarantee any kind of stabilty for the core xpdf API,
since any bugfix or security fix will touch some class and add or remove member
variables.  The problem is that xpdf was never designed with a library / app
split, so we're basically exposing the entire implementation.  This is why we've
chosen to write the wrappers - they provide a minimal, opaque API that we can
control and audit easily.  The xpdf headers are only installed as an option, and
are provided under "use-at-your-own-risk" terms.

What I'll commit to CVS, though, is a change to the poppler-glib.pc file that
will list poppler as a Requires.private dependency so that evince and other apps
using the glib bindings wont be linking directly to a library that break ABI. 
With that it should be possible to bump the libpoppler.so soname without
affecting poppler-glib users, but I'm reluctant to commit to even that. 
Basically any change in the poppler core will affect ABI, so we'd be bumping the
soname with almost every commit.

Oh one more reason for not bumping soname on libpoppler.so abi breakages: since
libtool is braindead and actively defeats clever linking (it expands all
dependencies of a library) I don't want to break poppler-glib user because of an
ABI change in a library they don't use directly.

(In reply to comment #2)
> Oh one more reason for not bumping soname on libpoppler.so abi breakages: since
> libtool is braindead and actively defeats clever linking (it expands all
> dependencies of a library)

Fortunatelly we are dropping libtool usage in Debian when .pc file can be used.

What about linking libpoppler statically in the glib and qt shared libs?

Ondrej, we could encourage people not to link against the shared libpoppler lib
and even drop it completely.  This is what ffmpeg did in Debian for some time. 
This also encourages people to link against the ABI stable bindings.

The fact that the ABI will break even with the smallest patches makes me worry
about security uploads; if we can get rid of the libpoppler shared library and
keep only the bindings as shared libraries (built from the same source), one
security upload would be enough, no package renames etc.

Instead of linking statically, perhaps libpoppler should use the -release
libtool flag to denote an ABI change on each release; at least for Debian, it
would cause more packaging work and longer delays to enter Debian, but it would
be technically safe.

(In reply to comment #1)

> What I'll commit to CVS, though, is a change to the poppler-glib.pc file that
> will list poppler as a Requires.private dependency so that evince and other apps
> using the glib bindings wont be linking directly to a library that break ABI. 
> With that it should be possible to bump the libpoppler.so soname without
> affecting poppler-glib users, but I'm reluctant to commit to even that. 
> Basically any change in the poppler core will affect ABI, so we'd be bumping the
> soname with almost every commit.

I know that you're not guaranteeing compatibility until 1.0 so you can do what ever you want, but just so you know, this may work on ELF systems, but it will break compatibility on OSX, where there is no such thing as an indirect link (pkgconfig on osx will add Requires.private to the link line).

On the other hand, it looks like there are Real Apps that are starting to use poppler, so it would be nice if the libtool conventions were followed.  :(

(In reply to comment #6)
> I know that you're not guaranteeing compatibility until 1.0 so you can do what
> ever you want, but just so you know, this may work on ELF systems, but it will
> break compatibility on OSX, where there is no such thing as an indirect link
> (pkgconfig on osx will add Requires.private to the link line).

What compatibility will be broken?  You don't expect copying binaries between OSX and Linux systems, do you?  OSX binaries will be built in the old fashion, and other systems with modern linkers will save themselves from the SONAME changes hell.

> On the other hand, it looks like there are Real Apps that are starting to use
> poppler, so it would be nice if the libtool conventions were followed.  :(

Libtool is yet another issue, but because it's not flexible enough to avoid the superfluous link flags on arches which can spare them, Debian already stopped installing *.la files in many packages, strips dependency_libs out of the one it can not stop shipping for now, and Debian's libtool will not pull dependencies recursively in *.la files.  Yes, it breaks plenty of libtool things such as (easy) static linking and it only works with modern linkers, but it also means we have way way faster transitions and smaller dependencies on a large scale.


At all rates, I think avoiding the link flag with libpoppler when people are using some high-level bindings is a good thing; especially if it permits having correct SONAME changes in libpoppler which truly breaks ABI frequently.

(In reply to comment #7)

> What compatibility will be broken?  You don't expect copying binaries between
> OSX and Linux systems, do you?  OSX binaries will be built in the old fashion,
> and other systems with modern linkers will save themselves from the SONAME
> changes hell.

I'm talking about upgrade compatibility on the same system.

If I compile my project against /usr/local/lib/libpoppler.1.dylib (or whatever the current soname is) and you change ABIs without upping the soname version, then my app will no longer run when libpoppler gets updated.  Even if it's an indirect dependency from poppler-glib, because my app will have "/usr/local/lib/libpoppler.1.dylib" in it's dependencies which are hardcoded into the application's binaries (Mach-O preserves the semi-equivalent of RPATH always).

> Libtool is yet another issue, but because it's not flexible enough to avoid the
> superfluous link flags on arches which can spare them, Debian already stopped
> installing *.la files in many packages, strips dependency_libs out of the one

Right, but on OSX these superfluous links are not superfluous, the linker fails without them.  That's why libtool encodes all that stuff, because libtool is supposed to hide platform-specific things -- and on the Mac OS X platform, they have to be there.

> At all rates, I think avoiding the link flag with libpoppler when people are
> using some high-level bindings is a good thing; especially if it permits having
> correct SONAME changes in libpoppler which truly breaks ABI frequently.

As long as you intend to only support ELF platforms (or whatever platforms allow indirect linking), this will work.  If you want to support other platforms, it will cause binary incompatibility.

As another person suggested, using -release works as well, although it's ugly.  It means that every release of poppler would have to be packaged with it's own runtime shared-library package that can never be removed until all things that depend on it are removed, but at least it preserves compatibility across upgrades.

I'm coming in late to this discussion and know little about how poppler's libs are arranged, so please bear with me if I ramble a bit...

Is the internal-use library (libpoppler I think) *completely* private and contains no symbols that are accessed in any way by programs (evince, for example) that link against the public libraries (libpoppler-glib and libpoppler-qt)? That is, "nm evince" would list no libpoppler symbols?

So libpoppler itself is hidden from the outside world except that the outside world knows "it exists". In that case (if I understand darwin's linkers), evince links against lipoppler but doesn't actually access its symbols, so I don't think evince is affected by changes to libpoppler's symbols (evince calls functions in libpoppler-glib which calls functions in libpoppler, not "everything is hauled into evince and calls happen there").

But really, if libpoppler is an internal convenience library, may as well just make it really an internal convenience library and not a shared library that happens to be for private use only.

If you're going to take the .pc route, then sounds like libpoppler is a private library, not a private package on its own, so Libs.private:-lpoppler sounds like the way to go. Libs.private is not propagated on OS X. If you have Requires.private:poppler and a separate poppler.pc, you're still just tempting folks to link against it directly ("oh, there's poppler, I'll just 'pkg-config --libs poppler'").

(In reply to comment #8)
> I'm talking about upgrade compatibility on the same system.

That doesn't change on OSX: each time the ABI breaks, the SONAME changes, and you have to rebuild everything deping on libpoppler directly (or indirectly on OSX).  The change is required because if you don't change the SONAME when the ABI changes, some apps will fail to run, but you will only discover when running them.

> If I compile my project against /usr/local/lib/libpoppler.1.dylib (or whatever
> the current soname is) and you change ABIs without upping the soname version,
> then my app will no longer run when libpoppler gets updated.  Even if it's an
> indirect dependency from poppler-glib, because my app will have
> "/usr/local/lib/libpoppler.1.dylib" in it's dependencies which are hardcoded
> into the application's binaries (Mach-O preserves the semi-equivalent of RPATH
> always).

Hmm don't get me wrong: I am personally in favor of changing libpoppler's SONAME each time the ABI change!  But I still would like applications using libpoppler indirectly to not suffer from the changes by not being linked to libpoppler (even if that's not possible on OSX).

> As long as you intend to only support ELF platforms (or whatever platforms
> allow indirect linking), this will work.  If you want to support other
> platforms, it will cause binary incompatibility.

That's a pkg-config or libtool problem to solve; by using .private pkg-config headers, pkg-config should include headers in all cases and include libs only when linking statically or on platforms not capable of indirect deps.

> As another person suggested, using -release works as well, although it's ugly.

Yeah, I did suggest that :)
 
> It means that every release of poppler would have to be packaged with it's own
> runtime shared-library package that can never be removed until all things that
> depend on it are removed, but at least it preserves compatibility across
> upgrades.

Exactly; safest but most painful bet -- not so painful if you don't have to link recursively.

(In reply to comment #9)
> Is the internal-use library (libpoppler I think) *completely* private and
> contains no symbols that are accessed in any way by programs (evince, for
> example) that link against the public libraries (libpoppler-glib and
> libpoppler-qt)? That is, "nm evince" would list no libpoppler symbols?

Evince would use only the glib bindings; but other apps do use libpoppler directly and it needs be a shared library; some examples in Debian:
texlive
kdegraphics-kfile-plugins
gambas2-gb-pdf
pdfcube
referencer
abiword-plugins

(In reply to comment #10)
> (In reply to comment #8)
> > I'm talking about upgrade compatibility on the same system.
> 
> That doesn't change on OSX: each time the ABI breaks, the SONAME changes, and
> you have to rebuild everything deping on libpoppler directly (or indirectly on
> OSX).  The change is required because if you don't change the SONAME when the
> ABI changes, some apps will fail to run, but you will only discover when
> running them.

Right, but then in Fink I can have two packages, libpoppler1-shlibs and libpoppler2-shlibs when the soname changes, so that we provide a compatible library until applications update.  We follow the debian model in that respect.

As long as you can have /path/to/libpoppler.1.dylib and /path/to/libpoppler.2.dylib coexist peacefully, it's fine.  If you change ABI and don't up the soname to 2, that's when there are problems.

> Hmm don't get me wrong: I am personally in favor of changing libpoppler's
> SONAME each time the ABI change!  But I still would like applications using
> libpoppler indirectly to not suffer from the changes by not being linked to
> libpoppler (even if that's not possible on OSX).

Honestly, I believe that should work, but I don't know the specifics enough to be able to say for certain.  Hopefully I can get Peter O'Gorman to chime in, he'll have a better idea how it will work on darwin.

Hi,

we are going to do libpoppler SONAME bump on Debian anyway when needed.

That would keep us incompatible with rest of world, but we are going to be at least compatible with our own packages.

I suggest you follow same path with Fink.  Fortunately with etch-freeze and my laziness we didn't have to do it for libpoppler1, since it never was in unstable (only in experimental).

Ondrej.

Closing the bug, any discussion about this, should take place on the mailing list.