Bug 130

Created attachment 64 [details] [review]
Proposed patch

There's been prior discussion of cross-compilation on xdg-list and on
gtk-devel-list I think.

Created attachment 67 [details] [review]
updated patch

I checked on the mentioned list you refered to, however, the topic at least on
one of the lists wasn't about where pkgconfig searches for .pc files (which can
be done manually by using $PKG_CONFIG_LIBDIR, but my patch makes that
automatic), but about how to re-use .pc files both for native and for cross
compiling. So I updated my patch:

* it now checks $PKG_CONFIG_X_LIBDIR when cross-compilation is detected, and
doesn't use $PKG_CONFIG_LIBDIR in this case
* updated configure.in and Makefile.am to current syntax and require current
autotools versions
* in case of cross-compilation, override the $prefix variable with
$prefix/<host> (--enable-xprefix configure option, $PKG_CONFIG_XPREFIX env var)
* disable prefix override with empty $PKG_CONFIG_XPREFIX

This way, .pc files can be re-used from a native Windows install.

(I don't see why the current macro would end up finding the build's
pkg-config ... it doesn't make any sense to have the build bin
directory in PATH.)

This patch seems to be adding a lot of assumptions about the directory
setup for cross-compilation that aren't necessarily universal.

While you've allowed changing these assumptions when building pkg-config, that
doesn't really do a lot of good, since presumably the native pkg-config on the
system is simply the pkg-config provided with the system built with
native options.

Frankly, setting PKG_CONFIG_LIBDIR doesn't seem any harder to me 
than creating a symlink to <host>-pkg-config. And much easier to
understand.

So, I'm not sure anything is needed at all. But if you want an automated
system, here's what I'd suggest:

 - Add support for building pkg-config with --target=; the only effect
   of which would be to install the binary as <target>-pkg-config
   (this might be close to working already?)

 - Add another configure option which sets the default value of  
   PKGCONFIG_LIB_DIR.

 - Change pkgconfig.m4 as you suggest.

Then, when building a cross-compilation environment, you just need to 
build another copy of pkg-config pointed at the correct place and 
install it along with your other cross tools, and it will be found
automatically. While there is some "inefficiency" in this, it's
pretty small.

Or, even simpler, we just change pkgconfig.m4 and tell people to
create a wrapper script:

 #!/bin/sh
 export PKG_CONFIG_LIBDIR=/cross-prefix/lib/pkgconfig
 pkgconfig $@

One could also argue that one should try to address the big problem -
sharing .pc files between cross compilation on the host and 
native compilation on the build - first and then perhaps the 
right solution little problem (finding the right .pc files for 
cross compilation) will fall out of that.

> (I don't see why the current macro would end up finding the build's 
> pkg-config ... it doesn't make any sense to have the build bin 
> directory in PATH.) 
 
Huh? pkg-config is usually installed in /usr/bin. That's usually in your 
$PATH. Having to change the $PATH for cross-compiling is tedious and 
error-prone. You nowadays never do that. 
 
> This patch seems to be adding a lot of assumptions about the directory 
> setup for cross-compilation that aren't necessarily universal. 
 
Which? They just assume that you install pkg-config and the host's in the same 
place, which is reasonable. And, as you see, you can override them with 
environment variable to match your layout. 
 
> Frankly, setting PKG_CONFIG_LIBDIR doesn't seem any harder to me  
> than creating a symlink to <host>-pkg-config. And much easier to 
> understand. 
 
No, it's not easier to understand. Have all your build's .pc files in a wierd 
place - just set PKG_CONFIG_LIBDIR. Have all yout cross-compile .pc files in a 
wierd place - just set PKG_CONFIG_X_LIBDIR. This way you adapt them to your 
layout, while otherwise you need to set them according to what you 
cross-compile just now. My approach means: configure your layout (with sane 
defaults given). Your approach: set an env var each time you cross-compile. 
See the difference? Your approach is basically manual intervention into the 
build process. Which is what you want to avoid. 
 
Of course, it works as well. In the same way that hacking a Makefile after a 
configure run works as well. 
 
> - Add support for building pkg-config with --target=; the only effect 
>   of which would be to install the binary as <target>-pkg-config 
>   (this might be close to working already?) 
 
Then you have basically one binary for each possible host platform. In my case 
you have a binary that fits all, and you just need to add a symlink. 
 
If there is an AC_ARG_PROGRAM, then that can be used, and you have --prefix 
anyway. OTOH, one could indeed do a specific configure option instead. 
 
It just depends on whether you want pkg-config to be able to do the right 
thing in the default install or whether you will require cross-compiler users 
to install their own copy of it. 
 
> - Change pkgconfig.m4 as you suggest. 
 
It would be nice if at least that would be in the next version because it 
needs endless time till the new .m4 is used everywhere anyway... 
 
> One could also argue that one should try to address the big problem - 
> sharing .pc files between cross compilation on the host and  
> native compilation on the build - first and then perhaps the  
> right solution little problem (finding the right .pc files for  
> cross compilation) will fall out of that. 
 
Well, that's what the prefix override solves. Of course you can argue that you 
could do that with a seperate copy of <host>-pkg-config as well. 
 
 

(This discussion might be better on xdg-list really)

> (I don't see why the current macro would end up finding the build's 
> > pkg-config ... it doesn't make any sense to have the build bin 
> > directory in PATH.) 
>  
> Huh? pkg-config is usually installed in /usr/bin. That's usually in your 
> $PATH. Having to change the $PATH for cross-compiling is tedious and 
> error-prone. You nowadays never do that. 
> 
Sorry, I was confusing host/build here. 
 
> This patch seems to be adding a lot of assumptions about the directory 
> > setup for cross-compilation that aren't necessarily universal. 
>  
> Which? They just assume that you install pkg-config and the host's in the same 
> place, which is reasonable. And, as you see, you can override them with 
> environment variable to match your layout. 
> 
You patch assumes an installation with prefix /usr/<host>/, which
while not unheard of, is likely uncommon ... most people don't want
to install custom builds into /usr. Yes, you can override that 
with an envvar, but I don't see the advantage of that over 
just setting PKG_CONFIG_LIBDIR for the usual case.
 
> Frankly, setting PKG_CONFIG_LIBDIR doesn't seem any harder to me  
> > than creating a symlink to <host>-pkg-config. And much easier to 
> > understand. 
>  
> No, it's not easier to understand. Have all your build's .pc files in a wierd 
> place - just set PKG_CONFIG_LIBDIR. Have all yout cross-compile .pc files in a 
> wierd place - just set PKG_CONFIG_X_LIBDIR. This way you adapt them to your 
> layout, while otherwise you need to set them according to what you 
> cross-compile just now. My approach means: configure your layout (with sane 
> defaults given). Your approach: set an env var each time you cross-compile. 
> See the difference? Your approach is basically manual intervention into the 
> build process. Which is what you want to avoid. 
 
If you are cross-compiling to half-a-dozen different architectures regularly
with a uniform directory layout, then I guess that might be true. How many
people do that? If we can keep the general use case
(one cross-compilation target at a time) simpler, I don't think asking
such people to have some shell scripts to set up their environment
is unreasonable.

Here's a possible compromise: instead of adding another environment
variable with an unclear interaction with PKG_CONFIG_LIBDIR, why don't
we simply allow variable substitutions of $PLATFORM into PKG_CONFIG_LIBDIR. (I
suggest that token because it's supported by RPATH in the linux dynamic loader,
but $HOST might make more sense) I think that's cleaner.

There is some issue there with native builds there, so perhaps a 
separate envvar PKG_CONFIG_CROSS_LIBDIR does makes sense (I really dislike the
X_LIBDIR naming ... it's abbreviation at the expense
of clarity.) But a variable substitution would IMO be much cleaner
in any case then a format string.

> Of course, it works as well. In the same way that hacking a Makefile after a 
> configure run works as well. 

That's an entirely unjustified comparison.
 
> - Add support for building pkg-config with --target=; the only effect 
> >   of which would be to install the binary as <target>-pkg-config 
> >   (this might be close to working already?) 
>  
> Then you have basically one binary for each possible host platform. In my case 
> you have a binary that fits all, and you just need to add a symlink. 
 
And figure out a printf-formatted envvar and get that into your environment...
Since you need a separate binary for all the other tools,
I don't see having one for pkg-config as much of an issue. As soon
as you have to do anything (create a symlink even), that's the majority
of the conceptual complexity. But sure, if we can avoid it in some
simple way, that's great.
 
[...]

> - Change pkgconfig.m4 as you suggest. 
>  
> It would be nice if at least that would be in the next version because it 
> needs endless time till the new .m4 is used everywhere anyway... 
 
I don't see any problem with making that change. (I don't have access
to pkg-config CVS at the moment, so Havoc will have to do it.) Most packages can
easily enough be autoreconf'ed.
 
> One could also argue that one should try to address the big problem - 
> > sharing .pc files between cross compilation on the host and  
> > native compilation on the build - first and then perhaps the  
> > right solution little problem (finding the right .pc files for  
> > cross compilation) will fall out of that. 
>  
> Well, that's what the prefix override solves. Of course you can argue that you 
> could do that with a seperate copy of <host>-pkg-config as well. 
 
I don't see how the prefix override really solves things, because you
the different-from-install path on the host system, not on the build system. 

Also, overriding the "prefix" variable is likely a bad idea, because the
prefix variable is just a convention that we happened to start using
in the config files. (The basic reason it's there, truth be told, is
because it avoids really nasty shell substitution untangling in the
configure file.) It also doesn't work if the user explicitely specified
--libdir on their configure line, because then libdir=@libdir@
will look like libdir=/usr/lib not libdir=${prefix}/lib.

It seems that a solution here would need to involve DESTDIR in some fashion so
that the correct host paths get put into the .pc files.

We could add a new special internal variable (not allowed to set it in the .pc
file) $_destdir say, so that in gtk+-2.0.pc we would write:

Libs: -L${_destdir}${libdir} -lgtk-${target}-@GTK_API_VERSION@ @GTK_EXTRA_LIBS@
Cflags: -I${_destdir}${includedir}/gtk-2.0 @GTK_EXTRA_CFLAGS@

Then you would set 

 PKG_CONFIG_DESTDIR=/mnt/build/$PLATFORM

(Considerations for adding PKG_CONFIG_CROSS_DESTDIR as well and $HOST vs.
$PLATFORM as above.)

If you did that, then setting DESTDIR to /mnt/build/arm-pc-gnu-linux
or whatever on your install would then work. 

It's unfortunate that this requires modification of every .pc
file, but I don't really see a good alternative.

Created attachment 86 [details] [review]
upupdated patch

Updated patch to be 2.13-safe. Also changed environment cariables to those
proposed by hp.

Darn, I'm blind. Make that "ifdef" instead of "m4_ifelse". 

Howdy, just chasing up old bugs. Ray, does pkg-config still suffer from the symptoms you mention in this bug?

(In reply to comment #9)
> Howdy, just chasing up old bugs. Ray, does pkg-config still suffer from
> the symptoms you mention in this bug?

As I haven't been involved with Debian's pkg-config package for a long time
and the proposed patches here are Rüdiger's, he is probably the right person
to ask.

Most of what's in attachment 86 [details] [review] is now supported by the PKG_CONFIG_SYSROOT_DIR variable. See pkg-config(1) for more description. As I understand things, sysroot style cross compiling is the standard way to do things nowadays. I think this removes the need for the separate PKG_CONFIG_CROSS_LIBDIR and PKG_CONFIG_CROSSPREFIX environment variables as it provides a cross specific variable to use. I tend to agree with comment 6 the extra variables are not needed in the common case and people seem to be getting along fine without them all these years.

The part I do like is the support for $host-pkg-config in pkg.m4 so that you can have an arch-specific pkg-config invoked automatically. I'll work that part into the next version and maybe try to make the pkg-config build support installing $host-pkg-config easily.

I've committed the change to install $host-pkg-config in commit 6b072969. Since that has been supported by PKG_PROG_PKG_CONFIG for a long time, even current packages will pick up arch-specific pkg-configs.

That combined with PKG_CONFIG_SYSROOT_DIR gives (I believe) complete functionality for cross-compiling scenarios. There are a couple things I'd like to make better such as adding a --sysroot command line arg and automatically adding the sysroot directory to the path like gcc, but those are just nice to have.