| Summary: | Cross-compilation broken by glsl builtin_compiler | ||
|---|---|---|---|
| Product: | Mesa | Reporter: | Thierry Reding <thierry.reding> |
| Component: | Other | Assignee: | mesa-dev |
| Status: | RESOLVED FIXED | QA Contact: | |
| Severity: | normal | ||
| Priority: | medium | CC: | chadversary, daniel, jfonseca, mattst88, net147, ross |
| Version: | git | ||
| Hardware: | Other | ||
| OS: | All | ||
| See Also: | https://bugs.gentoo.org/show_bug.cgi?id=412931 | ||
| Whiteboard: | |||
| i915 platform: | i915 features: | ||
| Attachments: |
Don't cross-compile src/glsl/builtin_compiler.
Don't cross-compile GLSL builtin compiler MesaLib-9.1.1-cross.patch Patch |
||
Scons supports cross-compilation using approach 1). It builds the builtin_compiler and its dependencies twice. This works because each platform/machine gets built in a separate build sub directory (e.g., build/linux-x86-debug, build/). Getting this to work with the Mesa make file subsystem is probably quite a lot of work. 2) seems an easier approach. But still tricky given that Mesa doesn't allow out of source build. Another option would be simply to eliminate builtin_compiler, and do whatever it does at runtime, when the drivers are loaded. (In reply to comment #1) > Scons supports cross-compilation using approach 1). It builds the > builtin_compiler and its dependencies twice. This works because each > platform/machine gets built in a separate build sub directory (e.g., > build/linux-x86-debug, build/). Are you saying that using SCons, Mesa already cross-builds properly? > Getting this to work with the Mesa make file subsystem is probably quite a lot > of work. You're probably right. But I believe it is the technically correct approach. > 2) seems an easier approach. But still tricky given that Mesa doesn't allow out > of source build. Out of tree builds would also be nice to have. Has anyone ever done some work on that? The problem with this option is that it requires special instructions to make cross-compilation work (compile for build system, then compile for host system with special flags). Cross-compilation really should work out-of-the-box. > Another option would be simply to eliminate builtin_compiler, and do whatever > it does at runtime, when the drivers are loaded. I think that was one of the reasons why builtin_compiler was introduced in the first place, to avoid having to do this at runtime. I think the benefits outweigh the trickiness required at build time. (In reply to comment #2) > Are you saying that using SCons, Mesa already cross-builds properly? Yes -- I use MingW cross compilation with SCons all the time -- but SCons doesn't build all of Mesa yet -- in particular most classic DRI drivers are missing -- so witching to SCons is not trivial either. Created attachment 55540 [details] [review] Don't cross-compile src/glsl/builtin_compiler. Now that Mesa actually builds past the src/glsl directory I get an internal
compiler error when building src/mesa/swrast/s_span.c. I was able to bisect
the ICE to this commit:
3d7c5a4f6f4155e249f6eac76d861eb1bc59eb69
I'm cross-compiling for ARM and the ICE can be triggered by some combination
of optimization flags. I use this command line:
(cd src/mesa && arm-none-linux-gnueabi-gcc -c -o swrast/s_span.o swrast/s_span.c -DFEATURE_GL=1 -DFEATURE_ES1=1 -DFEATURE_ES2=1 -D_GNU_SOURCE -DPTHREADS -DHAVE_POSIX_MEMALIGN -DUSE_XCB -DUSE_XSHM -DHAVE_MINCORE -DHAVE_LIBUDEV -I../../include -I../../src/glsl -I../../src/mesa -I../../src/mapi -I../../src/gallium/include -I../../src/gallium/auxiliary -mlittle-endian -mapcs -mabi=aapcs-linux -g -O2 -marm -march=armv7-a -mtune=cortex-a9 -mfpu=vfpv3-d16 -mfloat-abi=hard -Wall -Wmissing-prototypes -std=c99 -ffast-math -fno-strict-aliasing -fno-builtin-memcmp -fPIC -fvisibility=hidden -save-temps -v)
The error goes away if I, for instance, replace -O2 with -O1. It also goes
away if I drop either of the -mfloat-abi=hard or -mtune=cortex-a9 flags. I
was able to reproduce this with a number of CodeSourcery toolchains all the
way back until 2009q3. 2009q1 builds the file properly.
I guess I should be reporting that to GCC, right?
(In reply to comment #5) > I guess I should be reporting that to GCC, right? Right. http://gcc.gnu.org/bugzilla/show_bug.cgi?id=51882(In reply to comment #6) > (In reply to comment #5) > > I guess I should be reporting that to GCC, right? > > Right. Done: http://gcc.gnu.org/bugzilla/show_bug.cgi?id=51882 I think I can partially fix this with automake. Automake doesn't have a concept for host vs target $(CC)s, so you can't really compile a single binary on the build system which is used to produce code that'll be cross-compiled. So, that doesn't help, but: I think I can add builtin_function.cpp to the BUILT_SOURCES or EXTRA_DIST variables so that when you do `make dist` to generate the Mesa release tarball, it'll build enough of src/glsl/* to generate builtin_function.cpp, which it'll then include in the tarball. The cross-compiling system will then not have to worry about generating it. (In reply to comment #8) > I think I can partially fix this with automake. > > Automake doesn't have a concept for host vs target $(CC)s, so you can't really > compile a single binary on the build system which is used to produce code > that'll be cross-compiled. So, that doesn't help, but: While automake doesn't have built-in functionality for this, the method that I use in the patch that I posted is rather common in projects that generate code to be compiled at build time. Look for example at libXt: http://cgit.freedesktop.org/xorg/lib/libXt/tree/configure.ac#n59 Or libgpg-error: http://git.gnupg.org/cgi-bin/gitweb.cgi?p=libgpg-error.git;a=blob;f=src/Makefile.am;hb=HEAD There is also this: http://sources.redhat.com/autobook/autobook/autobook_270.html > I think I can add builtin_function.cpp to the BUILT_SOURCES or EXTRA_DIST > variables so that when you do `make dist` to generate the Mesa release tarball, > it'll build enough of src/glsl/* to generate builtin_function.cpp, which it'll > then include in the tarball. The cross-compiling system will then not have to > worry about generating it. The problem with that is that it will break cross-compilation from a git checkout. Also I believe the original reason for the decision to generate builtin_function.cpp at build time was that it didn't have to be included in the repository/tarball. Thierry (In reply to comment #9) > Thanks! Looks like I can do something similar! > Looks like I can do something similar!
I've been seeing your automake related commits and I'm really glad to see
someone working on this. Are there plans to actually get this done for 8.0
or will it only go into a later release?
(In reply to comment #11) > I've been seeing your automake related commits and I'm really glad to see > someone working on this. Are there plans to actually get this done for 8.0 > or will it only go into a later release? With how easily a change and break a particular configuration, I wouldn't feel comfortable trying to get automake into 8.0. I'd rather have a release of Mesa where everything is working with automake, instead of ~half anyway. Sorry :( (In reply to comment #12) > (In reply to comment #11) > > I've been seeing your automake related commits and I'm really glad to see > > someone working on this. Are there plans to actually get this done for 8.0 > > or will it only go into a later release? > > With how easily a change and break a particular configuration, I wouldn't feel > comfortable trying to get automake into 8.0. I'd rather have a release of Mesa > where everything is working with automake, instead of ~half anyway. Sorry :( I completely understand. Not a problem at all. Any chance of getting my patch (or something equivalent) merged for 8.0 in order to fix cross-compilation? Possibly -- please send it to mesa-dev@lists.freedesktop.org for review. Created attachment 67142 [details] [review] Don't cross-compile GLSL builtin compiler I've just posted an updated patch to mesa-dev@lists.freedesktop.org, here's the same patch for reference. The changes with respect to the previous version are that this no longer builds in the same directory because automake doesn't allow overriding the CC and CXX variables for specific binaries. So instead this adds a src/glsl/builtin_compiler subdirectory to build in. This has been tested with an ARM cross-compiler and DRI as well as Gallium swrast drivers. I haven't had time to do any runtime tests, but this patch shouldn't modify the runtime behaviour anyway. Ok looking good just some flag cleanups libstdc++ needs to be found so there should be a LDFLAGS for build in my cross toolchain i use sysroot with no system paths ill need to add this for build compile. also GL/gl.h cant be found add "-I $(top_srcdir)/include" to the flags in Makefile.am im abusing CXXFLAGS_FOR_BUILD and setting -L/usr/lib in it and added it as with /include above instead of spliting ball hairs and usinb CXX[C|LD]FLAGS thanks for all the fish. I see the patches landed, can this bug be closed or is there still outstanding work? Still broken I'm afraid: CXXLD builtin_compiler /usr/lib/gcc/sparc-sun-solaris2.10/4.5.3/libstdc++.so: could not read symbols: File in wrong format This is being snuck in via liblibglslcore.la: $ grep ^depend libglslcore.la dependency_libs=' /usr/lib/gcc/sparc-sun-solaris2.10/4.5.3/libstdc++.la' (host system is i686 linux) This seems to be coming from deep inside libtool. I guess a separate libtool needs to be generated for the host CC/CXX. Unfortunately I don't see any easy way of doing that. However, there is a simpler approach. Neither of the libraries in builtin_compiler have any real dependencies. So by changing them libtool libraries to normal ones, it should solve things neatly. Created attachment 77156 [details] [review] MesaLib-9.1.1-cross.patch Suggested fix. (In reply to comment #20) > Created attachment 77156 [details] [review] [review] > MesaLib-9.1.1-cross.patch > > Suggested fix. Looks like you've solved this by not using libtool libraries in src/glsl/builtin_compiler/ but at a cost of building everything twice. I certainly don't want to build everything twice. Any idea how we can solve this bug but avoid unnecessary rebuilds in the common case (i.e., not cross compiling)? Hi, (In reply to comment #20) > Created attachment 77156 [details] [review] [review] > MesaLib-9.1.1-cross.patch > > Suggested fix. Has this fix landed? Ilyes (In reply to comment #22) > Has this fix landed? No. See comment 21. So briefly this worked, but the libtoolisation broke it all again. At least in our build environment (oe-core/Yocto Project) our host sysroot has a host-libtool in, so if it were possible to override the *host* libtool then we could point it at that. Created attachment 79975 [details] [review] Patch Here is a new patch I did. I try to make it only affect the cross compile case. (In reply to comment #25) > Created attachment 79975 [details] [review] [review] > Patch > > Here is a new patch I did. I try to make it only affect the cross compile > case. I don't have access to my test equipment so I wasn't able to test it, but the patch looks good to me. Matt, it seems to address all of your concerns with the previous patches. Any other solution would likely involve fixing libtool to make it properly cope with this type of setup. But libtool hasn't seen a release in almost two years (and even if a new version with a fix were to be release, it'd take about another year to propagate into the various distributions). The patch has been submitted to the mesa-dev mailing list already (2013-06-04), just waiting for approval. I have been using the patch for quite some time now and it is working fine. Thanks! I've finally committed the patch. |
Use of freedesktop.org services, including Bugzilla, is subject to our Code of Conduct. How we collect and use information is described in our Privacy Policy.
When cross-compiling Mesa 7.11 and later, it errors out when trying to generate the glsl builtins. The reason is that the builtin_compiler binary is built for the host system instead of the build system. This was already discussed after the commit that introduced this error but was never solved: http://lists.freedesktop.org/archives/mesa-dev/2011-January/004780.html AFAICT there are two solutions: 1) Fix the build system to generate the builtin_compiler for the build system using something like CC_FOR_BUILD (I don't know if SCons has an equivalent) which will require pretty much all of the src/glsl subdirectory to be built twice. 2) Modify the build system to allow the builtin_compiler to be overridden, such that Mesa can be built once for the build host, and the resulting builtin_compiler binary can be used in the Mesa cross-build. I believe 1) would be the best option. 2) is cumbersome and will require two complete builds, while 1) would only require parts of the src/glsl directory to be rebuilt. 1) also has the advantage that the build will work out-of-the-box without the user having to follow special instructions. I'm willing to spend some time on this, but wanted to discuss things first.