Bug 25663

Created attachment 32101 [details] [review]
patch to make MesaLib-7.6.1-rc3 compile on a BG/L

The install process is also slightly broken when not building shared libraries:

/bin/sh ../../bin/minstall ../../lib/libGLU.so* /gpfs/gpfs0/home/matthb2/mesa
Unknown type of argument:  ../../lib/libGLU.so*
make[2]: *** [install] Error 1

libGLU.so doesn't exist but libGLU.a does. 

bugzilla-daemon@freedesktop.org writes:
> --- Comment #1 from Ben Matthews <mben12@gmail.com>  2009-12-15 21:17:36 PST 
> Created an attachment (id=32101)
>  --> (http://bugs.freedesktop.org/attachment.cgi?id=32101)
> patch to make MesaLib-7.6.1-rc3 compile on a BG/L

The compiler.h issue should have been fixed by
4e6e2462ea1fb5b7fc24bb0e707a9cf6507c47c9.  Does BG/L not define _AIX
for some reason?

With regards to your other modifications, you just need to use
-qcpluscmt when compiling with xlc.  Though I do support Mesa switching
to C style comments consistently, there should be a switch && not a
`wrapping' of all the C++ comments.  I was going to bring up this issue
on the devlist at some point, but it slipped my mind with all the 7.6.1
release stuff.

-tom

bugzilla-daemon@freedesktop.org writes:
> --- Comment #2 from Ben Matthews <mben12@gmail.com>  2009-12-15 21:40:25 PST 
> The install process is also slightly broken when not building shared
> libraries:

Actually, its broken even when using shared libraries.  Shared libs
have a ".a" extension on AIX, not a ".so" extension.

> /bin/sh ../../bin/minstall ../../lib/libGLU.so* /gpfs/gpfs0/home/matthb2/mesa
> Unknown type of argument:  ../../lib/libGLU.so*
> make[2]: *** [install] Error 1
> 
> libGLU.so doesn't exist but libGLU.a does. 

Again, this should have been fixed by
9282edfaa0ad24be16f5e395f346ca477050cdf6.  Does BG/L not use a host
triplet that starts with 'aix'?

-tom

BG/L is more or less Linux based (not AIX at all, though still an IBM product). The host string is powerpc-bgl-blrts-gnu-gcc. As such, _AIX is not defined. __blrts and __blrts__ should both be defined instead (at least gcc usually defines both, I'm not sure how to get xlc to dump it's list of preset macros).

I apologize for missing that you had a public git repository. Normally I would try that before reporting anything, but I didn't find it the first time through your website. 

This patch against your latest git version seems to fix the xlc build (gcc is still unhappy about this issue and some of the others -- I can try to figure that out later):

diff --git a/src/mesa/main/compiler.h b/src/mesa/main/compiler.h
index 87c3b2e..4eb249b 100644
--- a/src/mesa/main/compiler.h
+++ b/src/mesa/main/compiler.h
@@ -234,7 +234,7 @@ extern "C" {
 #elif defined(__APPLE__)
 #include <CoreFoundation/CFByteOrder.h>
 #define CPU_TO_LE32( x )       CFSwapInt32HostToLittle( x )
-#elif defined(_AIX)
+#elif (defined(_AIX) || defined(__blrts))
 #define CPU_TO_LE32( x )        x = ((x & 0x000000ff) << 24) | \
                                     ((x & 0x0000ff00) <<  8) | \
                                     ((x & 0x00ff0000) >>  8) | \

I'll pick through your patch, clean things up, and make some commits...

OK, I think I've fixed all the code issues.  Patches committed to the mesa_7_6_branch branch in git.  Can you test that?  I didn't touch the installer issues.

Aside from one missed semicolon that branch now compiles with xlc (though I do see some weird linker warnings at the end)

The fix is here:

diff --git a/src/mesa/main/dlopen.c b/src/mesa/main/dlopen.c
index 325de4b..81e0320 100644
--- a/src/mesa/main/dlopen.c
+++ b/src/mesa/main/dlopen.c
@@ -68,7 +68,7 @@ GenericFunc
 _mesa_dlsym(void *handle, const char *fname)
 {
 #if defined(__blrts)
-   return (GenericFunc) NULL
+   return (GenericFunc) NULL;
 #elif defined(__DJGPP__)
    /* need '_' prefix on symbol names */
    char fname2[1000];


Linker error (which I haven't seen/noticed before.. I'm pretty new to xlc)
/bgl/BlueLight/ppcfloor/blrts-gnu/powerpc-bgl-blrts-gnu/bin/ld: warning: cannot find entry symbol dantic; defaulting to 002000a0

Which I assume is coming from the -pedantic flag here (a couple lines up, but I'm doing parallel builds so it doesn't take all day so the sequencing is non-deterministic) 
/opt/ibmcmp/vacpp/bg/8.0/bin/blrts_xlc -c -I../../include -O3 -pedantic -D_POSIX_SOURCE -D_POSIX_C_SOURCE=199309L -D_SVID_SOURCE -D_BSD_SOURCE readtex.c

I have no idea why it's there or what it's supposed to do. Unfortunately I don't have any easy code to do runtime testing yet (the application of this is proving even harder to build).  

OK, I added the missing semicolon.

The gcc build is still tripping up on sys/endian.h not being found (I have no idea why):

/bgl/BlueLight/ppcfloor/blrts-gnu/bin/powerpc-bgl-blrts-gnu-gcc -c -I../../include -I../../src/mesa -I../../src/gallium/include -I../../src/gallium/auxiliary -O3 -ansi -pedantic -fPIC -ffast-math -D_POSIX_SOURCE -D_POSIX_C_SOURCE=199309L -D_SVID_SOURCE -D_BSD_SOURCE -fno-strict-aliasing main/accum.c -o main/accum.o
/bgl/BlueLight/ppcfloor/blrts-gnu/bin/powerpc-bgl-blrts-gnu-gcc -c -I../../include -I../../src/mesa -I../../src/gallium/include -I../../src/gallium/auxiliary -O3 -ansi -pedantic -fPIC -ffast-math -D_POSIX_SOURCE -D_POSIX_C_SOURCE=199309L -D_SVID_SOURCE -D_BSD_SOURCE -fno-strict-aliasing main/attrib.c -o main/attrib.o
/bgl/BlueLight/ppcfloor/blrts-gnu/bin/powerpc-bgl-blrts-gnu-gcc -c -I../../include -I../../src/mesa -I../../src/gallium/include -I../../src/gallium/auxiliary -O3 -ansi -pedantic -fPIC -ffast-math -D_POSIX_SOURCE -D_POSIX_C_SOURCE=199309L -D_SVID_SOURCE -D_BSD_SOURCE -fno-strict-aliasing main/arrayobj.c -o main/arrayobj.o
In file included from main/imports.h:39,
                 from main/arrayobj.c:44:
main/compiler.h:244:24: sys/endian.h: No such file or directory
In file included from main/imports.h:39In file included from main/mtypes.h:39In file included from main/mtypes.h:39,
                 from main/accum.h:41,
                 from main/accum.h:41,
                 from main/api_exec.c:34,
                 from main/accum.c:26In file included from main/mtypes.h:39:
,
                 from main/api_noop.h:28../../src/mesa/main/compiler.h:244:24:In file included from main/mtypes.h:39,
                 from main/api_noop.c:27In file included from main/mtypes.h:39:
,
                 from main/api_validate.h:31In file included from main/imports.h:39 :
sys/endian.h: No such file or directory../../src/mesa/main/compiler.h:244:24:,
                 from main/api_arrayelt.h:30../../src/mesa/main/compiler.h:244:24:,
                 from main/api_arrayelt.c:30,
                 from main/api_validate.c:26,
                 from main/macros.h:34
,
                 from main/api_loopback.c:33 :
sys/endian.h: No such file or directorymain/compiler.h:244:24:
 :
sys/endian.h: No such file or directory
../../src/mesa/main/compiler.h:244:24: :
sys/endian.h: No such file or directory../../src/mesa/main/compiler.h:244:24: 
sys/endian.h: No such file or directory
 sys/endian.h: No such file or directory
,
                 from main/attrib.c:27:
main/compiler.h:244:24: sys/endian.h: No such file or directory
In file included from ../../src/mesa/glapi/glapi.h:51,
                 from main/mtypes.h:41,
                 from main/api_noop.h:28,
                 from main/api_noop.c:27:
../../src/mesa/glapi/glthread.h:144:6: '#' is not followed by a macro parameter
In file included from ../../src/mesa/glapi/glapi.h:51In file included from ../../src/mesa/glapi/glapi.h:51In file included from ../../src/mesa/glapi/glapi.h:51,
                 from main/mtypes.h:41,
                 from main/mtypes.h:41,
                 from main/mtypes.h:41,
                 from main/accum.h:41,
                 from main/accum.h:41,
                 from main/api_exec.c:34,
                 from main/accum.c:26:
,
                 from main/api_arrayelt.h:30:
,
                 from main/api_arrayelt.c:30:
../../src/mesa/glapi/glthread.h:144:6:../../src/mesa/glapi/glthread.h:144:6: ../../src/mesa/glapi/glthread.h:144:6:  '#' is not followed by a macro parameter
'#' is not followed by a macro parameter
'#' is not followed by a macro parameter
In file included from ../../src/mesa/glapi/glapi.h:51,
                 from main/mtypes.h:41,
                 from main/api_validate.h:31,
                 from main/api_validate.c:26:
../../src/mesa/glapi/glthread.h:144:6: '#' is not followed by a macro parameter
In file included from ../../src/mesa/glapi/glapi.h:51,
                 from main/mtypes.h:41,
                 from main/context.h:53,
                 from main/arrayobj.c:45:
In file included from ../../src/mesa/glapi/glapi.h:51../../src/mesa/glapi/glthread.h:144:6:,
                 from main/mtypes.h:41,
                 from main/accum.h:41,
                 from main/attrib.c:28:
../../src/mesa/glapi/glthread.h:144:6: '#' is not followed by a macro parameter
 '#' is not followed by a macro parameter
In file included from ../../src/mesa/glapi/glapi.h:51,
                 from main/mtypes.h:41,
                 from main/api_loopback.c:35:
../../src/mesa/glapi/glthread.h:144:6: '#' is not followed by a macro parameter
In file included from main/api_arrayelt.h:30,
                 from main/api_arrayelt.c:30:
main/mtypes.h:3129: warning: comma at end of enumerator listIn file included from main/api_noop.h:28,
                 from main/api_noop.c:27:
main/mtypes.h:3129: warning: comma at end of enumerator list

In file included from main/accum.h:41,
                 from main/api_exec.c:34:
main/mtypes.h:3129: warning: comma at end of enumerator listIn file included from main/accum.h:41,
                 from main/accum.c:26:
main/mtypes.h:3129: warning: comma at end of enumerator list

In file included from main/api_validate.h:31,
                 from main/api_validate.c:26:
main/mtypes.h:3129: warning: comma at end of enumerator list
In file included from main/accum.h:41,
                 from main/attrib.c:28:
main/mtypes.h:3129: warning: comma at end of enumerator list
In file included from main/context.h:53,
                 from main/arrayobj.c:45:
main/mtypes.h:3129: warning: comma at end of enumerator list
In file included from main/api_loopback.c:35:
main/mtypes.h:3129: warning: comma at end of enumerator list
make[3]: *** [main/accum.o] Error 1
make[3]: *** Waiting for unfinished jobs....
make[3]: *** [main/api_validate.o] Error 1
make[3]: *** [main/arrayobj.o] Error 1
make[2]: *** [subdirs] Error 1
make[2]: Leaving directory `/gpfs/gpfs0/home/matthb2/mesa_git/mesa/src'
make[1]: *** [default] Error 1
make[1]: Leaving directory `/gpfs/gpfs0/home/matthb2/mesa_git/mesa'
make: *** [bluegene-osmesa] Error 2



That said, if I manually feed this code to the same gcc that mesa is using I get the expected results:
> cat test.c
#if defined(__blrts__)
#error __blrts__
#endif
#if (defined(__blrts) && !defined(blah) && !defined(_GNU_SOURCE)) 
#error defined
#else
#if defined(_GNU_SOURCE)
#warning _GNU_SOURCE
#endif
#endif
#ifdef __blrts
#error __blrts
#endif

/bgl/BlueLight/ppcfloor/blrts-gnu/bin/powerpc-bgl-blrts-gnu-gcc test.c 
test.c:2:2: #error __blrts__
test.c:8:2: warning: #warning _GNU_SOURCE
test.c:12:2: #error __blrts

I can ask around with a few of the other Bluegene users later if you don't see some obvious reason why this should be. 

Thanks for looking into this so quickly!

bugzilla-daemon@freedesktop.org writes:
> --- Comment #10 from Ben Matthews <mben12@gmail.com>  2009-12-16 09:42:25 PST
> The gcc build is still tripping up on sys/endian.h not being found (I have no
> idea why):
> 
> /bgl/BlueLight/ppcfloor/blrts-gnu/bin/powerpc-bgl-blrts-gnu-gcc -c
> -I../../include -I../../src/mesa -I../../src/gallium/include
> -I../../src/gallium/auxiliary -O3 -ansi -pedantic -fPIC -ffast-math
> -D_POSIX_SOURCE -D_POSIX_C_SOURCE=199309L -D_SVID_SOURCE -D_BSD_SOURCE
> -fno-strict-aliasing main/accum.c -o main/accum.o
> /bgl/BlueLight/ppcfloor/blrts-gnu/bin/powerpc-bgl-blrts-gnu-gcc -c
> -I../../include -I../../src/mesa -I../../src/gallium/include
> -I../../src/gallium/auxiliary -O3 -ansi -pedantic -fPIC -ffast-math
> -D_POSIX_SOURCE -D_POSIX_C_SOURCE=199309L -D_SVID_SOURCE -D_BSD_SOURCE
> -fno-strict-aliasing main/attrib.c -o main/attrib.o
> /bgl/BlueLight/ppcfloor/blrts-gnu/bin/powerpc-bgl-blrts-gnu-gcc -c
> -I../../include -I../../src/mesa -I../../src/gallium/include
> -I../../src/gallium/auxiliary -O3 -ansi -pedantic -fPIC -ffast-math
> -D_POSIX_SOURCE -D_POSIX_C_SOURCE=199309L -D_SVID_SOURCE -D_BSD_SOURCE
> -fno-strict-aliasing main/arrayobj.c -o main/arrayobj.o
> In file included from main/imports.h:39,
>                  from main/arrayobj.c:44:
> main/compiler.h:244:24: sys/endian.h: No such file or directory

Hrm.  The ifdef magic in compiler.h is:

  #if __linux__
    use byteswap.h's endian-swapping routine
  #elif __APPLE__
    use CFByteOrder's routine
  #elif _AIX or bluegene
    do the swap manually in C
  #else
    use sys/endian.h's routine.
  #endif

I guess it would be more robust if we used something like the attached
patch implements (warning, tested only on Linux ;).

> That said, if I manually feed this code to the same gcc that mesa is using I
> get the expected results:
> > cat test.c
> #if defined(__blrts__)
> #error __blrts__
> #endif
> #if (defined(__blrts) && !defined(blah) && !defined(_GNU_SOURCE)) 
> #error defined
> #else
> #if defined(_GNU_SOURCE)
> #warning _GNU_SOURCE
> #endif
> #endif
> #ifdef __blrts
> #error __blrts
> #endif
> 
> /bgl/BlueLight/ppcfloor/blrts-gnu/bin/powerpc-bgl-blrts-gnu-gcc test.c 
> test.c:2:2: #error __blrts__
> test.c:8:2: warning: #warning _GNU_SOURCE
> test.c:12:2: #error __blrts

Mesa adds a lot of defines, e.g. POSIX_C_SOURCE, which probably have an
effect on which system includes get brought in implicitly, and which
defines are added implicitly.

-tom

Created attachment 32135 [details] [review]
Patch to use the fallback code in more cases

It appears patches get eaten when using the email interface.

Created attachment 32137 [details] [review]
Fix LIB_EXTENSION in configure.ac

This should address the issue noted in comment #2, at least when using the autoconf build.  Make sure you `autoreconf' after applying.

Ben, can you expound on the host string?  Are all "*-bgl-*" systems going to use '.a' as the library extension?  I guess I'm just not clear on what the "blrts" means, so I've pretended it's important in this patch.

I think it's safe to assume that .a is always going to be the library extension.

Some background which hopefully helps answer the question better: 

It's important to note that there is no concept of a shared library on BG/L. BG/L systems are actually a rather complex architecture. There are three major user facing parts of a BG/L system. There's one or more fast IBM P series machines which provide the UI and cross compilers for the main compute system. The compilers generate a static binary image with the user application and a minimal operating system called blrts. The user can then give that binary image to the scheduler which loads it into RAM on each compute node using some largely undocumented magic. The compute nodes then interface with the rest of the world through a smaller number of IO nodes. The IO nodes are minimal Linux systems which run a server called CIOD which takes system calls on from the compute nodes (over one of the internal interconnects) and acts on them (over GigE.. NFS, Sockets, etc). Blrts is intended to look like Linux to the developer but have a lot less jitter. Unfortunately there are a bunch of features that were never implemented (presumably because they'd be hard to fit into this architecture), including shared libraries/dlopen, threads and listen().

If you want to support all Bluegene systems there are a couple places where this can break down. There are two other classes of Bluegene hardware: the /P and the /Q. BG/Q isn't publicly available yet, but BG/P is and may do things differently. I know it supports more of the features that are missing in /L, but I don't know too much more at the moment. I'm in the process of getting an account on a BG/P system to test the code I'm trying to port so if it's of interest to mesa I can probably tell you more in a few days/weeks. 
The second possible problem is that there's an unofficial alternative to blrts and a corresponding IO node image called ZeptoOS. ZeptoOS runs on both BG/L and BG/P hardware and provides more features than IBM's stack (as well as another toolchain). 



I'll try building with autoconf tomorrow. Thus far I've been using the old build system because it saves me having to trick autoconf into cross compiling (sometimes that's not too bad and sometimes it's a huge problem depending on what tests it wants to do). 

Ping.  Ben, have you tried the autoconf build?  Did you need the "Fix LIB_EXTENSION" patch from comment 13?

Sorry about disappearing, I've been having some trouble getting autoconf to do what I want and been distracted by the holidays.  

First off I had to tweak bin/config.sub to accept the bluegene/l 's host string. For now I used my usual lazy hack:
echo "echo \"powerpc-bgl-blrts-gnu\"" > bin/config.sub

I then tried building with this:
/configure --host=powerpc-bgl-blrts-gnu --target=powerpc-bgl-blrts-gnu --disable-glut --disable-shared --enable-static --disable-driglx-direct --with-driver=osmesa --without-x

Unfortunately the configure script seems to to turn on a lot more code than the old makefile. The first failure is in finding thread support (no threads on BG/L)

powerpc-bgl-blrts-gnu-gcc -c -I../../include -I../../src/mesa -I../../src/gallium/include -I../../src/gallium/auxiliary -g -O2 -Wall -Wmissing-prototypes -std=c99 -ffast-math -fno-strict-aliasing     -D_GNU_SOURCE -DPTHREADS -DHAVE_POSIX_MEMALIGN main/api_arrayelt.c -o main/api_arrayelt.o
In file included from ../../src/mesa/glapi/glapi.h:51,
                 from main/mtypes.h:41,
                 from main/api_arrayelt.h:30,
                 from main/api_arrayelt.c:30:
../../src/mesa/glapi/glthread.h:93:49: pthread.h: No such file or directory
In file included from ../../src/mesa/glapi/glapi.h:51,
                 from main/mtypes.h:41,
                 from main/api_arrayelt.h:30,
                 from main/api_arrayelt.c:30:
../../src/mesa/glapi/glthread.h:96: error: parse error before "pthread_key_t"
../../src/mesa/glapi/glthread.h:96: warning: no semicolon at end of struct or union
../../src/mesa/glapi/glthread.h:98: error: parse error before '}' token
../../src/mesa/glapi/glthread.h:98: warning: type defaults to `int' in declarati
(and so on..)

I did include the patch from comment 13, but I don't think it got far enough for that to matter. 

On a side note, it seems to build fine on BG/P as well (using the makefile). I did have to change the compiler paths but that's no big deal. 

There have been many releases since this stopped working. Seems safe to say this wont be fixed.