Bug 94072 - error: The command line is too long when building MESA on Windows with MinGW-W64
Summary: error: The command line is too long when building MESA on Windows with MinGW-W64
Status: NEW
Alias: None
Product: Mesa
Classification: Unclassified
Component: Other (show other bugs)
Version: git
Hardware: x86-64 (AMD64) Windows (All)
: medium normal
Assignee: mesa-dev
QA Contact: mesa-dev
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2016-02-10 04:51 UTC by sav_ix
Modified: 2016-02-11 15:18 UTC (History)
1 user (show)

See Also:
i915 platform:
i915 features:


Attachments
sample of too long command line (2.51 KB, application/octet-stream)
2016-02-11 12:47 UTC, sav_ix
Details

Note You need to log in before you can comment on or make changes to this bug.
Description sav_ix 2016-02-10 04:51:35 UTC
Hi,

Used stuff:
- Windows 10,
- MinGW-W64 (5.3.0-<i686,x86_64),
- MSYS2 (4.3.42(4)-release-(x86_64-pc-msys)),
- Python (2.7.11),
- SCons (2.4.1),
- MESA (11.1.*-dev from git-repository).

For <i686,x86_64> builds got error:
===============================================================
scons.py build=release platform=windows toolchain=mingw machine=x86_64 libgl-gdi

[snip]

  Archiving build\windows-x86\mesa\libmesa.a ...
  Compiling src\gallium\auxiliary\draw\draw_pipe_twoside.c ...
The command line is too long.
scons: *** [build\windows-x86_64\mesa\libmesa.a] Error 1
scons: building terminated because of errors.
===============================================================

I found that it's very similar to which was reported four years ago:
"The command line is too long..."
(details at https://lists.freedesktop.org/archives/mesa-users/2012-October/000496.html ).

Other users also experienced it:
"Compiling Mesa natively on Windows using GCC with SCons results in "the command line is too long" error during linking..."
(details at https://wiki.qt.io/Cross_compiling_Mesa_for_Windows ).

While looking for solution, found in SCons wiki:
"MingW had some problems during the link phase of the build process. It turned out our link lines were in excess of 10000 characters, and this was causing some major grief with python calling spawn()..."
(details at https://bitbucket.org/scons/scons/wiki/LongCmdLinesOnWin32 ).

I assume this error last for such long, because testing of MESA builds using MinGW* was made on Linux with cross-compiling to Windows. Could it be fixed?


Regards,

Alexander
Comment 1 sav_ix 2016-02-11 12:47:35 UTC
Created attachment 121672 [details]
sample of too long command line
Comment 2 Emil Velikov 2016-02-11 14:07:01 UTC
I believe that this issue is not that specific to mesa and/or the build system.

I've had similar experience with piglit (which uses cmake). The workaround I used was to use the shortest path possible for the repo and install location (iirc I used e:\p\ and e:\i\ respectively)

The mesa MSVC builds have been running fine [1] but I'm wondering if we cannot wire up the mingw/mingw-w64 ones ?

[1] https://ci.appveyor.com/project/jrfonseca-fdo/mesa/
Comment 3 sav_ix 2016-02-11 14:22:34 UTC
> I believe that this issue is not that specific to mesa and/or the build system.
Yes, that issue is somewhere between Windows and SCons. More details at https://bitbucket.org/scons/scons/wiki/LongCmdLinesOnWin32 , as I wrote above.

> use the shortest path possible for the repo and install location (iirc I used e:\p\ and e:\i\ respectively)
I assume that would be acceptable not for all users (Linux users don't have such restrictions). But Mesa builds using MinGW-W64 fails even from e:\p\ directory, because 'ar' utility receive command line with about 12,355 symbols (see attachment above), while Windows limit seems to be 10,000 symbols.

> https://ci.appveyor.com/project/jrfonseca-fdo/mesa/
As I understood your sample relates to Mesa builds using LLVM, while my error report relates to builds using MinGW-W64.


Regards,

Alexander
Comment 4 Jose Fonseca 2016-02-11 14:31:12 UTC
(In reply to Emil Velikov from comment #2)
> I believe that this issue is not that specific to mesa and/or the build
> system.
> 
> I've had similar experience with piglit (which uses cmake). The workaround I
> used was to use the shortest path possible for the repo and install location
> (iirc I used e:\p\ and e:\i\ respectively)

Unlike CMake, SCons uses option files with MSVC, so this doesn't affect MSVC.

It can pass as many options as theres free space in the hard drive.

It's just SCons + MinGW on Windows.

> The mesa MSVC builds have been running fine but I'm wondering if we
> cannot wire up the mingw/mingw-w64 ones ?

First need to fix this bug before it's worthwhile.
Comment 5 Jose Fonseca 2016-02-11 14:34:27 UTC
(In reply to sav_ix from comment #0)
> While looking for solution, found in SCons wiki:
> "MingW had some problems during the link phase of the build process. It
> turned out our link lines were in excess of 10000 characters, and this was
> causing some major grief with python calling spawn()..."
> (details at https://bitbucket.org/scons/scons/wiki/LongCmdLinesOnWin32 ).

We probably need something like this.

It's not the first time SCons usage of CRT exec causes problems.

We also had random build failures with SCons on Windows with parallel builds because the CRT is not thread safe.

It would be great if SCons migrated to subprocess.
Comment 6 Jose Fonseca 2016-02-11 15:18:59 UTC
I tried to update mesa/scons/fixes.py with the hack from https://bitbucket.org/scons/scons/wiki/LongCmdLinesOnWin32 without success.

The snippet from https://bitbucket.org/scons/scons/wiki/LongCmdLinesOnWin32 is missing the defition of `escape`

I also tried using subprocess but it fails.

Honestly, the only solution IMO is to fix upstream scons/src/engine/SCons/Platform/win32.py file...


Use of freedesktop.org services, including Bugzilla, is subject to our Code of Conduct. How we collect and use information is described in our Privacy Policy.