Bug 24650 - Visual Studio takes over 30 minutes to compile Mesa core
Summary: Visual Studio takes over 30 minutes to compile Mesa core
Status: RESOLVED FIXED
Alias: None
Product: Mesa
Classification: Unclassified
Component: Mesa core (show other bugs)
Version: 7.5
Hardware: x86 (IA32) Windows (All)
: medium normal
Assignee: mesa-dev
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2009-10-20 19:16 UTC by DigitalGibs
Modified: 2009-11-04 02:32 UTC (History)
0 users

See Also:
i915 platform:
i915 features:


Attachments

Description DigitalGibs 2009-10-20 19:16:36 UTC
The compiler appears to lock up on various files for unreasonably long times; like 20+ minutes for a 500 line source file.

I'm using:
* Windows Vista x64
* 4 GB RAM
* 4-core intel mobile CPU
* Visual Studio 2005 (w/ service packs)

It also often results in failure to link with GDI driver because those files will create invalid object files that are "corrupt".

/// example
1>.\Release\s_texcombine.obj : fatal error LNK1136: invalid or corrupt file
Comment 1 Matt Turner 2009-10-20 20:44:02 UTC
This is pretty vague.

Do you have a test case?
Comment 2 DigitalGibs 2009-10-20 21:55:06 UTC
(In reply to comment #1)
> This is pretty vague.
> 
> Do you have a test case?
> 

Well, I just opened "windows\VC8\mesa\mesa.sln" and hit build.  I know, it's the strangest thing.  I haven't been able to reproduce this with any other projects; only Mesa 3D.  It's like it's getting lost in pre-processor hell or something because it just sits and grinds on a file forever.  I am using 2005 "professional edition" so I'm not sure if it's just a compiler issue or if x64 Windows is doing something evil.
Comment 3 Tom Fogal 2009-10-20 22:12:06 UTC
bugzilla-daemon@freedesktop.org writes:
> --- Comment #2 from DigitalGibs <spam@inlandstudios.com>  2009-10-20 21:55:06
> (In reply to comment #1)
> > This is pretty vague.
> > 
> > Do you have a test case?
> 
> It's like it's getting lost in pre-processor hell or something
> because it just sits and grinds on a file forever.  I am using 2005
> "professional edition" so I'm not sure if it's just a compiler issue
> or if x64 Windows is doing something evil.

First, try compiling in debug mode.  Some of VC's optimizations take
an eternity, and they should be disabled in debug mode.  That would at
least help you narrow down where the issue lies.

Secondly, can you try vs2008?  It's a free download these days.
Supposedly there are differences between the free version and the
"professional", but AFAICT it's not anything substansive.
Comment 4 DigitalGibs 2009-10-20 23:33:48 UTC
(In reply to comment #3)
> bugzilla-daemon@freedesktop.org writes:
> > --- Comment #2 from DigitalGibs <spam@inlandstudios.com>  2009-10-20 21:55:06
> > (In reply to comment #1)
> > > This is pretty vague.
> > > 
> > > Do you have a test case?
> > 
> > It's like it's getting lost in pre-processor hell or something
> > because it just sits and grinds on a file forever.  I am using 2005
> > "professional edition" so I'm not sure if it's just a compiler issue
> > or if x64 Windows is doing something evil.
> 
> First, try compiling in debug mode.  Some of VC's optimizations take
> an eternity, and they should be disabled in debug mode.  That would at
> least help you narrow down where the issue lies.
> 
> Secondly, can you try vs2008?  It's a free download these days.
> Supposedly there are differences between the free version and the
> "professional", but AFAICT it's not anything substansive.
> 

2008 Express seems to fly through them at a more respectable compile time; Release and Debug.

Weird.. thanks.
Comment 5 Jose Fonseca 2009-10-21 01:45:33 UTC
We're using the WinSDK compilers and we're seeing the slow compilation times here too. I've never seen the corrput .obj files, though -- it might a bug be specific to 2005.

MSVC x64 compiler optimization algorithms appear to be inefficient. Certain source files plus any optimization option causes it to take ages -- probably trying to evaluate a combinatorial explosion of alternatives. 

I couldn't detect anything wrong or fancy with the source files in question. It is not even clear why some files are slower and other not. gcc compiles the same code, both in x86 or x64, without any sweat.

I haven't tried VS 2008 yet.
Comment 6 Karl Schultz 2009-11-02 06:30:43 UTC
I've seen the older (2005) compilers take a long time to compile one the
files, image.c I think, in 32-bit Release mode.  I think that this source
code file used a lot of macros that expanded out to a lot of code.  And this
code's size or structure caused the optimizer to take a long time to build
it.

I worked around it in the 2005 studio by turning off optimization just for
that one file.  I have not noticed the problem with the 2008 tools.

On Wed, Oct 21, 2009 at 1:45 AM, <bugzilla-daemon@freedesktop.org> wrote:

> http://bugs.freedesktop.org/show_bug.cgi?id=24650
>
>
>
>
>
> --- Comment #5 from José Fonseca <jfonseca@vmware.com>  2009-10-21
> 01:45:33 PST ---
> We're using the WinSDK compilers and we're seeing the slow compilation
> times
> here too. I've never seen the corrput .obj files, though -- it might a bug
> be
> specific to 2005.
>
> MSVC x64 compiler optimization algorithms appear to be inefficient. Certain
> source files plus any optimization option causes it to take ages --
> probably
> trying to evaluate a combinatorial explosion of alternatives.
>
> I couldn't detect anything wrong or fancy with the source files in
> question. It
> is not even clear why some files are slower and other not. gcc compiles the
> same code, both in x86 or x64, without any sweat.
>
> I haven't tried VS 2008 yet.
>
>
> --
> Configure bugmail: http://bugs.freedesktop.org/userprefs.cgi?tab=email
> ------- You are receiving this mail because: -------
> You are the assignee for the bug.
>
> ------------------------------------------------------------------------------
> Come build with us! The BlackBerry(R) Developer Conference in SF, CA
> is the only developer event you need to attend this year. Jumpstart your
> developing skills, take BlackBerry mobile applications to market and stay
> ahead of the curve. Join us from November 9 - 12, 2009. Register now!
> http://p.sf.net/sfu/devconference
> _______________________________________________
> Mesa3d-dev mailing list
> Mesa3d-dev@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/mesa3d-dev
>
Comment 7 Ian Romanick 2009-11-02 10:06:31 UTC
(In reply to comment #6)
> I've seen the older (2005) compilers take a long time to compile one the
> files, image.c I think, in 32-bit Release mode.  I think that this source
> code file used a lot of macros that expanded out to a lot of code.  And this
> code's size or structure caused the optimizer to take a long time to build
> it.
> 
> I worked around it in the 2005 studio by turning off optimization just for
> that one file.  I have not noticed the problem with the 2008 tools.

Perhaps we could use some #ifdef to detect Visual Studio 2005 and disable optimization via a #pragma?
Comment 8 Karl Schultz 2009-11-03 12:20:32 UTC
Yeah, I'm preparing a set of changes to allow Mesa 7.6 to build on Visual
Studio.  Right now, the 7.6 tarballs do not build.

And I'm sitting here watching the image code compile for a LONG time.

I think I can put this at the top of the file:

#if _MSC_VER == 1400
#pragma optimize("", off)
#endif

This may also work and only turns off global optimizations, which I am
guessing is the issue here.  I'll have to experiment.

#if _MSC_VER == 1400
#pragma optimize("g", off)
#endif

and that should not be too horrible.  But I do hate putting MSFT hacks like
this in the source.

I don't see a good way to turn off the optimization for this file by using
only the build files (Visual Studio project files).  We ship VC8 (the 2005
compiler) project files.  People with VC9 (the 2008 compiler) convert the
VC8 files to VC9 when they load them.  This lets us maintain only one
version to support 2005 and 2008.  If we moved to shipping separate VC8 and
VC9 project files, we could turn off the optimizer in the mesa Project file
for the file(s) in question, but that would be the only difference between
the two versions.  I'm not sure that is worth doing because it is hard
enough keeping one set of project files up to date.

I'll work on it a bit more and submit the appropriate patches.

Karl


On Mon, Nov 2, 2009 at 11:06 AM, <bugzilla-daemon@freedesktop.org> wrote:

> http://bugs.freedesktop.org/show_bug.cgi?id=24650
>
>
>
>
>
> --- Comment #7 from Ian Romanick <idr@freedesktop.org>  2009-11-02
> 10:06:31 PST ---
> (In reply to comment #6)
> > I've seen the older (2005) compilers take a long time to compile one the
> > files, image.c I think, in 32-bit Release mode.  I think that this source
> > code file used a lot of macros that expanded out to a lot of code.  And
> this
> > code's size or structure caused the optimizer to take a long time to
> build
> > it.
> >
> > I worked around it in the 2005 studio by turning off optimization just
> for
> > that one file.  I have not noticed the problem with the 2008 tools.
>
> Perhaps we could use some #ifdef to detect Visual Studio 2005 and disable
> optimization via a #pragma?
>
>
> --
> Configure bugmail: http://bugs.freedesktop.org/userprefs.cgi?tab=email
> ------- You are receiving this mail because: -------
> You are the assignee for the bug.
>
>
> ------------------------------------------------------------------------------
> Come build with us! The BlackBerry(R) Developer Conference in SF, CA
> is the only developer event you need to attend this year. Jumpstart your
> developing skills, take BlackBerry mobile applications to market and stay
> ahead of the curve. Join us from November 9 - 12, 2009. Register now!
> http://p.sf.net/sfu/devconference
> _______________________________________________
> Mesa3d-dev mailing list
> Mesa3d-dev@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/mesa3d-dev
>
Comment 9 Tom Fogal 2009-11-03 12:48:22 UTC
bugzilla-daemon@freedesktop.org writes:
> --- Comment #8 from Karl Schultz <Karl.W.Schultz@gmail.com>  2009-11-03 12:20
> :32 PST ---
> I think I can put this at the top of the file:
> 
> #if _MSC_VER == 1400
> #pragma optimize("", off)
> #endif
[snip -- or variation]
> and that should not be too horrible.  But I do hate putting MSFT
> hacks like this in the source.

FWIW, I like the in-the-source annotations better than build system
changes, because build systems (particularly VS) are decoupled from
source code.  It makes lifting subsets of a tree difficult.

> We ship VC8 (the 2005 compiler) project files.  People with VC9 (the
> 2008 compiler) convert the VC8 files to VC9 when they load them. [.
> . .] If we moved to shipping separate VC8 and VC9 project files, we
> could [make this work, but] I'm not sure that is worth doing because
> it is hard enough keeping one set of project files up to date.

What about a makefile generator?  There's some (out of date?) scons
files in the tree, IIRC.  Not that I care for scons in particular,
but some type of abstracted representation of the build seems to
workaround this issue, and presumably allows one to script builds, even
on windows.

OT, but btw, the default assignee is the mesa dev list; no need to CC
the list on replies unless that's been changed for the bug (your reply
came through twice).
Comment 10 Karl Schultz 2009-11-03 15:07:05 UTC
Hi Tom,

Sorry about the duplicate posts.  Thanks for the info.

I played with this for awhile and didn't get any result I am happy with.
When the optimizer is on, the compiler seems to compile a handful of files
at once and generates code for all of them at the same time.  This makes it
hard to figure out which files are slowing things down.  Adding the pragma
didn't seem to help for the files I tried.  And no, the global optimization
options are not turned on.  The build also seemed slow for files in addition
to image.c.  Part of the problem may be caused by the thing chewing on so
many files at once, and I could not find a way to stop that other than
disabling optimizations completely.

Since this problem is for an old compiler (2005), 2008 (express) is free,
available and does not have this problem, and 2010 is in beta, I'm inclined
to punt and just say that if you want an optimized build, it may take
awhile.  Ideally, most people are only going to compile it that way once.
There may be a SP for 2005; I don't think I have it - I really do not use
Windows much for this sort of thing.

scons - I think that there's another window driver over in the gallium area
and they build it with scons.  We're just doing the visual studio files
because some people like them and simply aren't using scons or similar
systems.

in-source annotations vs build system changes- I see your point, but one can
argue it either way.

Karl


On Tue, Nov 3, 2009 at 1:48 PM, <bugzilla-daemon@freedesktop.org> wrote:

> http://bugs.freedesktop.org/show_bug.cgi?id=24650
>
>
>
>
>
> --- Comment #9 from Tom Fogal <tfogal@alumni.unh.edu>  2009-11-03 12:48:22
> PST ---
> bugzilla-daemon@freedesktop.org writes:
> > --- Comment #8 from Karl Schultz <Karl.W.Schultz@gmail.com>  2009-11-03
> 12:20
> > :32 PST ---
> > I think I can put this at the top of the file:
> >
> > #if _MSC_VER == 1400
> > #pragma optimize("", off)
> > #endif
> [snip -- or variation]
> > and that should not be too horrible.  But I do hate putting MSFT
> > hacks like this in the source.
>
> FWIW, I like the in-the-source annotations better than build system
> changes, because build systems (particularly VS) are decoupled from
> source code.  It makes lifting subsets of a tree difficult.
>
> > We ship VC8 (the 2005 compiler) project files.  People with VC9 (the
> > 2008 compiler) convert the VC8 files to VC9 when they load them. [.
> > . .] If we moved to shipping separate VC8 and VC9 project files, we
> > could [make this work, but] I'm not sure that is worth doing because
> > it is hard enough keeping one set of project files up to date.
>
> What about a makefile generator?  There's some (out of date?) scons
> files in the tree, IIRC.  Not that I care for scons in particular,
> but some type of abstracted representation of the build seems to
> workaround this issue, and presumably allows one to script builds, even
> on windows.
>
> OT, but btw, the default assignee is the mesa dev list; no need to CC
> the list on replies unless that's been changed for the bug (your reply
> came through twice).
>
>
> --
> Configure bugmail: http://bugs.freedesktop.org/userprefs.cgi?tab=email
> ------- You are receiving this mail because: -------
> You are the assignee for the bug.
>
>
> ------------------------------------------------------------------------------
> Come build with us! The BlackBerry(R) Developer Conference in SF, CA
> is the only developer event you need to attend this year. Jumpstart your
> developing skills, take BlackBerry mobile applications to market and stay
> ahead of the curve. Join us from November 9 - 12, 2009. Register now!
> http://p.sf.net/sfu/devconference
> _______________________________________________
> Mesa3d-dev mailing list
> Mesa3d-dev@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/mesa3d-dev
>
Comment 11 Jose Fonseca 2009-11-04 02:32:57 UTC
(In reply to comment #10)
> 
> scons - I think that there's another window driver over in the gallium area
> and they build it with scons.  We're just doing the visual studio files
> because some people like them and simply aren't using scons or similar
> systems.

SCons can also generates visual studio project files. I never tried that myself as I use Linux and eclipse myself, but I'll gladly help if anybody's interested in giving a try.


Use of freedesktop.org services, including Bugzilla, is subject to our Code of Conduct. How we collect and use information is described in our Privacy Policy.