Bug 90600 - IOError: [Errno 2] No such file or directory: 'gl_API.xml'
Summary: IOError: [Errno 2] No such file or directory: 'gl_API.xml'
Status: RESOLVED FIXED
Alias: None
Product: Mesa
Classification: Unclassified
Component: Mesa core (show other bugs)
Version: git
Hardware: x86-64 (AMD64) Linux (All)
: medium normal
Assignee: Jose Fonseca
QA Contact: mesa-dev
URL:
Whiteboard:
Keywords: bisected, regression
Depends on:
Blocks:
 
Reported: 2015-05-22 22:17 UTC by Vinson Lee
Modified: 2015-05-26 14:27 UTC (History)
3 users (show)

See Also:
i915 platform:
i915 features:


Attachments

Description Vinson Lee 2015-05-22 22:17:03 UTC
mesa: 491adb61d25eef8afe2615e0fd842dda20b17004 (master 10.7.0-devel)

CentOS 6 build error

  Generating build/linux-x86_64-debug/mesa/main/dispatch.h ...
Traceback (most recent call last):
  File "src/mapi/glapi/gen/gl_table.py", line 244, in <module>
    main()
  File "src/mapi/glapi/gen/gl_table.py", line 230, in main
    args = _parser()
  File "src/mapi/glapi/gen/gl_table.py", line 225, in _parser
    return parser.parse_args()
  File "/usr/lib/python2.6/site-packages/argparse.py", line 1703, in parse_args
    args, argv = self.parse_known_args(args, namespace)
  File "/usr/lib/python2.6/site-packages/argparse.py", line 1725, in parse_known_args
    default = self._get_value(action, default)
  File "/usr/lib/python2.6/site-packages/argparse.py", line 2248, in _get_value
    result = type_func(arg_string)
  File "src/mapi/glapi/gen/gl_XML.py", line 42, in parse_GL_API
    api.parse_file( file_name )
  File "src/mapi/glapi/gen/gl_XML.py", line 899, in parse_file
    doc = ET.parse( file_name )
  File "/usr/lib64/python2.6/xml/etree/ElementTree.py", line 862, in parse
    tree.parse(source, parser)
  File "/usr/lib64/python2.6/xml/etree/ElementTree.py", line 579, in parse
    source = open(source, "rb")
IOError: [Errno 2] No such file or directory: 'gl_API.xml'
scons: *** [build/linux-x86_64-debug/mesa/main/dispatch.h] Error 1
Comment 1 Brian Paul 2015-05-22 22:22:27 UTC
Vinson, what does your scons command look like?
I haven't been able to reproduce this here.
Comment 2 Vinson Lee 2015-05-22 22:25:26 UTC
(In reply to Brian Paul from comment #1)
> Vinson, what does your scons command look like?
> I haven't been able to reproduce this here.

It is just 'scons', no arguments.

This is the final build command that fails.

/usr/bin/python src/mapi/glapi/gen/gl_table.py -m remap_table -f src/mapi/glapi/gen/gl_and_es_API.xml
Comment 3 Ilia Mirkin 2015-05-22 22:54:05 UTC
IIRC Dylan even mentioned this would break on python2.6 (without the extra argument parser module).

I'm not so eager to drop support for py2.6, not sure why this was done =/
Comment 4 Vinson Lee 2015-05-23 00:04:01 UTC
Build is also broken on Ubuntu 12.04 and it has Python 2.7.

  Generating build/linux-x86_64-debug/mapi/glapi/glapitable.h ...
Traceback (most recent call last):
  File "src/mapi/glapi/gen/gl_table.py", line 244, in <module>
    main()
  File "src/mapi/glapi/gen/gl_table.py", line 230, in main
    args = _parser()
  File "src/mapi/glapi/gen/gl_table.py", line 225, in _parser
    return parser.parse_args()
  File "/usr/lib/python2.7/argparse.py", line 1688, in parse_args
    args, argv = self.parse_known_args(args, namespace)
  File "/usr/lib/python2.7/argparse.py", line 1710, in parse_known_args
    default = self._get_value(action, default)
  File "/usr/lib/python2.7/argparse.py", line 2233, in _get_value
    result = type_func(arg_string)
  File "src/mapi/glapi/gen/gl_XML.py", line 42, in parse_GL_API
    api.parse_file( file_name )
  File "src/mapi/glapi/gen/gl_XML.py", line 899, in parse_file
    doc = ET.parse( file_name )
  File "/usr/lib/python2.7/xml/etree/ElementTree.py", line 1183, in parse
    tree.parse(source, parser)
  File "/usr/lib/python2.7/xml/etree/ElementTree.py", line 647, in parse
    source = open(source, "rb")
IOError: [Errno 2] No such file or directory: 'gl_API.xml'
Comment 5 Ilia Mirkin 2015-05-23 00:05:50 UTC
(In reply to Ilia Mirkin from comment #3)
> IIRC Dylan even mentioned this would break on python2.6 (without the extra
> argument parser module).
> 
> I'm not so eager to drop support for py2.6, not sure why this was done =/

While everything above is true, it's also a bit misleading as I just realized (sorry). Looks like your CentOS box has the separate argparse module as well. If you didn't have it, the errors would be different.
Comment 6 Vinson Lee 2015-05-23 07:50:07 UTC
cf718cc964f86dc49c1fc9ed5e39aa5bd87ad931 is the first bad commit
commit cf718cc964f86dc49c1fc9ed5e39aa5bd87ad931
Author: Dylan Baker <dylanx.c.baker@intel.com>
Date:   Wed Nov 19 13:36:35 2014 -0800

    glapi: gl_table.py: replace getopt with argparse.
    
    This results in slightly less code, but code that is much more readable.
    It has the advantage of putting everything together in one place, all of
    the code is self documenting, help messages are auto-generated, choices
    are automatically enforced, and the syntax is much less C like, taking
    advantage of python features and idioms.
    
    Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
    Acked-by: Matt Turner <mattst88@gmail.com>

:040000 040000 8ced3859bc44a3f2a5cf9d247139b57b6285fc51 4ba8ea55200c5fcdcce7480dbecca41d8f9f88e4 M	src
bisect run success
Comment 7 Pali Rohár 2015-05-24 19:01:03 UTC
Same problem here on Ubuntu 12.04.5 with GNU make:

Making all in mapi/glapi/gen
make[3]: Entering directory `/«PKGBUILDDIR»/build/dri/src/mapi/glapi/gen'
python2  ../../../../../../src/mapi/glapi/gen/../../../../src/mapi/mapi_abi.py \
		--printer glapi --mode lib ../../../../../../src/mapi/glapi/gen/gl_and_es_API.xml > ../../../../src/mapi/glapi/glapi_mapi_tmp.h
python2  ../../../../../../src/mapi/glapi/gen/gl_procs.py -c -f ../../../../../../src/mapi/glapi/gen/gl_and_es_API.xml > ../../../../src/mapi/glapi/glprocs.h
Traceback (most recent call last):
  File "../../../../../../src/mapi/glapi/gen/gl_procs.py", line 191, in <module>
    main()
  File "../../../../../../src/mapi/glapi/gen/gl_procs.py", line 186, in main
    args = _parser()
  File "../../../../../../src/mapi/glapi/gen/gl_procs.py", line 181, in _parser
    return parser.parse_args()
  File "/usr/lib/python2.7/argparse.py", line 1688, in parse_args
    args, argv = self.parse_known_args(args, namespace)
  File "/usr/lib/python2.7/argparse.py", line 1710, in parse_known_args
    default = self._get_value(action, default)
  File "/usr/lib/python2.7/argparse.py", line 2233, in _get_value
    result = type_func(arg_string)
  File "../../../../../../src/mapi/glapi/gen/gl_procs.py", line 168, in <lambda>
    api_type = lambda x: gl_XML.parse_GL_API(x, glX_XML.glx_item_factory())
  File "/«PKGBUILDDIR»/src/mapi/glapi/gen/gl_XML.py", line 42, in parse_GL_API
    api.parse_file( file_name )
  File "/«PKGBUILDDIR»/src/mapi/glapi/gen/gl_XML.py", line 899, in parse_file
    doc = ET.parse( file_name )
  File "/usr/lib/python2.7/xml/etree/ElementTree.py", line 1183, in parse
    tree.parse(source, parser)
  File "/usr/lib/python2.7/xml/etree/ElementTree.py", line 647, in parse
    source = open(source, "rb")
IOError: [Errno 2] No such file or directory: 'gl_API.xml'
make[3]: *** [../../../../src/mapi/glapi/glprocs.h] Error 1
make[3]: Leaving directory `/«PKGBUILDDIR»/build/dri/src/mapi/glapi/gen'
make[2]: *** [all-recursive] Error 1
make[2]: Leaving directory `/«PKGBUILDDIR»/build/dri/src'
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory `/«PKGBUILDDIR»/build/dri'
make: *** [debian/stamp/x86_64-linux-gnu-build-dri] Error 2
Comment 8 Ilia Mirkin 2015-05-24 20:04:17 UTC
I tried actually tracing some of the code in the backtraces, but my line numbers don't match up. The big one that I don't find in argparse.py at all but is there in the trace is:

    default = self._get_value(action, default)

Which... seems disastrous -- the default is 'gl_API.xml', but it's in who-knows-what-directory. This happens to work for in-tree builds due to... luck. However it won't work for out-of-tree builds. The solution is probably to get rid of the default, or to make the "type" more forgiving about missing files.
Comment 9 Brian Paul 2015-05-25 14:18:08 UTC
(In reply to Vinson Lee from comment #6)
> cf718cc964f86dc49c1fc9ed5e39aa5bd87ad931 is the first bad commit
> commit cf718cc964f86dc49c1fc9ed5e39aa5bd87ad931
> Author: Dylan Baker <dylanx.c.baker@intel.com>
> Date:   Wed Nov 19 13:36:35 2014 -0800
> 
>     glapi: gl_table.py: replace getopt with argparse.
>     
>     This results in slightly less code, but code that is much more readable.
>     It has the advantage of putting everything together in one place, all of
>     the code is self documenting, help messages are auto-generated, choices
>     are automatically enforced, and the syntax is much less C like, taking
>     advantage of python features and idioms.
>     
>     Signed-off-by: Dylan Baker <dylanx.c.baker@intel.com>
>     Acked-by: Matt Turner <mattst88@gmail.com>
> 
> :040000 040000 8ced3859bc44a3f2a5cf9d247139b57b6285fc51
> 4ba8ea55200c5fcdcce7480dbecca41d8f9f88e4 M	src
> bisect run success


Vinson, if you revert this commit, does that fix your build?

Maybe we can do that until Dylan can fix it.
Comment 10 Vinson Lee 2015-05-25 19:27:08 UTC
Reverting commit cf718cc964f86dc49c1fc9ed5e39aa5bd87ad931 triggers a similar build error in another file.

Traceback (most recent call last):
  File "src/mapi/glapi/gen/remap_helper.py", line 199, in <module>
    main()
  File "src/mapi/glapi/gen/remap_helper.py", line 189, in main
    args = _parser()
  File "src/mapi/glapi/gen/remap_helper.py", line 184, in _parser
    return parser.parse_args()
  File "/usr/lib/python2.6/site-packages/argparse.py", line 1703, in parse_args
    args, argv = self.parse_known_args(args, namespace)
  File "/usr/lib/python2.6/site-packages/argparse.py", line 1725, in parse_known_args
    default = self._get_value(action, default)
  File "/usr/lib/python2.6/site-packages/argparse.py", line 2248, in _get_value
    result = type_func(arg_string)
  File "src/mapi/glapi/gen/gl_XML.py", line 42, in parse_GL_API
    api.parse_file( file_name )
  File "src/mapi/glapi/gen/gl_XML.py", line 899, in parse_file
    doc = ET.parse( file_name )
  File "/usr/lib64/python2.6/xml/etree/ElementTree.py", line 862, in parse
    tree.parse(source, parser)
  File "/usr/lib64/python2.6/xml/etree/ElementTree.py", line 579, in parse
    source = open(source, "rb")
IOError: [Errno 2] No such file or directory: 'gl_API.xml'


The recent changes must be running into a bug with older versions of Python argparse.

I tested out different Python versions. The build fails with Python <= 2.7.3. A build using Python >= 2.7.4 passes. With Python 2.6 even the latest argparse 1.3.0 module fails.
Comment 11 Dylan Baker 2015-05-25 19:33:46 UTC
I tested with auto tools and python 2.6, so I'm so I'm rather suprised that isn't working. I'm out of town today, but I'm pretty sure I know how to fix it, I'll take care of it first thing in the morning, and send out patches.
Comment 12 Jose Fonseca 2015-05-26 12:05:32 UTC
I posted a fix for review to http://lists.freedesktop.org/archives/mesa-dev/2015-May/085079.html 

But just now I noticed this bug report.
Comment 13 Jose Fonseca 2015-05-26 14:27:06 UTC
commit b787f48ed2a7e1855100afd943ae6b407abb401f
Author: Jose Fonseca <jfonseca@vmware.com>
Date:   Tue May 26 11:01:57 2015 +0100

    glapi: Avoid argparse type argument for API XML input files.
    
    argparse type is a nice type saver for simple data types, but it doesn't
    look a good fit for the input XML file:
    
    - Certain implementations of argparse (particularly python 2.7.3's)
      invoke the type constructor for the default argument even when an
      option is passed in the command line.  Causing `No such file or
      directory: 'gl_API.xml'` when the current dir is not
      src/mapi/glapi/gen.
    
    - The parser takes multiple arguments.  This is currently worked around
      using lambdas, but that unnecessarily complex and hard to read.
      Furthermore it's odd to have a side-effect as heavy as parsing XML
      happening deep inside the argument parsing.
    
    https://bugs.freedesktop.org/show_bug.cgi?id=90600
    
    Reviewed-by: Brian Paul <brianp@vmware.com>


Use of freedesktop.org services, including Bugzilla, is subject to our Code of Conduct. How we collect and use information is described in our Privacy Policy.