Specifically, the full-escaped-HTML-document-in-a-single-line comments in _gen/gtypes.h can be too large to fit in flex's parse buffer. Vala's vala-gen-introspect depends upon flex to generate an XML file to represent the content of headers, which is a major part of the bindings generation process. The simplest option seems to be to just break the HTML document comments into reasonably-sized lines (I'm not sure what the exact limit is, but it's well over 100 characters).
Danielle, do you think you could pick this up? Simon said you had plans for a long-term fix for this as well (involving generating gtk-doc/docbook instead of inlined HTML documents, I believe).
http://git.collabora.co.uk/?p=user/danni/telepathy-glib.git;a=shortlog;h=refs/heads/readable-docstrings This is a first pass at this problem using an XSLT stylesheet to process the docstrings into something managable by GTK-Doc. It's certainly not perfect, because it's not trivial to map from XHTML back to GTK-Doc. <tp:dbus-ref> and <tp:member-ref> could specifically do with some work. I wonder if it would be better to process the spec with this stylesheet _before_ running the parser on it, converting all <tp:docstring> elements to some sort of <glib-comment> element that we could copy in as the comment, this would allow simpler access to DOM elements such as <interface>, and also remove the requirement on python-libxslt that I've introduced.
(In reply to comment #2) > http://git.collabora.co.uk/?p=user/danni/telepathy-glib.git;a=shortlog;h=refs/heads/readable-docstrings > > This is a first pass at this problem using an XSLT stylesheet to process the > docstrings into something managable by GTK-Doc. It's certainly not perfect, > because it's not trivial to map from XHTML back to GTK-Doc. > > <tp:dbus-ref> and <tp:member-ref> could specifically do with some work. > > I wonder if it would be better to process the spec with this stylesheet > _before_ running the parser on it, converting all <tp:docstring> elements to > some sort of <glib-comment> element that we could copy in as the comment, this > would allow simpler access to DOM elements such as <interface>, and also remove > the requirement on python-libxslt that I've introduced. I didn't check the XSLT stylesheet in detail, but it seems to work well-enough for now after a few small build, etc. fixes: http://git.collabora.co.uk/?p=user/treitter/telepathy-glib.git;a=shortlog;h=refs/heads/readable-docstrings
I'd like to veto this, because it adds a new dependency (Python libxml2/libxslt bindings) that we don't really need. New dependencies on Python modules are particularly awkward in Scratchbox. specparser.py already knows how to process <tp:dbus-ref> etc. using only the Python standard library; a simple HTML-subset-to-Docbook conversion wouldn't be rocket science, although that's blocked by using specparser in the first place (Bug #24641). Alternatively: on IRC, Danielle and I wondered about just replacing the <tp:docstring> with a hyperlink into telepathy-spec, which would certainly simplify things! Review comments for information: > +pythonpath = '' > +if 'PYTHONPATH' in os.environ: > + pythonpath = os.environ['PYTHONPATH'] > styledoc = libxml2.parseFile(os.path.join( > - os.path.dirname(sys.argv[0]), > + pythonpath, I don't like abusing PYTHONPATH for "other-stuff-path". Having the XSLT inline in the source code (as a """...""" long-literal), having its path passed in as a command-line option to the appropriate tool, or even using Python's __path__ would be better. Relying on PYTHONPATH being set in the top-level Makefile.am also breaks `make -C`, which I use all the time.
I started to write a branch to remove docstrings from the doc-comments, but then realised a ridiculously simpler fix: this is generated code, so nobody really cares where the doc-comments end up, so we can put them into a separate file that only gtk-doc will ever parse. :-P So, we end up with three files: foo.h, foo.c (or foo-body.h for implementations designed to be #included), and foo-gtk-doc.h. Patches on the way!
How's this? I confirm that things are still documented in the gtk-doc (I did a full gtk-doc rebuild by removing *.stamp, and I'll do a build from a clean git checkout before merging). http://git.collabora.co.uk/?p=user/smcv/telepathy-glib-smcv.git;a=shortlog;h=refs/heads/split-codegen-doc We should still do something more sensible with the docstrings, but this branch should fix the immediate breakage in a very straightforward way, and means we don't have to worry about line lengths.
(In reply to comment #6) > How's this? I confirm that things are still documented in the gtk-doc (I did a > full gtk-doc rebuild by removing *.stamp, and I'll do a build from a clean git > checkout before merging). > > http://git.collabora.co.uk/?p=user/smcv/telepathy-glib-smcv.git;a=shortlog;h=refs/heads/split-codegen-doc > > We should still do something more sensible with the docstrings, but this branch > should fix the immediate breakage in a very straightforward way, and means we > don't have to worry about line lengths. Looks good to me. I've confirmed it fixes this bug, so please commit when ready.
Thanks, fixed in git for 0.11.5.
Use of freedesktop.org services, including Bugzilla, is subject to our Code of Conduct. How we collect and use information is described in our Privacy Policy.