Bug 79692

Summary: intltool and gettext required to build systemd from archive
Product: systemd Reporter: Filipe Brandenburger <filbranden>
Component: generalAssignee: Filipe Brandenburger <filbranden>
Status: RESOLVED FIXED QA Contact: systemd-bugs
Severity: minor    
Priority: medium CC: piotrdrag
Version: unspecified   
Hardware: All   
OS: All   
Whiteboard:
i915 platform: i915 features:

Description Filipe Brandenburger 2014-06-05 17:35:34 UTC
Hi,

The README file from systemd says it only needs intltool when building from git but that does not seem to be true, since the IT_PROG_INTLTOOL() call in configure.ac basically makes the ./configure break if intltool is not present...

I hacked around this by downloading the intltool-* Perl scripts and putting them in my path but that didn't help since then the build broke because I didn't have gettext available. So, even if --disable-nls is passed to ./configure, the build will require gettext.

I think the correct solution would be for the files generated by intltool to be included in the "make dist" archive. This is the same as it's done with the files generated by autoconf, automake, libtool and is probably what the README file was implying... But even if that's fixed by including them in the dist_* lists (or removing them from the nodist_* lists in the Makefile.am files), there's still the issue of configure breaking if it doesn't find the tools...

I'm wondering to what extent this is a systemd build issue or an intltool bug, but I thought I'd report it here first.

I'll try to look into other packages that use intltool to see if any of them fixes this issue in a more elegant way.

Cheers,
Filipe
Comment 1 Filipe Brandenburger 2014-06-20 12:54:58 UTC
My plan is to get rid of the build time dependency on intltool and replace the cases where intltool-merge is used (namely *.policy.in -> *.policy) with a Python script (using lxml and Python's built-in gettext module) to achieve the same conversion. I have a prototype of a ~60 lines Python script that does that already.

intltool would only be used to generate the po/*.po translation templates when a new *.policy.in is generated or when one of the existing ones is modified (i.e. only needs to be used by developers who want to modify them.)

The gettext package would be used to compile the po/*.po into po/*.mo binaries during build time but I'm planning to support --disable-nls in which case valid *.policy files would be generated but without any translations in them. I also think the right approach is to include the *.policy files into the distribution, so typically a build from the tarball would not really require gettext either.

Any comments on this approach?
Comment 2 Zbigniew Jedrzejewski-Szmek 2014-06-21 02:12:35 UTC
I personally don't have any problem with requiring intltools. And I wouldn't want to add a replacement script, even if it is fairly small, unless it brings some other benefits than simply avoiding a dependency.

What is the exact problem that is being solved here?
Comment 3 Filipe Brandenburger 2014-06-25 06:22:43 UTC
Hi Zbigniew,

Ordinarily, I'd agree with you... However:

* intltool macro IT_PROG_INTLTOOL absolutely requires that intltool is installed
  and doesn't add any flags (like --disable-intltool or --without-intltool) to
  disable it.

* Patches to intltool to add such an option were proposed, but were never merged:
  https://bugs.launchpad.net/intltool/+bug/904647
  https://code.launchpad.net/~hiberis/intltool/bug-904647
  http://bazaar.launchpad.net/~hiberis/intltool/bug-904647/revision/735

* It looks like intltool is pretty much unmaintained these days, I tried contact
  with the maintainers (from the AUTHORS file) and got a bounce for one of the
  addresses and never heard back from the other one.

* "yum install intltool" shows it pulls as dependencies:
  - autoconf
  - automake
  - gettext
  - git(???)
  - perl + tons of perl modules
  I'm not sure if this is a packaging issue, but this clearly looks wrong.

* autoconf, automake and libtool make huge efforts so that those tools are not
  needed when you are building from an archive. Due to the dependencies listed
  above, intltool makes those efforts irrelevant by forcing all these to be
  pulled in again.

* Last, but not least, systemd's README already mentioned that intltool was
  only supposed to be required when building from git (which is understandable,
  in that case autoconf and automake will also be required) but not when
  building from the distribution archive. If this bug is wrong, then there
  is a documentation bug instead...

In any case, I agree with you that my first idea of replacing it with a custom script (even though it was replacing ~1,500 lines of Perl that parses XML using regexps and parses the textual .po's instead of using gettext proper with a ~140 line very well commented python-lxml script) was a bit too extreme.

I took a middle ground and simply found a way to use --disable-nls (which is already defined by intltool, even though it is poorly supported by it) and make it into something useful.

I'm also repeating one of the checks for intltool and disabling NLS support if intltool is not found (as currently NLS is only used for intltool.)

The patch set is here:
http://lists.freedesktop.org/archives/systemd-devel/2014-June/020504.html

I hope you agree with my points here and that the patch set will be acceptable. Let me know if you have any questions about them.

Cheers,
Filipe
Comment 4 Piotr Drąg 2014-06-25 17:23:16 UTC
Could it be possible to write a native .policy parser for gettext, similarly to its .desktop or GSettings parsers? I think that would be in line with what the current gettext maintainer is trying to achieve, which is functionally getting rid of the need for intltool in modern systems. Most GNOME 3 applications can now use just gettext 0.19 for i18n, except for those with policy.in files (like gnome-system-monitor).
Comment 5 Filipe Brandenburger 2014-06-25 17:32:51 UTC
That's interesting Piotr.

Do you have some pointers or background on it?

I'd be happy into looking at trying to contribute an implementation of XML policy files support to gettext. I agree with you that that makes a lot more sense!

However, even if support is added to gettext, it still takes a while until that version of gettext is released, packaged and available in a considerable number of distributions, so probably something to look for in the future (say, remove intltool support from systemd completely in 2015), I still think the patch set I proposed to allow it to be *disabled* when building from archive still makes sense.

Cheers,
Filipe
Comment 6 Piotr Drąg 2014-06-25 17:44:03 UTC
This commit should be helpful: <http://git.savannah.gnu.org/cgit/gettext.git/commit/?id=9a87e6aadd91994d11f1edb33caf0b786d79d86e> (GSettings schemas are also XML-based.)

First step though would be filing a bug against gettext at <http://savannah.gnu.org/bugs/?func=additem&group=gettext>.

And yes, I agree that improving gettext is a long-term solution.
Comment 7 Filipe Brandenburger 2014-06-25 17:50:27 UTC
Thanks for the pointers Piotr!

I just filed the bug:
http://savannah.gnu.org/bugs/index.php?42615

I will take a look at the GSettings patch and work on the PolicyKit ones, I've done the equivalent in ~140 lines of Python so it shouldn't be hard to do a gettext patch to do the same thing cleanly.

Cheers,
Filipe
Comment 8 Zbigniew Jedrzejewski-Szmek 2014-06-26 06:00:21 UTC
Fixed in http://cgit.freedesktop.org/systemd/systemd/commit/?id=f15515b5e6.
Comment 9 Filipe Brandenburger 2014-06-27 06:52:00 UTC
Hi Piotr,

(In reply to comment #6)
> This commit should be helpful:
> <http://git.savannah.gnu.org/cgit/gettext.git/commit/
> ?id=9a87e6aadd91994d11f1edb33caf0b786d79d86e> (GSettings schemas are also
> XML-based.)

I looked at this commit and, while it seems to cover the part of extracting the interesting text from the XML files, it doesn't seem to cover the case of merging the internationalization strings into a new XML as intltool-merge does...

By looking at it a little more, my impression is that for GSettings the internationalization is only done while the XML is compiled.

Do you have an example where intltool was replaced with native gettext support for XML GSettings? A commit on a project that did this transition would be immensely useful.

Cheers!
Filipe
Comment 10 Piotr Drąg 2014-06-27 14:09:23 UTC
I don't think there are any projects requiring gettext 0.19 yet. All I could find is hello-c-gnome3 example in gettext itself:

http://git.savannah.gnu.org/cgit/gettext.git/log/?qt=grep&q=gsettings

Glade/GtkBuilder and .desktop support might also be relevant:

http://git.savannah.gnu.org/cgit/gettext.git/log/?qt=grep&q=glade

http://git.savannah.gnu.org/cgit/gettext.git/log/?qt=grep&q=desktop

I also noticed a project using intltool 0.50 or newer to get rid of the .in hack, which may or may not be helpful to you:

https://git.gnome.org/browse/polari/commit/?id=f188e2e9e2e20e4e075416902b5e1756895c1f7e

https://launchpad.net/intltool/trunk/0.50.0

https://bugs.launchpad.net/intltool/+bug/580526

intltool 0.50 first appeared in Fedora 17 in late 2011. I'm sorry I couldn't help you more, I'm not a programmer or anything.
Comment 11 Piotr Drąg 2014-06-27 16:49:08 UTC
I think Daiki Ueno, gettext maintainer, has a better idea how to cleanly support XML-based formats: http://lists.gnu.org/archive/html/bug-gettext/2014-06/msg00026.html

Use of freedesktop.org services, including Bugzilla, is subject to our Code of Conduct. How we collect and use information is described in our Privacy Policy.