Bug 16475

Created attachment 17321 [details] [review]
ct_encoding patch for Thai XLC_LOCALE's

Change component to Lib/Xlib (data). Add i18n, patch tags.

Is there any plan for UTF-8 support?

(In reply to comment #3)
> Is there any plan for UTF-8 support?

Oh, yes. You mean adding cs2 as ISO10646-1, right?

Created attachment 17324 [details] [review]
updated patch with ISO10646-1 added

Hello. How is the patch? Is it OK for the commit?

This bug seems to have caused LP#273856 [1] against Ubuntu Intrepid, and a workaround concluded in a local forum is to remove scim. But I think it had better been fixed where the problem is caused.

  [1] https://bugs.launchpad.net/ubuntu/+bug/273856

*ping*

Note that this patch has been applied in Ubuntu Intrepid [1], and Thai Ubuntu users are happy with that.

  [1] https://bugs.launchpad.net/ubuntu/+bug/273856/comments/26

In fact, this bug is found in Debian, too. I just didn't file a Debian bug because I've already filed it here upstream. It seems to be a common practice for Debian to just forward bugs upstream (e.g. [2]). So, filing bug there appears to be redundant, once it's reported upstream.

  [2] http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=443800

How about fixing this?

I found this bug in openSUSE 11.1 which is using libX11-1.1.5, too.

And everything work fine after i applied these patches:

  - Patch of bug #12759 for fix Thai XIM fails to retrieve multi-byte surrounding text on UTF-8 locale
  - Patch of bug #12517 for fix Thai XIM does not filter inputs when NumLock/CapsLock is on
  - Patch of bug #16457 (this bug) for fix CharSet-to-CompoundText Conversion Failed for Thai Locales

But when i was trying to run the simple X applications e.g. Xclock, Xmessage,
it will delay serveral seconds before the app window appeared 
and the terminal have shown the message:
"Warning: Missing charsets in String to FontSet conversion"

I guess this problem was came from the font set which assigned to TIS620-0,
so i had replaced the fs section from TIS620-0:GR to ISO8859-11:GR and try to run X apps again, then it Worked!!

In fact, openSUSE does not generates "tis620-0" encoding lines for Thai font records in neither fonts.dir nor fonts.scale, they have only iso8859-11 and tis620-2 lines.

So, How about replacing the TIS620-0:GR in the fs section to ISO8859-11:GR same as in the cs section ?

(In reply to comment #9)

> But when i was trying to run the simple X applications e.g. Xclock, Xmessage,
> it will delay serveral seconds before the app window appeared 
> and the terminal have shown the message:
> "Warning: Missing charsets in String to FontSet conversion"
> 
> I guess this problem was came from the font set which assigned to TIS620-0,
> so i had replaced the fs section from TIS620-0:GR to ISO8859-11:GR and try to
> run X apps again, then it Worked!!

Confirmed on Debian with all tis620-0 fonts.{dir,alias} entries removed (i.e. until "xlsfonts '*-tis620-0'" matches nothing). libx11 version used is 1.1.5.

(In reply to comment #9)

> In fact, openSUSE does not generates "tis620-0" encoding lines for Thai font
> records in neither fonts.dir nor fonts.scale, they have only iso8859-11 and
> tis620-2 lines.
> 
> So, How about replacing the TIS620-0:GR in the fs section to ISO8859-11:GR same
> as in the cs section ?

In fact, the tis620-0 entries provided in Debian are hard-coded by the font package maintainer (me), not by auto-generation with mkfontdir/mkfontscale. So, the absence of this problem on Debian is rather an exception.

So, if the X font utilities are made to generate iso8859-11 entries, not tis620-0, for Thai fonts, it should be reasonable to adjust either the X locale or the utilities to match each other. And, as iso8859-11 has been used in many parts of X11 for Thai, I think adjusting the X locale to iso8859-11 as proposed is more sensible.

Created attachment 22854 [details] [review]
with ISO8859-11 fontset

(In reply to comment #12)
> Created an attachment (id=22854) [details]
> with ISO8859-11 fontset
> 

I've tested the new patch already.
Now, everything works fine, any X apps can launch properly without tis620-0 entries in fonts.{dir,scale}.

commit d108d3c706af3502820b5202564488ea19908b77
Author: Theppitak Karoonboonyanan <thep@linux.thai.net>
Date:   Thu Apr 9 11:25:25 2009 +0700

    CharSet-to-CompoundText Conversion Failed for Thai Locales
    
    SCIM fails to commit Thai input characters on Thai locales, because it commi
    string in compound text form, which was converted via
    XwcTextListToTextProperty(). But the XLC_LOCALE for th_TH and th_TH.UTF-8
    declares cs1's ct_encoding as TIS620-0:GR, which was commented out in
    src/xlibi18n/lcCT.c default_ct_data, in favor of ISO8859-11 ESC sequence.
    So, declaring cs1 as ISO8859-11:GR instead makes it work.
    
    Besides, for th_TH.UTF-8, adding cs2 class with ISO10646-1 encoding also add
    support for UTF-8 input.
    
    And, along discussion in the bug, a similar problem was found for fontset, t
    by causing delays on X apps startups on systems without tis620-0 fonts. This
    is normally the case, as mkfontdir and mkfontscale generate iso8859-11 entri
    by default for Thai X fonts. So, Thai fontset charset is also patched.
    
    X.Org Bug 16475 <http://bugs.freedesktop.org/show_bug.cgi?id=16475>
    
    Signed-off-by: Theppitak Karoonboonyanan <thep@linux.thai.net>
    Signed-off-by: Julien Cristau <jcristau@debian.org>