Bug 1024

Summary: Japanese ctext conversion bug
Product: xorg Reporter: Alexander Gottwald <ago>
Component: Lib/XlibAssignee: Xorg Project Team <xorg-team>
Status: RESOLVED FIXED QA Contact:
Severity: normal    
Priority: high CC: cyamauch, cyamauch, roland.mainz
Version: git   
Hardware: x86 (IA32)   
OS: Windows (All)   
Whiteboard:
i915 platform: i915 features:
Bug Depends on:    
Bug Blocks: 351    
Attachments:
Description Flags
Patch for wrong sjis conversion
none
patch for ctext conversion
none
test program for conversion none

Description Alexander Gottwald 2004-08-10 03:51:42 UTC
from http://freedesktop.org/pipermail/xorg/2004-August/002027.html

Compound Text Encoding say below 
> Once a GL or GR set has been defined, all further octets in that range (except
within control sequences and
> extended segments) are interpreted with respect to that character set
encoding, until the GL or GR set is
> redefined. GL and GR sets can be defined independently, they do not have to be
defined in pairs.

But in sjis_cttombs, sjis_ctstowcs, euc_ctstowcs and euc_ctstombs,
once GL or GR set is defined, the other set becomes undefined.
Ctext.patch will fix this. (Toshio Takabe)

> Note that when actually using a character set encoding as the GR set, you must
force the most significant
> bit (08/00) of each octet to be a one, so that it falls in the range 10/00 to
15/15.

sjis.patch fixs the problem that sjis_mbstocts doesn't set the most
significant bit to be a one. (Kensuke Matsuzaki)


If you need document about compound text, it is in xc/doc/*/CTEXT/.

I attached a test program. This program converts multi-byte text to
compound text, and re-converts compound text to multi-byte text.

compile
$ gcc -DX_LOCALE cnvtest.c -L /usr/X11R6/lib/ -lX11  -o cnvtest

and run. Input text and output text must be same. But
====
$ ./cnvtest.exe
from:0x82 0xa0 0x82 0xa2 0x82 0xa4 0x61 0x62 0x63 0x82 0xa0 0x82 0xa2 0x82 0xa4

¤¢¤¤¤¦abc¤¢¤¤¤¦
XmbTextListToTextProperty: 0
COMPOUND_TEXT
CompoundText:0x1b 0x24 0x29 0x42 0x24 0x22 0x24 0x24 0x24 0x26 0x61 0x62 0x63 0x
24 0x22 0x24 0x24 0x24 0x26 .
Segmentation fault (core dumped)


After appling patch
====
$ ./cnvtest.exe
from:0x82 0xa0 0x82 0xa2 0x82 0xa4 0x61 0x62 0x63 0x82 0xa0 0x82 0xa2 0x82 0xa4

¤¢¤¤¤¦abc¤¢¤¤¤¦
XmbTextListToTextProperty: 0
COMPOUND_TEXT
CompoundText:0x1b 0x24 0x29 0x42 0xa4 0xa2 0xa4 0xa4 0xa4 0xa6 0x61 0x62 0x63 0x
a4 0xa2 0xa4 0xa4 0xa4 0xa6
XmbTextPropertyToTextList: 0
1
to:0x82 0xa0 0x82 0xa2 0x82 0xa4 0x61 0x62 0x63 0x82 0xa0 0x82 0xa2 0x82 0xa4
¤¢¤¤¤¦abc¤¢¤¤¤¦
Comment 1 Alexander Gottwald 2004-08-10 03:53:26 UTC
Created attachment 589 [details] [review]
Patch for wrong sjis conversion
Comment 2 Alexander Gottwald 2004-08-10 03:53:46 UTC
Created attachment 590 [details] [review]
patch for ctext conversion
Comment 3 Alexander Gottwald 2004-08-10 03:54:20 UTC
Created attachment 591 [details]
test program for conversion
Comment 4 Alexander Gottwald 2004-08-10 03:55:40 UTC
This compound text bug appears if X_LOCALE is defined (which is not on most
linux systems)

With Cygwin/X, XIM is broken because of this. And also that causes
Cygwin/X's clipbord integration problem in Japanese environment.

If there are no objections I'll commit this
Comment 5 Kevin E. Martin 2004-08-13 08:44:37 UTC
Since there no objections, go ahead and commit the patches.
Comment 6 Alexander Gottwald 2004-08-13 09:29:03 UTC
/cvs/xorg/xc/ChangeLog,v  <--  ChangeLog
new revision: 1.263; previous revision: 1.262
/cvs/xorg/xc/lib/X11/lcEuc.c,v  <--  lcEuc.c
new revision: 1.3; previous revision: 1.2
/cvs/xorg/xc/lib/X11/lcSjis.c,v  <--  lcSjis.c
new revision: 1.3; previous revision: 1.2

marking as fixed

Use of freedesktop.org services, including Bugzilla, is subject to our Code of Conduct. How we collect and use information is described in our Privacy Policy.