Bug 81875

Summary: cs_CZ.UTF-8 should interpret dead_caron + u as uring
Product: xorg Reporter: vlmarek
Component: Input/KeyboardAssignee: Xorg Project Team <xorg-team>
Status: RESOLVED MOVED QA Contact: Xorg Project Team <xorg-team>
Severity: normal    
Priority: medium CC: cloos, havran.jan, j-pi, mkyral, nicky726, semerad, tomi, victor
Version: unspecified   
Hardware: Other   
OS: All   
Whiteboard:
i915 platform: i915 features:
Attachments:
Description Flags
Patch to add cs_CZ.UTF-8/Compose.pre
none
[PATCH] nls: fix handling of cs_CZ.UTF-8 locale
none
fix handling of cs_CZ locale - remove reference to empty XLC_LOCALE none

Description vlmarek 2014-07-29 13:12:04 UTC
Hi,

[ I'm making copy of https://bugzilla.novell.com/show_bug.cgi?id=888446 where I reported the bug previously ]

My .profile contains:
export GTK_IM_MODULE=xim
export QT_IM_MODULE=xim
export LC_ALL=cs_CZ.UTF-8

Now if I press dead_caron + u, I get ǔ, but I should get ů. The same for
capital letters - I get Ǔ but I should get Ů.

The workaround I use is to set ~/.XCompose to
include "%S/en_US.UTF-8/Compose"
<dead_caron> <u>                        : "ů"   U016F # LATIN SMALL LETTER U
WITH RING ABOVE
<dead_caron> <U>                        : "Ů"   U016E # LATIN CAPITAL LETTER U
WITH RING ABOVE



This is already defined in /usr/share/X11/locale/iso8859-2/Compose , but one
would have to use iso8859-2 instead of UTF-8:
<dead_caron> <U>                        : "\331"        Uring
<dead_caron> <u>                        : "\371"        uring


(of course the binary value for Uring is different in iso8859-2 and UTF-8, so
it can't be fixed by straight copy/paste)


I don't understand the xkb too well, but I guess that there should be new
directory and file /usr/share/X11/locale/cs_CZ.UTF-8/Compose which would
contain something similar to what I have in ~/.XCompose

Please note that it does not seem to be logical to use dead_caron for ring
above u, but since we don't use ring above any other letter, and ǔ does not
exist in czech, it's very convenient to have it print ů. This way it works on
czech keyboards since ever.

It's interesting that I have hard time finding evidence that it should be set
this way.

For example
https://ufal.mff.cuni.cz/pdt/Support/czech-info.html
The ring accent can occur only above an "u" where, in contrary, no caron can
occur. That is why many keyboards use the sequence "dead caron" - "u"/"U" to
enter an "u" with ring above. However, other keyboards (including Microsoft
Windows, I guess) alocate a "dead ring" key elsewhere, often to shifted
semicolon, replacing the U.S. key for tilde.

https://lists.debian.org/debian-gtk-gnome/2003/11/msg00235.html




Thank you
__ 
  Vlad

Reproducible: Always

Steps to Reproduce:
1. Enable XIM input method by putting this into your .profile:
export GTK_IM_MODULE=xim
export QT_IM_MODULE=xim
export LC_ALL=cs_CZ.UTF-8

2. restart X so that the options take effect

2.5 start xterm

3. set czech keyboard map
setxkbmap "us,cz(qwerty)"  -option grp:shifts_toggle -option ctrl:swapcaps

4. switch into czech map by pressing both shifts together

5. Press dead_caron (shift + key "=" which should normally produce character
"+")

6. Press letter "u"

7. ǔ appears, but I should see ů
Actual Results:  
ǔ appears

Expected Results:  
I should see ů
Comment 1 James Cloos 2014-07-29 19:17:53 UTC
Based on this, libX11 needs to add a nls/cs_CZ.UTF-8 directory with a
Compose.pre which includes X11_LOCALEDATADIR/en_US.UTF-8/Compose and
overrides <dead_caron> <u> and <dead_caron> <U>.

I’m less certain of what the XI18N_OBJS and XLC_LOCALE.pre files should
look like.  In fi_FI.UTF-8, as an example, they are empty; the others
which include X11_LOCALEDATADIR/en_US.UTF-8/Compose have similar content
for XI18N_OBJS, but not so much for XLC_LOCALE.pre.
Comment 2 James Cloos 2014-07-29 19:29:04 UTC
Created attachment 103653 [details] [review]
Patch to add cs_CZ.UTF-8/Compose.pre

@Vladimír:  please test this patch; the compose change should just work, but I’m not sure about the other two files in the nls directories.

Since you mention that you use xim, you are better able to test that part of the patch than I.

I expect to push this to libX11 git.
Comment 3 vlmarek 2014-07-30 21:27:25 UTC
Hi,

thank you for your quick reply. I guess it's a patch agains xorg sources. At the moment I have no means of compiling xorg :( Nor I know what is the meaning of the two empty files ...
Comment 4 James Cloos 2014-07-30 22:45:18 UTC
> I guess it's a patch agains xorg sources.

The Compose files come from libX11.

I tested it a bit, but I do not use XIM so cannot test that.
Comment 5 James Cloos 2014-08-03 18:36:18 UTC
I pushed the change to the libX11 repository.

It still would help if someone who uses xim and cs_eCZ.UTF-8 were to
test it.
Comment 6 vlmarek 2014-08-06 13:29:05 UTC
I'm sorry I can't be more helpful testing it. The patch adds three files
 - cs_CZ.UTF-8/Compose.pre
 - cs_CZ.UTF-8/XI18N_OBJS (empty)
 - cs_CZ.UTF-8/XLC_LOCALE.pre (empty)

At my opensuse installation I don't have any *.pre files or XI18N_OBJS files. I can confirm that if I open the file in utf-8 encoding the characters are correct.

Thank you
-- 
  Vlad
Comment 7 Michal Srb 2015-11-19 12:15:15 UTC
I tested the current git HEAD of libX11 and it doesn't work out of the box: The records in nls/locale.dir.pre still point to en_US instead of cs_CZ. After correcting that the composite combination ˇ + u = ů works correctly.

(In reply to vlmarek from comment #6)
> At my opensuse installation I don't have any *.pre files or XI18N_OBJS
> files. I can confirm that if I open the file in utf-8 encoding the
> characters are correct.

The "Compose.pre" is converted into "Compose" during build.
Comment 8 vlmarek 2015-11-20 09:44:34 UTC
I'm confused, is there anything I can do to help with this issue?

Thanks!
__ 
  Vlad
Comment 9 James Cloos 2015-12-03 23:26:40 UTC
I pushed 33840a5465a2e5fecab520bfbdd2d1bd0a456f51 with:

-en_US.UTF-8/XLC_LOCALE:                        cs_CZ.UTF-8
+cs_CZ.UTF-8/XLC_LOCALE:                        cs_CZ.UTF-8

I forgot about nls/locale.dir.pre when I wrote the earlier patch.

And then rememberred nls/compose.dir.pre, and pushed 5f0da8311a61498edf073cc877f5b467bfd5f863, too.
Comment 10 vlmarek 2015-12-04 09:13:48 UTC
Thanks for taking care of this!

__ 
  Vlad
Comment 11 Jiří Klimeš 2016-10-10 13:36:47 UTC
(In reply to James Cloos from comment #9)
> I pushed 33840a5465a2e5fecab520bfbdd2d1bd0a456f51 with:
> 
> -en_US.UTF-8/XLC_LOCALE:                        cs_CZ.UTF-8
> +cs_CZ.UTF-8/XLC_LOCALE:                        cs_CZ.UTF-8
> 
> I forgot about nls/locale.dir.pre when I wrote the earlier patch.
> 
> And then rememberred nls/compose.dir.pre, and pushed
> 5f0da8311a61498edf073cc877f5b467bfd5f863, too.

I use dmenu in Arch Linux. After libX11 upgrade from 1.6.3 to 1.6.4 I spotted that dmenu crashed. Spending some time debugging showed that the problem is related to handling cs_CZ.UTF-8 locale in libX11.

$ LANG=cs_CZ.UTF-8 dmenu_run
$ warning: no locale support
... and after a keypress there is a crash (the same as in [1])

Reverting 33840a5465a2e5fecab520bfbdd2d1bd0a456f51 fixes the issue because XLC_LOCALE was not defined for the mentioned locale.
See the next comment for the patch.

[1] https://bbs.archlinux.org/viewtopic.php?id=217913
Comment 12 Jiří Klimeš 2016-10-10 13:38:32 UTC
Created attachment 127166 [details] [review]
[PATCH] nls: fix handling of cs_CZ.UTF-8 locale

Fixes locale.dir for cs_CZ.UTF-8 locale.
Comment 13 Nicky726 2016-10-20 20:10:59 UTC
A related burgreport: https://bugs.archlinux.org/task/51442

Summary:

Upgrading libx11 to version 1.6.4-1 breaks diacritics with LibreOffice:
- using a deadkey to write diacritics is no longer possible (only the character without diacritics appears),
- opening/saving files with filenames or path containing characters with diacritics is not possible (GUI error file does not exist)

Patching libx11 1.6.4 with the patch by Jiří Klimeš, the results are mixed:
- diacritical characters using a deadkey can now be written in LibreOffice,
- files with diacritics in name/path still cannot be opened in LibreOffice.
Comment 14 Jiri Slaby 2016-11-08 15:06:18 UTC
*** Bug 98219 has been marked as a duplicate of this bug. ***
Comment 15 Pinky 2016-11-21 22:51:41 UTC
Created attachment 128124 [details] [review]
fix handling of cs_CZ locale - remove reference to empty XLC_LOCALE
Comment 16 Pinky 2016-11-21 23:04:11 UTC
line (1) is duplicit

en_US.UTF-8/XLC_LOCALE:			cs_CZ.UTF-8
Comment 17 Marián Kyral 2016-12-09 10:06:31 UTC
(In reply to Pinky from comment #15)
> Created attachment 128124 [details] [review] [review]
> fix handling of cs_CZ locale - remove reference to empty XLC_LOCALE

Superb. Applying this patch fixed the issue with non working dead key in qt4, java applications and wine. Now I'm able to write "Š" by using dead key as before.

Thanks.
Comment 18 tuxayo 2016-12-14 18:04:17 UTC
I have the same issue with en_DK.UTF-8 on Arch Linux with crashes on dmenu.

If I understand correctly, the currently proposed patch fixes only the cs_CZ locale, is that correct?
Comment 19 GitLab Migration User 2018-08-10 20:55:45 UTC
-- GitLab Migration Automatic Message --

This bug has been migrated to freedesktop.org's GitLab instance and has been closed from further activity.

You can subscribe and participate further through the new bug through this link to our GitLab instance: https://gitlab.freedesktop.org/xorg/driver/xf86-input-keyboard/issues/16.

Use of freedesktop.org services, including Bugzilla, is subject to our Code of Conduct. How we collect and use information is described in our Privacy Policy.