Bug 1887 - libX11 locale defs severely adrift from glibc
Summary: libX11 locale defs severely adrift from glibc
Status: RESOLVED FIXED
Alias: None
Product: xorg
Classification: Unclassified
Component: Lib/Xlib (show other bugs)
Version: git
Hardware: All All
: high normal
Assignee: Xorg Project Team
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2004-11-22 08:42 UTC by Daniel Stone
Modified: 2006-05-09 01:59 UTC (History)
3 users (show)

See Also:
i915 platform:
i915 features:


Attachments
update locales to list from glibc 2.3.2 (large) (47.25 KB, patch)
2004-11-22 08:42 UTC, Daniel Stone
no flags Details | Splinter Review

Description Daniel Stone 2004-11-22 08:42:28 UTC
libX11's locale definitions are quite severely adrift from modern versions of
glibc, in terms of what they support.  Debian (and, subsequently, Ubuntu) has
had a patch to update this for quite a long time, with no ill effects; I believe
the impact on other platforms (e.g. Solaris) should be nil, as if the locale is
not supported by the underlying libc, XSupportsLocale() will still fail in the
same way.

Ubuntu/Debian 011a_recognize_glibc_2.3.2_locale_names.diff
Comment 1 Daniel Stone 2004-11-22 08:42:53 UTC
Created attachment 1328 [details] [review]
update locales to list from glibc 2.3.2 (large)
Comment 2 Roland Mainz 2004-11-22 08:49:58 UTC
Comment on attachment 1328 [details] [review]
update locales to list from glibc 2.3.2 (large)

Three things needs to be done:
1. Ienup Sung should review the patch
2. The patch needs to be commited to Xorg CVS trunk first before it can go into
the X11R6.8.x branch
3. xc/programs/Xserver/XpConfig/IMakefile needs to be sync'ed to include the
new locales listed in the patch...
Comment 3 Daniel Stone 2004-11-23 11:02:19 UTC
Comment on attachment 1328 [details] [review]
update locales to list from glibc 2.3.2 (large)

make roland less angry
Comment 4 Roland Mainz 2005-01-10 04:02:53 UTC
ienup:
ping!
Can you check whether the patch is OK for you, please ?
Comment 5 Daniel Stone 2005-06-04 05:21:45 UTC
... this bug is bitrotting ...
Comment 6 Daniel Stone 2005-06-14 23:26:39 UTC
I propose to commit this patch on Friday the 24th if there are no objections.
Comment 7 FreeDesktop Bugzilla Database Corruption Fix User 2005-06-30 17:37:15 UTC
Created attachment 3000 [details] [review]
Fix loop exit condition

This version is the one most likely to be in the next Debian version.
Comment 8 FreeDesktop Bugzilla Database Corruption Fix User 2005-06-30 18:13:22 UTC
Created attachment 3001 [details] [review]
Put FT_Done_Face and num_open_faces-- in _font_map_release_face_lock_held
Comment 9 FreeDesktop Bugzilla Database Corruption Fix User 2005-06-30 18:15:04 UTC
The Debian patch actually does several things in addition to the     
pure update to the glibc 2.3.2 list.  They're mostly described at the top of     
the patch.  It won't apply cleanly due to additional changes to xorg since    
6.8.2.    
    
I'd really like to see the additional aliases for various existing locales get    
in ASAP.  The sheer scale of these tends to break all other locale.alias    
patches and makes it very hard to submit them individually.  They're also    
isolated to the locale.alias file.  
  
Comment 10 FreeDesktop Bugzilla Database Corruption Fix User 2005-06-30 18:20:35 UTC
Created attachment 3002 [details] [review]
Treat a 0 reference count same as -1 in reference and destroy functions

The introduction of bs_BA on HEAD was totally misapplied to locale alias.  The
attachment fixes it.
This has been confounding my attempts to apply patches to locale.alias, which
is why it's being introduced in this (somewhat inappropriate) bug.  I would
send patches to a mailing list, but there doesn't seem to be an appropriate way
to do so, and it feels wrong to open another bug in this case.
Comment 11 FreeDesktop Bugzilla Database Corruption Fix User 2005-06-30 18:32:26 UTC
Created attachment 3003 [details] [review]
fixed version of the mesa pic patch

This (which applies after the bs_BA bugfix) adds all the new aliases to
existing locales and does nothing else.  All aliases are in use in Debian or
Ubuntu.

I believe it should be considered obvious and applied ASAP.  It will also mean
that the remaining patches from Debian to sync things to glibc 2.3.2 and fix
other locales bugs will be one hell of a lot more comprehensible to anyone
reading them.

(I have a version of this against 6.8.2 as well if anyone cares.)
Comment 12 FreeDesktop Bugzilla Database Corruption Fix User 2005-06-30 18:44:38 UTC
Created attachment 3004 [details]
Blank SVG file to use for testing.

This is the other change, which I also believe is totally obvious and
mechanical, which prevents other patches from being incomprehensible.  It makes

the Win32 aliases alias to properly capitalized versions of the canonical
locales.
Comment 13 FreeDesktop Bugzilla Database Corruption Fix User 2005-06-30 18:47:58 UTC
Created attachment 3005 [details] [review]
patch for warnings

Yeeargh, bogus bits got into the last copy of this.  Fixed.
Comment 14 FreeDesktop Bugzilla Database Corruption Fix User 2005-06-30 20:10:20 UTC
Created attachment 3006 [details] [review]
My version (derived from the Mozilla patch)

This last patch contains all the remaining Debian changes to locales, ported to
patch against HEAD.  I'm not actually sure all of these are right; I might
break it down into smaller bits at some point.	The point is that the
straightforward patches *before* this one (3002, 3004, 3005) make patches like
this one actually reviewable, whereas with all the patches merged together,
it's quite unreadable.
Comment 15 Daniel Stone 2005-07-18 19:42:05 UTC
So #3002, #3004, #3005 and #3006 all need to be applied?
Comment 16 FreeDesktop Bugzilla Database Corruption Fix User 2005-07-19 15:04:46 UTC
#3002, 3004, 3005 definitely need to be applied.   
   
I'm not sure that 3006 is entirely correct.  I'd hold off on it.  I think it's   
out of date w.r.t. Debian (viz. zh_* locales, which I screwed up and then 
fixed), and I think it has useless junk in it.  (For instance, why comment out 
pd_* rather than deleting it?) 
 
It also contains locale deletions (obsolete and non-ISO codes, some Euro 
variants for non-Euro-zone countries, and a few others), which have the 
potential to impact other platforms and so probably actually need to be looked 
at. 
 
I'll try to whip up a new version which doesn't have anything questionable in  
it.  Remind me.  
 
At any rate, after 3002/3004/3005 are committed, the diff to Debian will be so 
much more readable it won't be funny, and that will make it easier for me to 
tell what the heck is going on. 
 
Comment 17 Daniel Stone 2005-08-04 11:40:19 UTC
Nathaniel: consider yourself reminded.
Comment 18 Alan Coopersmith 2005-08-04 13:00:12 UTC
Patches 3002 & 3005 committed to CVS head for 6.9/7.0 RC1:

CVSROOT:	/cvs/xorg
Module name:	xc
Changes by:	alanc@gabe.freedesktop.org	05/08/03 19:51:30

Log message:
  2005-08-03  Alan Coopersmith  <alan.coopersmith@sun.com>
  
  	* nls/locale.alias
  	Bug #1887 <https://bugs.freedesktop.org/show_bug.cgi?id=1887>
  	Patch #3002 <https://bugs.freedesktop.org/attachment.cgi?id=3002>
  	libX11 locale defs severely adrift from glibc: locales bugfix for bs_BA
  	(From Debian via Nathanael Nerode)

Modified files:
      ./:
        ChangeLog 
      xc/nls/:
        locale.alias 
  
  Revision      Changes    Path
  1.1175        +8 -0      xc/ChangeLog
  1.8           +6 -11     xc/nls/locale.alias

CVSROOT:	/cvs/xorg
Module name:	xc
Changes by:	alanc@gabe.freedesktop.org	05/08/03 19:55:49

Log message:
  2005-08-03  Alan Coopersmith  <alan.coopersmith@sun.com>
  
  	* nls/locale.alias
  	Bug #1887 <https://bugs.freedesktop.org/show_bug.cgi?id=1887>
  	Patch #3005 <https://bugs.freedesktop.org/attachment.cgi?id=3005>
  	libX11 locale defs severely adrift from glibc: adding new aliases
  	(From Debian via Nathanael Nerode)

Modified files:
      ./:
        ChangeLog 
      xc/nls/:
        locale.alias 
  
  Revision      Changes    Path
  1.1176        +8 -0      xc/ChangeLog
  1.9           +209 -19   xc/nls/locale.alias
Comment 19 Dwayne Bailey 2005-08-05 19:11:01 UTC
Since there seems to be some action here to get locales updates.  Please check
https://bugs.freedesktop.org/show_bug.cgi?id=2268
which adds South African locales to X.
Comment 20 FreeDesktop Bugzilla Database Corruption Fix User 2005-08-07 21:45:38 UTC
Created attachment 3282 [details] [review]
Same fix we had for new freetype but cleaner

This adds all the new locales to locale.aliases, fixes a number of buggy
aliases, corrects the charset for Finland, removes the bogus nynorsk ny_
aliases (ny_ is Nyanja), removes the unusable nunacom aliases (nunacom can't be
used in X.org), removes the pd_ aliases (not an ISO language code and not
supported by glibc), and realphabetizes.

Everything in this is definitely correct and should be committed.  Debian has a
few extra tweaks which I'll post separately, which I'm not sure about Obsoletes
most previous patches.
Comment 21 FreeDesktop Bugzilla Database Corruption Fix User 2005-08-07 21:55:16 UTC
Created attachment 3283 [details] [review]
Security handler and plugins

These are the ones I'm not sure about.
* remove ee_ locales
* remove eo_ locales
* remove antique lt_LN comment
* remove euro and iso8859-15 locales for a bunch of non-Eurozone countries
* remove sh_HE.iso88592
* make Chinese(Signapore) and Chinese(Taiwan) point to canonical versions
rather than aliases
Comment 22 FreeDesktop Bugzilla Database Corruption Fix User 2005-08-07 22:15:33 UTC
Created attachment 3284 [details] [review]
do not unlink on win32

These are the corresponding locale.dir changes to go with 3282.  Sorry they're
separate.

This adds new locales, removes pd_ and bogus ny_ locales, fixes bugs,
alphabetizes; makes zh_SG.gbk refer to zh_CN.gbk since zh_SG.gbk doesn't exist
yet; and does a similar thing with zh_HK.UTF-8.  (Those two changes should be
removed if-and-only-if the real versions exist now.)

I believe this is all correct and should be committed.	It is all used in
Debian.  The removal of the eo_ locales is also in Debian but not in this
patch.
Comment 23 FreeDesktop Bugzilla Database Corruption Fix User 2005-08-07 22:34:25 UTC
Created attachment 3286 [details] [review]
Splash modified region

Add Compose entries for every locale in locale.dir which has Compose tables,
remove entries for locales not in locale.dir or whose charsets don't have
Compose tables, fix bugs, alphabetize, improve comments, replace ny_ (wrong)
with nn_ (right), attach zh_ Compose tables to canonical locales rather than
aliases, remove pd_*.  Debian also removed eo_*, but I didn't.

I believe this is all correct and should go in.  3282/3284/3285 should all go
in together.

Once this enormous clump goes in, we should probably take another pass at
comparing the locales against glibc.

(And Debian's patch should be reworked to do nothing but drop Esperanto.  ;-) 
I didn't feel comfortable doing that here because some other system may
actually support the Esperanto stuff, unlike the pd_ stuff which was clearly
broken.)
Comment 24 Alan Coopersmith 2005-10-02 15:46:11 UTC
Looks like the patches here got lost in the bugzilla disk death - are there any
more remaining patches that still need to be applied?   If so, please reattach.
Comment 25 Erik Andren 2006-05-09 06:25:56 UTC
(In reply to comment #11)
> Looks like the patches here got lost in the bugzilla disk death - are there any
> more remaining patches that still need to be applied?   If so, please reattach.

Looks that we are out of patches, close?
Comment 26 Daniel Stone 2006-05-09 18:59:12 UTC
i think they've all been committed, but who knows, really ...


Use of freedesktop.org services, including Bugzilla, is subject to our Code of Conduct. How we collect and use information is described in our Privacy Policy.