Bug 13277

Created attachment 12597 [details] [review]
default keympa with correct comma characters

In the patch, I left the wrong cedilla versions as optional for historical reasons, and made comma version with alt right default as there aren't commercially available keyboards with romanian layout. Still, that layout is available as "std" with correct comma characters.

Good, committed.

Sergey, please revert this commit. I see that the new variants have also been added to the base.xml.in and variantRename.lst files, those changes should also be reverted. I'm a bit worried about the fact that anyone can manage to change these files in xkeyboard-config as he/she desires by just attaching a patch to a bug report. They are often the object of technical, cultural and political debates. I think that the maintainers (last persons listed in the "symbols/*" files) should at least be consulted, especially when they are still active contributors. Please read on for details on this particular issue if you feel inclined...

Although probably well-intended, Alexandru Szasz doesn't fully understand the implications of his patch. Most people that do the localization of free software in Romanian are aware that "commabelow" diacritics are the right ones for *writing* Romanian, however there are still problems on the *reading* side, as the most common OS's do not fully support these diacritics. I've done my best to alleviate these problems in the free software world by adding the necessary glyphs to the DejaVu family of fonts three years ago, but until recently there wasn't any solution from Microsoft for the Windows family of operating systems and the fonts included with them. There is little use in writing with "correct" diacritics if most of the readers cannot read them...

However, as the increasingly present Windows Vista has full support for them and there is now an (unfortunately little known) hotfix[1] for Windows XP that patches the problem with the included fonts, the Romanian teams of translators are in the process of migrating to the "commabelow" diacritics on various levels. In regards to the default Romanian variant in xkeyboard-config, there was an endless debate[2] on this topic in the "diacritice" mailing list[3], which is "a place for people interested in Romanian localization of free software to discuss general localization issues not specific to a desktop or an operating system". 

As the maintainer of the Romanian variants in xkeyboard-config I have proposed as a target date for this change the 1.3 or 1.4 releases of xkeyboard-config and I think I have reached a certain consensus on this issue with most of the Romanian translators from the localization projects of Ubuntu, Debian, OpenOffice, Gnome, Xfce etc. Alexandru Szasz, team leader of the Romanian localization team for the Mozilla project, was also a member of this discussion group, he's still in the top 5 of the posters[4], but he chose to part away because of personal reasons.

Regarding Alexandru Szasz as a person, unfortunately it is not the first time that he is attempting something like that, I have been already bitten once in regards to the Romanian translation of Pidgin[5]. Luckily the developers of Pidgin directed his broken update to me, the maintainer, as usual. His patch to xkeyboard-config also has issues. Not taking into account the bliss with which he ignored the base.xml.in file and his sloppy indentation, he also introduced the pleonastic term of "Cedilla below" for describing one of the new variants.

Thank you for the attention.

 1. http://www.microsoft.com/downloads/details.aspx?FamilyID=0ec6f335-c3de-44c5-a13d-a1e7cea5ddea&DisplayLang=en
 2. http://groups.google.com/group/diacritice/browse_thread/thread/f2951391fc9651ae/a58999baea37c5e3
 3. http://groups.google.ro/group/diacritice
 4. http://groups.google.ro/group/diacritice/about
 5. http://developer.pidgin.im/ticket/1451

Lads, I am lost here. Could you please make some agreement first before I start reverting changes?

> be reverted. I'm a bit worried about the fact that anyone can manage to change
> these files in xkeyboard-config as he/she desires by just attaching a patch to
> a bug report. 
If you have better ideas on how to organize the process - I am open to suggestions.

> debates. I think that the maintainers (last persons listed in the "symbols/*"
> files) should at least be consulted, especially when they are still active
> contributors. 
Yes, that's good idea. But since that bugzilla is public and xkb maillist is public as well - I expected _active_ maintainers to be on alert..

Thanks for the information. It gives important background. But I still would prefer to have some sort of agreement, if it is possible.

First, my apologies for this awkward situation. I'm here to help, not to mess things up.

Note that with the modification I proposed, people can write documents with the
correct romanian diacritics. This is important to know because the romanian language IS standardized.

I have proposed this change as a response to requests from romanian Linux users
(e.g.: https://bugzilla.redhat.com/show_bug.cgi?id=337271 ) that were
complaining that they select Romanian keyboard layout but they write with the
wrong characters with cedilla underneath. I for one, can easily change the
keymap I use directly in xorg.conf.

IMHO, if you choose to keep the mapping with incorrect characters as default,
maybe there should be more opinions than just a few to support keeping an
incorrect default mapping.

Also note that the number of people who use Linux in romanian is quite low, and
out of those, the number that use the romanian keymap is lower, you can check
this by the progress and activity in various romanian teams that localize
various Linux distributions compared to french for example.

(In reply to comment #4)
> > I think that the maintainers (last persons listed in the "symbols/*"
> > files) should at least be consulted, especially when they are still active
> > contributors. 
> Yes, that's good idea. But since that bugzilla is public and xkb maillist is
> public as well - I expected _active_ maintainers to be on alert..

Well, I am on alert, sort of. I do checkout this Bugzilla from time to time, but if there would a way I could be announced when someone fills bugs against the Romanian layout, I'd be more than happy.
 
> Thanks for the information. It gives important background. But I still would
> prefer to have some sort of agreement, if it is possible.
 
I would very much want to agree with each and every Romanian translator, but the reality is that sometimes this is not feasible. Alexandru seems to be an energetic person with a "getting things done fast" attitude. While I do appreciate that in situations like a translation marathons, I am very much against hurried solutions when it comes to painful compromises like the default variant in the Romanian layout. 

Alexandru and the Red Hat bug reporter mention these days the SR13992:2004 standard that demands "commabelow" diacritics and cite the diacritice.sf.net project. Well, thank you, I was a member of the board that devised the standard and I've built that SF project from scratch to spread the knowledge about the issues of the Romanian localization of free software. Alexandru mentions that distributions now have the "commabelow" diacritics. Well, thanks again, I've learned how to patch font files and I've spent some days adding the needed characters and that was also back in 2004. That's also the year in which I've updated the xkeyboard-config Romanian layout with the variants demanded by the newly adopted standard.

Of course I am myself eager to have a default variant with "commabelow" diacritics in xkeyboard-config, but I'm painfully aware of the dangers of migrating too soon. Moreso, I have discussed this particular issue openly on the "diacritice" mailing list which unite Romanian translators from at least half a dozen major projects and in the end we have reached a certain consensus. And the decision was that it's still too early for changing the default layout in xkeyboard-config. Alexandru left that mailing list because of personal differences and to my knowledge as a maintainer of the list, he is the only one who did that. 

Now do I hope to get to agree with Alexandru on this simple, YES or NO, matter?   Not really, unfortunately... We had discussions on this very issue on a much larger scale on the "diacritice" mailing list and this is one of the things that practically divide the people involved in the Romanian localization.

Please feel free to ignore my opinion if it's wrong.

Personally, I simply cannot agree with this default wrong keymap that will just spread the wrong diacritics more than they are now.

The number of romanian internet users is estimated to be around 25% of the population. The number of romanian computer users is close to that. That is 5,5 millions from 22 millions. These numbers are growing, the rate should grow to 50% as in most developed european countries and will grow according to internet providers and economic development. I'm saying let's not give a headache to the next 5 million romanian googlers.

Sadly, there are very few people that use romanian diacritics 100% of the time. I'd estimate a mere 100 000, and that's a optimistic value.

Another interesting fact is that microsoft.com ( http://www.microsoft.com/romania/ , see the menu on the left for example) started to use the correct diacritics on the romanian website although in Windows XP you cannot see them unless you install an optional patch. Now are they crazy to take the chance of XP users visiting the website and not seeing the ș and ț's when Windows XP is the most used operating system in romania and yes, people do not install that optional patch? I don't think so, I think they're just doing what's right.


Statistcs also available at http://www.internetworldstats.com/stats4.htm#europe

Well, there are three kind of lies in the world: lies, damn lies and statistics. I could also argue using web statistics that +80% of the Romanians online use a version of Windows older than Windows Vista and given the +75% piracy rate in Romania most of them cannot apply the hotfix that solves the font problem in XP because it requires validation. Microsoft doesn't give a damn about these users, in fact this would rather be an argument for them to use the "commabelow" diacritics in their web pages, which they did.

I eagerly look out on the net for people using the "correct" Vista and OSX keyboard layouts with "commabelow" diacritics and what do I see? Bloggers complaining and desperately (in the case of OSX) looking for a way to write with "cedilla" diacritics in order to make their texts intelligible to other Romanians. And as usual, I see users cursing the writers that use "comabelow" characters without really understanding what's the problem. This is rather detrimental to the use of diacritics, because we all know it, the typical Romanian computer user doesn't use any kind of diacritics, be them "correct" or "wrong".

The default variant in xkeyboard-config is not about beliefs, it is about making it as easy as possible for the Romanians to write in their language. "Cedilla" diacritics have made that possible for a long time, all OS's have full support for them and +90% of the Romanians cannot tell the difference. The minority that knows about "correct" diacritics are a few clicks or keyboard strokes away from changing to a variant with "commabelow" diacritics in X.org and that was already achieved in 2004. But we are talking here about a *very* vocal minority, indeed... Alexandru, I wonder what your opinion would have been in 2004 when the new standard was adopted and there was virtually zero chances for someone to read "commabelow" diacritics in a standard OS? 

My opinion is that when a significant part of Romanian users (in my opinion, significantly more than the ~10% that we have now) will have a setup that allows them to read "commabelow" diacritics, then I'll agree to change the default layout. And I'm counting Linux and OSX users too here, because they got rid of this problem years ago. The compromise that was proposed and accepted by the majority on the "diacritice" list was the 1.3 or 1.4 release of xkeyboard-config, depending on the spread of fonts that allow the users to read the "commabelow" diacritics.

Lads, let me interrupt you for a sec. Just to check I am on the same page.

So, what I see so far is:
1. There are two approaches to diacritic characters: one is "classic", another is "modern" and "standardized". The old set of chars is using "cedillabelow", the new one is using "commabelow"

2. The "classic" approach at the moment seems to be more popular - it is used in all pre-Vista Windows. From Unicode POV, it is not exactly correct.

3. The "standard" approach is very new. The characters it produces are more correct from the Unicode POV. But these chars create problems for all Windows users earlier than Vista (except for XP with some patch available only to users of licensed copies).

Please correct me where I am wrong.

You're mostly right Sergey, with two minor corrections:

 * from an Unicode point of view both sets of diacritics are correct. But the only correct from a linguistic point of view for Romanian are the "commabelow" diacritics which were introduced very late, with ISO-8859-16.

 * I'll keep nitpicking about the "cedillabelow" term, it's a pleonasm, "cedilla" is a diacritical mark that can only be added *under* certain consonant letters.

>  * from an Unicode point of view both sets of diacritics are correct. But the
> only correct from a linguistic point of view for Romanian are the "commabelow"
> diacritics which were introduced very late, with ISO-8859-16.
Point taken.

>  * I'll keep nitpicking about the "cedillabelow" term, it's a pleonasm,
> "cedilla" is a diacritical mark that can only be added *under* certain
> consonant letters.
Ok, you are really nitpicking here:)

(In reply to comment #9)
> Lads, let me interrupt you for a sec. Just to check I am on the same page.
> 
> So, what I see so far is:
> 1. There are two approaches to diacritic characters: one is "classic", another
> is "modern" and "standardized". The old set of chars is using "cedillabelow",
> the new one is using "commabelow"
> 

Cedilla variant is not classic, it's just wrong, the romanian language never had s and t with cedilla bellow. Somebody made a mistake somewhere in the past and some people still support it for the sake of under 20% of people that would be affected.

I'm saying under 20%, because of the people that don't use Vista or Linux, yes, the majority, few write with diacritics and VERY few write with mac or Linux as you would expect. So few have to deal with characters that won't display correctly.

I think it's more important to write documents that print out correctly than articles on the web, content which may or may not be there in the future or can be corrected in the future.

Please note that, sadly, using romanian diacritics on computers is not so popular yet. Most of the people which use diacritics use it because they have to print something with diacritics. So give the valuable user the possibility to write correctly without frustration and stop spreading wrong characters. I might see kids that write by hand with cedilla if we continue to spread these characters.

(In reply to comment #12)

I see now how deep some of your misunderstandings are... Let me check them one by one.

> Cedilla variant is not classic, it's just wrong, the romanian language never
> had s and t with cedilla bellow. Somebody made a mistake somewhere in the past
> and some people still support it for the sake of under 20% of people that would
> be affected.

+80% of the Romanian computer users will be affected, I'll show you why below...

 
> I'm saying under 20%, because of the people that don't use Vista or Linux, yes,
> the majority, few write with diacritics and VERY few write with mac or Linux as
> you would expect. So few have to deal with characters that won't display
> correctly.

Alexandru, it's the readers that are affected, not the writers... +80% of the Romanian computer users cannot currently *read* the "commabelow" diacritics, because they don't have the glyphs in their fonts. And they have no clue what's missing because they are not aware of the difference between "cedilla" and "commabelow".

 
> I think it's more important to write documents that print out correctly than
> articles on the web, content which may or may not be there in the future or can
> be corrected in the future.

How would you print out those documents correctly if you don't have the necessary glyphs in the installed fonts? Have you thought about that? Or are you suggesting that documents never make it from one user to another?


> Please note that, sadly, using romanian diacritics on computers is not so
> popular yet. Most of the people which use diacritics use it because they have
> to print something with diacritics. So give the valuable user the possibility
> to write correctly without frustration and stop spreading wrong characters. I
> might see kids that write by hand with cedilla if we continue to spread these
> characters.
 
"The possibility to write correctly" already exists. The comma variant was added back in 2001 and all the layouts from the official SR13992:2004 standard were implemented back in 2004. Your little patch is only about changing the default layout to a variant that will make the ordinary clueless users confused and very frustrated. 

"Cedilla" diacritics are better than "no diacritics", and sadly "comma" diacritics have very little support in the most common OS's. For now, only people that know the difference and understand the implications should use the "commabelow" diacritics. +90% of Romanian users cannot tell the difference between "cedilla" and "commabelow", so don't you worry about the kids, nobody has started to write on paper with "cedilla" diacritics in all those years of using "cedillas" in computer-related documents.

(In reply to comment #13)
> (In reply to comment #12)
> 
> I see now how deep some of your misunderstandings are... Let me check them one
> by one.
> 
> > Cedilla variant is not classic, it's just wrong, the romanian language never
> > had s and t with cedilla bellow. Somebody made a mistake somewhere in the past
> > and some people still support it for the sake of under 20% of people that would
> > be affected.
> 
> +80% of the Romanian computer users will be affected, I'll show you why
> below...

Please, let's stick to the subject and leave out the personal analysis.
Take a look at the romanian forums. Take a look at the romanian blogs. More sadly take a look at romanian online newspapers. How many of them use diacritics in the content ? Under 20%, not 80%. I'd sure wish it would be 80%, but most sites don't even allow posting with diacritics, including the most popular ones. For example hotnews.ro, myjob.ro, bestjobs.ro, etc.

> 
> How would you print out those documents correctly if you don't have the
> necessary glyphs in the installed fonts? Have you thought about that? Or are
> you suggesting that documents never make it from one user to another?
> 
> 

We are talking about xorg and Linux here right? I talked about printing. Because that's where diacritics are most used. To be shown on newspapers, television and commercials.

You are overgrowing the effect.

Why didn't you just proposed to the romanian academy to adopt the cedilla instead of comma if most romanians are to stupid (in your opinion) to tell the difference between a thing that lies under the letter and a thing that is tied to the letter ?

Another couple of questions, lads:

1. What is the _default_ Romanian layout in Vista?
2. What is the _default_ Romanian layout in MacOSX 10.4 and 10.5?

(In reply to comment #14)
> Please, let's stick to the subject and leave out the personal analysis.
> Take a look at the romanian forums. Take a look at the romanian blogs. More
> sadly take a look at romanian online newspapers. How many of them use
> diacritics in the content ? Under 20%, not 80%. I'd sure wish it would be 80%,
> but most sites don't even allow posting with diacritics, including the most
> popular ones. For example hotnews.ro, myjob.ro, bestjobs.ro, etc.

Well, the subject of this bug report is the default variant in
xkeyboard-config... That has very little to do with the above situation.

[snip]

> Why didn't you just proposed to the romanian academy to adopt the cedilla
> instead of comma if most romanians are to stupid (in your opinion) to tell the
> difference between a thing that lies under the letter and a thing that is tied
> to the letter ?

Please cut the zealotry. I didn't say that Romanians are stupid. I am one of
them. But I know (and I'm quite sure you do too) that very few of us know the
difference between "commabelow" and "cedilla". So most of us don't realize
what's wrong when our computers cannot display the "commabelow" diacritics.

Hello all,

As a Romanian user and native speaker, I fully support Alexandru's position on the matter. I am a Network Administrator too and I deal, day by day, with a WAN of about 1000 computers.

If we'll wait for „other mainsteam OSes” to have correct support and standards compliance, we will wait until 3012, Anno Domini.

Please don't forget that there is a Romanian National Standard, SR 13392:2004, that started to bring some light into Romanian-specific system settings (the keyboard part). It is *very* important that a correctly mapped Romanian keyboard don't generate wrong Unicode codes - at least from now on !

IMHO, GNU/Linux must pe much more aggressive in imposing on the marked this kind of matters, instead of trying to mimic old Windows bugs, for the sake of compliance. If we continue to assume this „challenger” position, we will remain on it for good !

The cedilla-comma matter is an old, uncorrected Microsoft bug and it must become clear for everybody that *they* have to assume it !

However, Microsoft corrected this in Windows Vista, so we are talking here only about the pre-Vista versions of Windows (please see 

http://www.secarica.ro/html/s-uri_si_t-uri.html  and http://www.secarica.ro/html/info_winvista.html

both in Romanian)


Regards,
Răzvan

(In reply to comment #15)
> Another couple of questions, lads:
> 
> 1. What is the _default_ Romanian layout in Vista?
> 2. What is the _default_ Romanian layout in MacOSX 10.4 and 10.5?
> 

In Vista there is only one layout, "Romanian Legacy" that has the cedillas.
Besides that there's "Romanian standard" and "Romanian programmers" which have
commas. I don't know if there's a default in english Vista, if you add it
through control panel you have to choose which one you want to add of those
three.

Judging by the name I'd guess in romanian vista or if you can choose romanian
during installation Romanian Standard is the default, that is the romanian
layout for keyboards with diacritics on them.

If no one has a os x at hand, I'll provide the answer later on.

Also, additional question:

Do I understand right that only 2 characters are unreadable in unpatched XP (and earlier versions): t and s with comma/cedilla? How frequent are these characters in Romanian? Is the text readable/understandable without them?

Hello,


IMHO, the *default* setting for any Linux workstation should be:

- keyboard layout according to Romanian National Standard SR 13392:2004 (please
see in Wikipedia for actual keyboard pictures);

- the "secondary" layout (as designated in the standard) should be the default;

- keyboard should generate the comma-versions of s,S,t and T;


All other programs in a GNU/Linux distro should take this into account (mainly
the fonts packages, in both text-mode and X).


Regards,
Răzvan

(In reply to comment #19)
> Also, additional question:
> 
> Do I understand right that only 2 characters are unreadable in unpatched XP
> (and earlier versions): t and s with comma/cedilla? How frequent are these
> characters in Romanian? Is the text readable/understandable without them?
> 

There are actually 4 characters: s, t, S and T witc commas/cedillas.

s with comma/cedilla below is equvalent of English „sh” sound (as in "show", "shine").

For t with comma/cedilla below is harder to get an English equivalent (aproximately "tz" as in "tzar") 

This sounds are very frequent in Romanian, but, in writing with an English keyboard, many Romanian users ommit them an use plain s, t, S and T.

Răzvan

(In reply to comment #19)
> Also, additional question:
> 
> Do I understand right that only 2 characters are unreadable in unpatched XP
> (and earlier versions): t and s with comma/cedilla? How frequent are these
> characters in Romanian? Is the text readable/understandable without them?
> 

In unpatched windows xp, s and t with cedilla bellow are readable everywhere if the regional's settings are set to romanian. s and t with comma bellow are not available and usually users see squares instead. Yes, the text is readible, probably the most used diacritics are ă and î in romanian.

If the regional settings are not set to romanian, which case is the majority of people, there are many situations where no diacritic is shown correctly especially in the folder/file names and program interface (due to windows codepage usage).

(In reply to comment #22)

Some corrections:

> In unpatched windows xp, s and t with cedilla bellow are readable everywhere if
> the regional's settings are set to romanian. s and t with comma bellow are not
> available and usually users see squares instead. Yes, the text is readible,
> probably the most used diacritics are ă and î in romanian.

They will *always* see squares, usually is an unneeded ambiguation. You'll have to go to unusual lengths to replace the installed fonts with corresponding patched fonts without applying the hotfix, The text would be bearly readable, but let's face it, most readears will give up.


> If the regional settings are not set to romanian, which case is the majority of
> people, there are many situations where no diacritic is shown correctly
> especially in the folder/file names and program interface (due to windows
> codepage usage).

There are *some* situations where diacritics are not shown correctly and those are usually bugs in third-party programs. Most of the time "cedilla" diacritics are not problematic in any supported version of Windows as they are part of the ISO-8895-2 codepage like the diacritics from Polish, Hungarian, Czech etc. I'm not counting the issue regarding mapping the filenames to the local codepage, that's another problem, common to all ISO-8859-2 languages.

(In reply to comment #23)
> (In reply to comment #22)
> 
> Some corrections:
> 
> > In unpatched windows xp, s and t with cedilla bellow are readable everywhere if
> > the regional's settings are set to romanian. s and t with comma bellow are not
> > available and usually users see squares instead. Yes, the text is readible,
> > probably the most used diacritics are ă and î in romanian.
> 
> They will *always* see squares, usually is an unneeded ambiguation. You'll have
> to go to unusual lengths to replace the installed fonts with corresponding
> patched fonts without applying the hotfix, The text would be bearly readable,
> but let's face it, most readears will give up.

Not true, most users ask someone with more knowledge to fix it. 

They will always see squares if they don't apply the hotfix that is presented on the microsoft.com romania front page.

Overgrown again. How many people give up reading a page because one or two letters are not shown correctly? People are more stubborn than that.

> 
> 
> > If the regional settings are not set to romanian, which case is the majority of
> > people, there are many situations where no diacritic is shown correctly
> > especially in the folder/file names and program interface (due to windows
> > codepage usage).
> 
> There are *some* situations where diacritics are not shown correctly and those
> are usually bugs in third-party programs. Most of the time "cedilla" diacritics
> are not problematic in any supported version of Windows as they are part of the
> ISO-8895-2 codepage like the diacritics from Polish, Hungarian, Czech etc. I'm
> not counting the issue regarding mapping the filenames to the local codepage,
> that's another problem, common to all ISO-8859-2 languages.
> 

If you not set romanian in regional settings, I must repeat myself, this is the case of most romanian users, you won't see diacritics in non-unicode programs, windows applies the iso-8859-1 code page or whatever is set in regional settings.

(In reply to comment #24)
> Not true, most users ask someone with more knowledge to fix it. 
> 
> They will always see squares if they don't apply the hotfix that is presented
> on the microsoft.com romania front page.

Well, I work in a department of admins and I have yet to see a Windows network administrator that knows about this problem. Let aside the regular power user that is asked about such things. Hope is that this will change as Vista is slowly adopted, MS Romania also does it's best to alleviate the problem (for licensed users, that is). 


> If you not set romanian in regional settings, I must repeat myself, this is the
> case of most romanian users, you won't see diacritics in non-unicode programs,
> windows applies the iso-8859-1 code page or whatever is set in regional
> settings.

Well, that is the difference between "many situations" as you stated initially and "some situations". Most of the time, users do not have a problem with the "cedilla" diacritics as they encounter them in web pages, e-mails and office documents. They rarely encounter them in filenames or in third-party non-unicode programs run in an environment that has a different 8 bit codepage.

Hi,

I'm one of the early (2001) contributors to the RO keymap.
I agree with Misu that we should wait on changing this to the correct model for 6-9 mo, until more people adopt Vista and they increase the popularity of the hotfix for XP. We have nothing to lose on this, as documents are rendered correct *now* on both XP and Vista.

Thanks

Hello,

I am the coordinator for the Romanian Gnome Translation team for some years now, and a free software translator for even more years. I must agree with Mişu Moldovan on this one: I remember of the first days of KDE/Gnome translations, when just because they were unaware of how to switch to a font with Romanian diacritics, most Romanian linux users switched to English interface. With the proposed default setup by Alexandru Szasz, we'll just go back to those times: you may have problems writing to a blog, you may have problems sharing a doc with a friend, you may have problems with you're friend reading your email. So if you don't know how to fix this, in the end you'll just give up and switch to good old English. And we don't want that.

I'm also a network administrator, and I know a lot of people which are also network administrators. Just because we're admins doesn't mean that we know how to fix this. Also, this is just the type of problems that's usually disconsidered by the netadmins. Internet not working => important, diacritics not correctly displayed => get lost.

So please stick with the safe default for now, and do the switch when the time is right. Which is not now, by any means.

Hello,


Well, I think this „sit and wait” approach from an important part of the Romanian development/localization team is the main reason why we don't have a correct implementation now. Sorry ! No offense, they do a *very good* work, but we have very little long-time perspective about our national issues... This IT problem is still unsolved since the '80s, when *other people* (non-Romanians) have chosen a (suboptimal) solution for us, because of our traditional lack of decision at national level...

We face a real problem and we still want take no action.

As the present localization team leader, Alexandru is the first that breaks the „sit and wait” tradition on the matter.

Regarding Marius' saying „let's wait for Vista to become mainstream, to have the compatibility problem solved first”, this is a „goal” I deeply wish we'll never reach (please see http://badvista.fsf.org for a reason why).


Given the Vista's and Office 2007's *big* compatibility break with previous versions, this is an *optimal* moment to try to turn national momentum to free software solutions, following the European mainstream current. If we *really* want that...

The "price" is just the need for us, Romanian IT teams, to install a few more Windows XP patches... And make apparent to every manager that *they* (Microsoft systems) are out of standards...


Regards,
Răzvan

Also please be aware of the danger that lies ahead:

- we have now around ten millions of web pages with s and t with cedilla in search indexes
- if we keep letting s and t with cedilla be typed, the number will grow rapidly
- when we'll finally switch, there will be a BIGGER number of pages that won't be found when search term contains s and with comma.

There are many big sites that store romanian content with s and t diacritics that can change to correct ones (e.g. Wikipedia, official sites), however there are many which won't be maintained or modified anymore.

I don't see the point of discussing this so much. The characters with cedilla are simply WRONG. Even if there wouldn't be fonts with the comma characters, the characters with cedilla would still be wrong.

(In reply to comment #28)
[snip]
> As the present localization team leader, Alexandru is the first that breaks the
> „sit and wait” tradition on the matter.

Letting aside most of the zealotry, let's clear the facts a bit: Alexandru is the newly appointed team leader of the Romanian localization of Fedora.

Regarding the "sit and wait" part, I would like to know where were you when the new standard was discussed back in 2003-2004, I haven't seen you there. Nor did I heard of you in regards to building, maintaining or testing the Romanian keyboard layouts. I'm pretty sure you haven't been involved in the Romanian localizations either. You are just a Linux user with strong opinions. 

Interesting that the first three relevant hits of a google search for your identity yields a mail, an article and a blog post written by you with "cedilla" diacritics: 
http://www.mail-archive.com/tic-lobby@agora.ro/msg00395.html
http://www.agora.ro/index.php?qs_sect_id=997
http://rsandu.myblog.ro/post/2007/10/02/virgil-vin-an-un-sf-r-it-tragic

Your http://www.linuxwill.go.ro project also uses the "broken" diacritics. What side are you anyway and what are you trying to prove?

(In reply to comment #30)
> (In reply to comment #28)
> [snip]
> > As the present localization team leader, Alexandru is the first that breaks the
> > „sit and wait” tradition on the matter.
> 
> Letting aside most of the zealotry, let's clear the facts a bit: Alexandru is
> the newly appointed team leader of the Romanian localization of Fedora.
> 
> Regarding the "sit and wait" part, I would like to know where were you when the
> new standard was discussed back in 2003-2004, I haven't seen you there. Nor did
> I heard of you in regards to building, maintaining or testing the Romanian
> keyboard layouts. I'm pretty sure you haven't been involved in the Romanian
> localizations either. You are just a Linux user with strong opinions. 
> 
> Interesting that the first three relevant hits of a google search for your
> identity yields a mail, an article and a blog post written by you with
> "cedilla" diacritics: 
> http://www.mail-archive.com/tic-lobby@agora.ro/msg00395.html
> http://www.agora.ro/index.php?qs_sect_id=997
> http://rsandu.myblog.ro/post/2007/10/02/virgil-vin-an-un-sf-r-it-tragic
> 
> Your http://www.linuxwill.go.ro project also uses the "broken" diacritics. What
> side are you anyway and what are you trying to prove?
> 

We are all very greatful to anyone that contributed to the adopted standard.
There's nothing to appreciate about anyone that does not want to apply the standard although the standard is from 2004. We are in 2007! Do you really think that 10 months (as you and a few others suggested) will make a difference ?

Let's try to be more open. This is open source and I'm beginning to feel like I'm begging microsoft to follow the romanian standard.

Alexandru, I'm glad in a certain way that translators like you and the early adopters like Răzvan Sandu have learned about the "comma" layouts and that some of the Romanians are starting to write correct Romanian when using a computer. I can't fully explain why this is happening so late, but it matches older previsions regarding this matter and shows signs that people are slowly becoming aware of some of the problems with the Romanian diacritics.

However, my opinion is that decisions like this one regarding the default layout in xkeyboard-config should not be taken by the vocal minority of early adopters. So, as the maintainer of this layout, I have opened a discussion in the biggest group of Romanian translators of free software, the "diacritice" mailing list. 

Taking into account the inevitable issues of this change we have settle for the 1.3 or 1.4 release of xkeyboard-config. The whole discussion, which refers to even more arguments on both sides may be consulted at http://groups.google.com/group/diacritice/browse_thread/thread/f2951391fc9651ae/a58999baea37c5e3   (Romanian only).

(In reply to comment #28)
> Hello,
> 
> 
> Well, I think this „sit and wait” approach from an important part of the
> Romanian development/localization team is the main reason why we don't have a
> correct implementation now. Sorry ! No offense, they do a *very good* work, but
> we have very little long-time perspective about our national issues... This IT
> problem is still unsolved since the '80s, when *other people* (non-Romanians)
> have chosen a (suboptimal) solution for us, because of our traditional lack of
> decision at national level...
> 
> We face a real problem and we still want take no action.
> 

Never confuse motion with action.

(In reply to comment #32)
> Alexandru, I'm glad in a certain way that translators like you and the early
> adopters like Răzvan Sandu have learned about the "comma" layouts and that
> some of the Romanians are starting to write correct Romanian when using a
> computer. I can't fully explain why this is happening so late, but it matches
> older previsions regarding this matter and shows signs that people are slowly
> becoming aware of some of the problems with the Romanian diacritics.
> 
> However, my opinion is that decisions like this one regarding the default
> layout in xkeyboard-config should not be taken by the vocal minority of early
> adopters. So, as the maintainer of this layout, I have opened a discussion in
> the biggest group of Romanian translators of free software, the "diacritice"
> mailing list. 
> 
> Taking into account the inevitable issues of this change we have settle for the
> 1.3 or 1.4 release of xkeyboard-config. The whole discussion, which refers to
> even more arguments on both sides may be consulted at
> http://groups.google.com/group/diacritice/browse_thread/thread/f2951391fc9651ae/a58999baea37c5e3
>   (Romanian only).
> 

In that discussion I see there are even more people that think that the correct comma should be default. 

There are 7 people discussing there. 3 say that we should change to the correct comma as default now, 3 do not express any preference, and 1 (you) say that we should postpone.

Correct me if I'm wrong.

I wish I could read Romanian...;)

(In reply to comment #34)
> (In reply to comment #32)
[snip]
>http://groups.google.com/group/diacritice/browse_thread/thread/f2951391fc9651ae/a58999baea37c5e3
> >   (Romanian only).
> > 
> 
> In that discussion I see there are even more people that think that the correct
> comma should be default. 
> 
> There are 7 people discussing there. 3 say that we should change to the correct
> comma as default now, 3 do not express any preference, and 1 (you) say that we
> should postpone.
> 
> Correct me if I'm wrong.

You must be a bit confused, maybe because the discussion also covered replacing the default US-alike layout with a true Romanian one. People that have agreed in that thread to preserve the cedillas in the default layout of xkeyboard-config until the 1.3 or 1.4 releases include:
 * Cristian Secară, translator, the initiator and the main contributor to the new Romanian standard
 * Eddy Petrişor, the coordinator of the Debian localization (with the observation that I should provide Debian developers with a patch if Lenny is frozen before that)
 * Jani Monoses, main Xubuntu developer (with the condition that I fix Ubuntu bug #108057 [1], which I did in xkeyboard-config bug #11945 [2])

There was no coordinator or major contributor opposing this in the final discutions. Moreso, you have in this bug report two other major figures on the same side:
 * Marius Andreiana, former contributor to the Romanian X layout and former coordinator of the GNOME translation
 * Mugurel Tudor, coordinator of the GNOME localization.

That makes it 6 to 1. You, the coordinator of the Fedora and Mozilla localizations, are the only major contributor to Romanian localization that pushes for this change now. 

 1. https://bugs.launchpad.net/ubuntu/+source/xkeyboard-config/+bug/108057/
 2. https://bugs.freedesktop.org/show_bug.cgi?id=11945

(In reply to comment #35)
> I wish I could read Romanian...;)
> 

(In reply to comment #36)
> (In reply to comment #34)
> > (In reply to comment #32)
> [snip]
> >http://groups.google.com/group/diacritice/browse_thread/thread/f2951391fc9651ae/a58999baea37c5e3
> > >   (Romanian only).
> > > 
> > 
> > In that discussion I see there are even more people that think that the correct
> > comma should be default. 
> > 
> > There are 7 people discussing there. 3 say that we should change to the correct
> > comma as default now, 3 do not express any preference, and 1 (you) say that we
> > should postpone.
> > 
> > Correct me if I'm wrong.
> 
> You must be a bit confused, maybe because the discussion also covered replacing
> the default US-alike layout with a true Romanian one. People that have agreed
> in that thread to preserve the cedillas in the default layout of
> xkeyboard-config until the 1.3 or 1.4 releases include:
>  * Cristian Secară, translator, the initiator and the main contributor to the
> new Romanian standard
>  * Eddy Petrişor, the coordinator of the Debian localization (with the
> observation that I should provide Debian developers with a patch if Lenny is
> frozen before that)
>  * Jani Monoses, main Xubuntu developer (with the condition that I fix Ubuntu
> bug #108057 [1], which I did in xkeyboard-config bug #11945 [2])
> 
> There was no coordinator or major contributor opposing this in the final
> discutions. Moreso, you have in this bug report two other major figures on the
> same side:
>  * Marius Andreiana, former contributor to the Romanian X layout and former
> coordinator of the GNOME translation
>  * Mugurel Tudor, coordinator of the GNOME localization.
> 
> That makes it 6 to 1. You, the coordinator of the Fedora and Mozilla
> localizations, are the only major contributor to Romanian localization that
> pushes for this change now. 
> 
>  1. https://bugs.launchpad.net/ubuntu/+source/xkeyboard-config/+bug/108057/
>  2. https://bugs.freedesktop.org/show_bug.cgi?id=11945
> 

I don't think I have a crown over my head because I am the coordinator for Fedora, Mozilla and Open Office. I'm doing it because nobody wanted to or had the time to and because I want to give other less tehnically gifted the chance to contribute.

6 people is not a number that can justify to not follow the standard. IMHO, even a 100 isn't compared to 5 millions.

I want to remind that I do not ask to remove the wrong cedilla layout, I just want the correct comma layout to be the default. People who don't want to use the standard should take the trouble to change the keymap, not the other way around.

To respond to a previous question, Windows Vista has the correct commas default, as well as MacOS. Windows Vista includes a legacy layout with cedillas, MacOs doesn't.

(In reply to comment #37)
> 6 people is not a number that can justify to not follow the standard. IMHO,
> even a 100 isn't compared to 5 millions.

Turning the numbers around like this is just childish. I'm only counting the major contributors to the Romanian localization of free software. The world is full of zealotry and vocal users like our Razvan Sandu here.

However, being the only major Romanian translator to strongly recommend this change should make you wonder. It's not that you have a little secret that we don't know, quite the opposite, you are fairly new to this problem and to l10n in general.

Regarding Vista and OSX, I wish that the OS market would be 80% Linux, then we would have changed the default layout back in 2004 without caring much about users of other OS's. The sad situation is that Vista and OSX are also insignificant, the vast majority of Romanian users are using XP and they have major issues with "commabelow" diacritics as shown above. However, I have already noticed Vista and OSX Romanian users that are posting instructions on how to revert to a "cedilla" layout or create one from scratch. It's sad but true, reality bites...

(In reply to comment #35)
> I wish I could read Romanian...;)

Sergey, I wish you were a Romanian, that would have made everything much simpler.  But I appreciate the fact that you're genuinely interested in the deeper aspects of this change and I hope we'll sort this out sooner rather than latter, we all have plenty of work to do in localizing free software.

However, I don't understand why any external contributer may overwrite an existing layout in xkeyboard-config... I've been building and maintaining these layouts for years and it's not that I've been ignoring bugs that were reported, I have contributed patches even after I've implemented all the necessary layouts as described in the new Romanian standard (the "commabelow" layouts) and as demanded by real needs (the "cedilla" layouts). I should have been asked about this beforehand, the waste of time and energy would never had happened.

As a conclusion, please revert this hasted patch, there is a consensus in the world of Romanian l10n and that is to not make this change now, but in about 6 to 9 months from now. I will get back with a saner patch, most probably for the 1.3 release.

(In reply to comment #38)
> (In reply to comment #37)
> > 6 people is not a number that can justify to not follow the standard. IMHO,
> > even a 100 isn't compared to 5 millions.
> 
> Turning the numbers around like this is just childish. I'm only counting the
> major contributors to the Romanian localization of free software. The world is
> full of zealotry and vocal users like our Razvan Sandu here.
> 

Please stop judging people here. You are in no position to do that and this is not the place for that. Put your opinions and arguments here, no extras please, this subject is getting hard to follow if we put in more than plain arguments. I'm sure the asignee wants to simply read the arguments, not personal appreciations.

This subject is just a matter of respecting a standard or not. You and a few others say nay, I say yay. We each put our arguments here.

It's just my opinion that to approve a wrong thing, you should have a large number of people that agree to that. If such opinion can't be obtained, just follow the standard. Or you should change the worng thing to right

> Sergey, I wish you were a Romanian, that would have made everything much
Indeed. But trust me we have our own problems in Russia:)

> However, I don't understand why any external contributer may overwrite an
> existing layout in xkeyboard-config... 
Well, his arguments seemed sensible (at least at that point of time). I am not saying they are not now. I am still listening to both of you.

> and as demanded by real needs (the "cedilla" layouts). I should have been asked
Yes, there might be some overlooking on my side. Since I do not have formal "Layout-->Maintainer" list (which may be I should have), there can be some clashes like this (though it is the first on my memory which caused such a hot discussion).

> to 9 months from now. I will get back with a saner patch, most probably for the
> 1.3 release.
1. In any circumstances (sorry, I have not made my mind yet - we still have time before next release), I would prefer not to "just revert it" - because I like the new naming convention for variants.
2. How would you explain the fact that both Vista and MacOS found it was acceptable to use new, commabelow way (if it is true)? Microsoft and Apple are usually reasonably conservative, you know...

Thanks

(In reply to comment #41)
> 1. In any circumstances (sorry, I have not made my mind yet - we still have
> time before next release), I would prefer not to "just revert it" - because I
> like the new naming convention for variants.

Please take into account that a name like "cedillabelow" is wrong both linguistically (a pleonasm) and technically (cedilla being a diacritical mark that can only be added under certain consonant letters). Thus I fully oppose the name of "cedillabelow". This unfortunate name and the name of "std_cedilla" for what is now "std" were introduced by Alexandru for the switch regarding the default layout, there is no new layout added. And there is none removed because they are all needed in the real world. Not for "historical reference" as Alexandru commented it shouting "WRONG" in the "ro" file, but for very real needs. Hmmm, it seems the more I look at his little patch, the more I dislike it... Again, Alexandru, I think it's childish to shout "WRONG" in the comments included in the xkeyboard-config file.


> 2. How would you explain the fact that both Vista and MacOS found it was
> acceptable to use new, commabelow way (if it is true)? Microsoft and Apple are
> usually reasonably conservative, you know...

Microsoft has made two important changes in Vista, they replaced the old broken QWERTZ layout ("winkeys" in xkeyboard-config) with a QWERTY one ("academic" in xkeyboard-config) and they also changed from "cedilla" to "commabelow" diacritics. Luckily, they were also forced[1] by the European Union to fix the font problem for XP prior to getting Romania and Bulgaria in the EU, which they did with the aforementioned hotfix. The catch is that the hotfix is optional and requires validation. Couple this with the fact that the vast majority of the Romanian users are not aware of the "cedilla" versus "commabelow" issue and you get the present situation, where ~90% of the installations cannot render the "commabelow" diacritics. Worse is that, when they encounter the little squares that replace the 4 missing glyphs in their fonts, they blame it on the writer, "knowing" that they can read Romanian with diacritics in their installation (what they don't know is that the more common Romanian diacritics, present in their fonts, use cedillas and the text with squares uses "commabelow" diacritics).

That is why we advocate waiting on this. We had "commabelow" layouts since a long time ago but the default layout has always used "cedilla", because regular users are not aware of these issues and chances are still very high that their diacritics will not be rendered  on other computers. This is bound to change with the rising levels of adoption for Vista, OSX and Linux.

I don't know much about Apple, except that they have a really weird layout that nobody has bothered to implement in xkeyboard-config, the old xkbdata or the Linux console kbd. Moreso, Romanian Mac users have built their own layouts[2][3][4] to make it usable to write diacritics both from an ease of use point of view and regarding compatibility with most of the Romanian users of other OS's. In three of the four links included below there are also references to the pain of writing with "commabelow" diacritics these days (again, in Romanian).

 1. http://blogs.msdn.com/michkap/archive/2006/11/19/1104093.aspx
 2. http://www.macuser.ro/index.php/forums/viewthread/1725/P0/
 3. http://andreimaxim.ro/2007/3/20/keylayouts_pentru_mac_os_x/
 4. http://www.picsel.ro/b/2007/10/03/diacritice-pe-osx/

Yes, the vast majority of romanian users now uses Windows XP.
Yes, the majority of them use it without a license.
Now, are we supposed to support users that use the operating system without a license?
I think not; we should tell them to go buy a vista romanian license, the first microsoft OS that is fully localized in romanian. Or we should tell them to use Linux because it's free and they can write/read with diacritics.
Yes, it may be hard to do this, but it's the *right* thing to do.

Let's not get into zealotry again. I don't care much about the distribution of desktop OS's or about the fact that XP users have a license or not, I care about two specific issues:
 * how many Romanian users have major problems reading the "commabelow" diacritics (currently ~90%)
 * how many Romanian users are aware of the problems with "commabelow" diacritics and thus would be incited to look for solutions (currently, almost none).

I've done my share of work in the last three years to improve both of these issues by patching the DejaVu set of fonts (which is now default in most distributions) and by starting and coordinating the http://diacritice.sf.net project and it's associated "diacritice" mailing list on Google Groups. It hurts to see someone hit and run, like you did on the "diacritice" mailing list almost a year ago, in the Pidgin tracker 6 months ago and now in the xkeyboard-config bugzilla.

> Please take into account that a name like "cedillabelow" is wrong both
> linguistically (a pleonasm) and technically (cedilla being a diacritical mark
> that can only be added under certain consonant letters). Thus I fully oppose
> the name of "cedillabelow". This unfortunate name and the name of "std_cedilla"
> for what is now "std" were introduced by Alexandru for the switch regarding the
> default layout, there is no new layout added. And there is none removed because
> they are all needed in the real world. Not for "historical reference" as
> Alexandru commented it shouting "WRONG" in the "ro" file, but for very real
> needs. Hmmm, it seems the more I look at his little patch, the more I dislike
> it... Again, Alexandru, I think it's childish to shout "WRONG" in the comments
> included in the xkeyboard-config file.
Ok, if you don't like "cedillabelow", I am open to any other consistent naming schema. It is just the old one did not look logical to me. The point about "WRONG" word is completely valid, I'll remove it regardless.


> and requires validation. Couple this with the fact that the vast majority of
> the Romanian users are not aware of the "cedilla" versus "commabelow" issue and
> you get the present situation, where ~90% of the installations cannot render
> the "commabelow" diacritics. Worse is that, when they encounter the little
> squares that replace the 4 missing glyphs in their fonts, they blame it on the
> writer, "knowing" that they can read Romanian with diacritics in their
> installation (what they don't know is that the more common Romanian diacritics,
So, it means people with Win2k and earlier, with unpatched WinXP (even legal!) would have troubles reading web pages created by Vista, is this correct? And Microsoft accepted that incompatibility, don't they?

> I don't know much about Apple, except that they have a really weird layout
Well, my question was actually not about layout per se - but about the unicode characters they use (whatever the actual mapping is) - whether these symbols are cedillabelow or commabelow. It seems they are commabelow. So Apple accepted that Romanian people with early versions of Windows would have troubles with web pages created in MacOS, is this correct?

Actually, my question here is the following. I totally understand your concern, but if these two huge companies, who are usually rather picky about compatibility issues (because incompatibility costs money to them), decided they could afford using commabelow characters, may be X.Org could make a step forward as well? Also, please consider the fact that next xkeyboard-config release is going to happen in January. Which means actual distros will deliver it to users at some time in the spring (for example, Ubuntu will be out in April). May be, by that time Vista is going to be mainstream amoun Romanian users (or at least most of them manage to patch XP somehow)? Would it be reasonable assumption or not - what's your opinion, Mişu? 

(In reply to comment #45)
> So, it means people with Win2k and earlier, with unpatched WinXP (even legal!)
> would have troubles reading web pages created by Vista, is this correct? And
> Microsoft accepted that incompatibility, don't they?

That's true, they have abandoned those users. Initially they have abandoned XP users too, but luckily the European Union forced Microsoft to correct the issue in XP as that's what they use in Bruxelles.

 
> > I don't know much about Apple, except that they have a really weird layout
> Well, my question was actually not about layout per se - but about the unicode
> characters they use (whatever the actual mapping is) - whether these symbols
> are cedillabelow or commabelow. It seems they are commabelow. So Apple accepted
> that Romanian people with early versions of Windows would have troubles with
> web pages created in MacOS, is this correct?

Yes, again, they use commabelow diacritics. And this has been true for some years now, so we are familiar with the problem and its perception among the Romanian users. Moreso, Apple seems to have a complete disregard for the Romanian keyboard layout standard as they haven't implemented at all, as far as I know.


> Actually, my question here is the following. I totally understand your concern,
> but if these two huge companies, who are usually rather picky about
> compatibility issues (because incompatibility costs money to them), decided
> they could afford using commabelow characters, may be X.Org could make a step
> forward as well? 

Well, when crafting the new standard in 2004 we had one Microsoft employee among us and he was very open and willing to solve the issues, but what we have learned from him and from what happened in the following years was that those changes follow a very bureaucratic path and the results are sometimes surprising. E.g. he said they couldn't fix the keyboard layout in XP because they never introduce new things with a service pack or a hotfix (with the grand exception of SP2 for XP). So they delayed the adoption of the standard keyboard layouts to Vista, but they completely disregarded the display problems in older versions of Windows of "commabelow" diacritics that are generated by the new layouts. "God help us" we said when we found out about the fact. Luckily, the EU came to the rescue and they had the weight to force Microsoft to at least look like they tried to fix the issue. I say that because an optional hotfix that requires validation is not very effective in a country like Romania, as we found out in the last year. MS Romania is doing it's best though, as Alexandru noted, they have a sticky link on their front page to an article that documents the issues.

As a conclusion regarding Microsoft as a whole, they don't seem to carefully consider these options at the decision-making level. We had Microsoft Romania on our side and still couldn't push for the necessary changes. As for Apple, I can't tell, they didn't participate when the new standard was forged, they have never implemented it and I suppose this kind of decisions is taken in Apple by some technical people with a focus on the uniformity of Apple keyboard layouts and very little concern for real world issues in an East-Europe country with negligible sales. They have been using commabelow for such a long time that all but the most determined Romanian Mac users have given up using diacritics altogether because of the compatibility issues.


> Also, please consider the fact that next xkeyboard-config
> release is going to happen in January. Which means actual distros will deliver
> it to users at some time in the spring (for example, Ubuntu will be out in
> April). May be, by that time Vista is going to be mainstream amoun Romanian
> users (or at least most of them manage to patch XP somehow)? Would it be
> reasonable assumption or not - what's your opinion, Mişu? 

That's exactly why we are worried. Spring is quite early, we are aiming for 1.3 or 1.4 to be in time for the autumn releases of Ubuntu and Fedora and then the others. Of course, that's because we have reasons to hope that things will improve a bit, Vista will gain some momentum, more users will become aware of the issues, they will find out about the hotfix etc. I was initially optimistic and thought this will happen in 2007 already, but unfortunately only the very early adopters of Vista, the users of OSX and the Linux l10n crowd are aware of the problems. And not all of them, just the ones that use diacritics, which are very few among regular users.

> Well, when crafting the new standard in 2004 we had one Microsoft employee
> among us and he was very open and willing to solve the issues, but what we 
What kind of solution for the issues did you expect? Did you expect them to provide free (without checking for legality hehe) patches for all Windows version? Did you expect not to have correct layout as defaul in Vista? Like, what would you do (in regard to Romanian layout in Vista) if you were Bill Gates?;)

> surprising. E.g. he said they couldn't fix the keyboard layout in XP because
> they never introduce new things with a service pack or a hotfix (with the 
Yes, I heard that statement more than once. It would seem very wise - if you do not the consider price of upgrade (both SW and HW) and the time interval between MS OS releases... Well, anyway, this is offtopic here.

> that requires validation is not very effective in a country like Romania, as we
Well, the point of supporting users of pirated software (with all comments about unfair licences and Romanian statistics) does not seem 100% valid to me. When users choose to use unlicensed software - they are aware of the fact of illegality and absense of any kind of guarantees (in most cases) - and they expect issues and troubles originating from that fact. I am originally from Russia, another country full of pirated Windows - I know what I am talking about. 

> found out in the last year. MS Romania is doing it's best though, as Alexandru
> noted, they have a sticky link on their front page to an article that documents
> the issues.
That's good. It means every Romanian has access to the solution - free (switching to Linux) or non-free (getting licensed XP).

> all
> but the most determined Romanian Mac users have given up using diacritics
> altogether because of the compatibility issues.
Could we get confirmation of this serious statment somehow? Please, no links in Romanian, if you don't mind;)

> Of course, that's because we have reasons to hope that things will
Ok, please share them with us. I just fail to see the serious difference between April and October in this regard. What exactly do you expect to happen and why? Since pirating Vista is rather difficult, it has a serious upgrade price in terms of HW (and works slowly anyway - I already used it:)

As you may suggest, at this point I am inclined to accept commabelow as default.  I can tell you why. I remember the time when Fedora first released UTF-8-based version and huge crouds of people became very angry (because a lot of software was UTF-incompatible) - but RedHat had very strong will for a change - and now we have most distros using UTF locales by default, and most of that software now works as expected. May be, we should demonstrate we have a strong will for introducing proper Unicode characters in Romanian? At least as strong as MS and Apple do have. If you do not push for changes - they most likely will not happen at all. If MS and Apple would not push for commabelow layouts (leaving FOSS aside) - people would never switch to using proper characters, don't you think?

Regarding web pages - I guess at that point majority of serious Web masters in Romania are aware of the problem (again, I have some parallels in Russia) - and they either provide proper conversion (warnings?) or ban diacritics altogether. Am I wrong?

(In reply to comment #47)

> Regarding web pages - I guess at that point majority of serious Web masters in
> Romania are aware of the problem (again, I have some parallels in Russia) - and
> they either provide proper conversion (warnings?) or ban diacritics altogether.
> Am I wrong?
> 

Most sites don't have diacritics and many strip them because they rely on google results and in google most type without diacritics. And lately google gave up on offering search results with diacritics on terms written without them.

Searching for "și" ("and" in english), probably the most used preposition with ș in it, gives now 300 000 results on google, many of them on blogs, probably from vista and mac users that do some blogging since there isn't a official kb mapping for xp and in linux it takes some knowledge to change to comma.

(In reply to comment #47)
> > Well, when crafting the new standard in 2004 we had one Microsoft employee
> > among us and he was very open and willing to solve the issues, but what we 
> What kind of solution for the issues did you expect? Did you expect them to
> provide free (without checking for legality hehe) patches for all Windows
> version? Did you expect not to have correct layout as defaul in Vista? Like,
> what would you do (in regard to Romanian layout in Vista) if you were Bill
> Gates?;)

Older versions of Windows should have been fixed long ago in regards to displaying the "commabelow" diacritics. If I were Billy G. I would have made the hotfix for XP mandatory and free for all XP users (like IE7 is now).

[snip]

> > found out in the last year. MS Romania is doing it's best though, as Alexandru
> > noted, they have a sticky link on their front page to an article that documents
> > the issues.
> That's good. It means every Romanian has access to the solution - free
> (switching to Linux) or non-free (getting licensed XP).

*If* they are aware of the issue, but the vast majority of them are not. Besides, I don't think these upgrade paths are solutions, most real users won't take any of them. Users of unlicensed XP will probably propagate updated MS fonts through unofficial channels like the peer-to-peer networks, this is what happens in such a situation. But first of all, they need to realize the issue with "commabelow" diacritics.

 
> > all but the most determined Romanian Mac users have given up using diacritics
> > altogether because of the compatibility issues.
> Could we get confirmation of this serious statment somehow? Please, no links in
> Romanian, if you don't mind;)

Sorry, that's hard, there hasn't been any statistical research regarding this, it's my personal opinion of a Romanian user interested in l10n issues. The "commabelow" diacritics are used mostly on some Mac-centric pages and in the blogging and mailing posts of a handful of early Vista adopters (MS Romania included) and some Linux l10n people.

Web statistics would be worth investigating and a possible path would be to search for web pages with commabelow diacritics and see how little of them actualy exist. E.g. the search for the Romanian conjunction "şi" (which means "and" and it's present in almost all texts) yields the following in google.ro with "commabelow" and "cedilla":

http://www.google.ro/search?q=%C8%99i&ie=UTF-8&oe=UTF-8 - 308.000 results
http://www.google.ro/search?q=%C5%9Fi&ie=UTF-8&oe=UTF-8 - 25.300.300 results

That roughly means that ~1% of the Romanian pages with diacritics use them.


> > Of course, that's because we have reasons to hope that things will
> Ok, please share them with us. I just fail to see the serious difference
> between April and October in this regard. What exactly do you expect to happen
> and why? Since pirating Vista is rather difficult, it has a serious upgrade
> price in terms of HW (and works slowly anyway - I already used it:)

The adoption of Vista is inevitable, irrespective of its technical merits or deficiencies. OSX and Linux are also on the rise in the desktop market. Our hope is that the raising penetration of all three (but especially Vista) will raise substantially in 9-12 months the number of people *aware* of the "commabelow" issues and that will raise even more the number of people with good fonts that will have no problems reading "commabelow" diacritics because they'll know that they have to fix their installation somehow. That's why we planned on the xkeyboard-config change to take place in 6-9 months from now.


> As you may suggest, at this point I am inclined to accept commabelow as
> default.  I can tell you why. I remember the time when Fedora first released
> UTF-8-based version and huge crouds of people became very angry (because a lot
> of software was UTF-incompatible) - but RedHat had very strong will for a
> change - and now we have most distros using UTF locales by default, and most of
> that software now works as expected. May be, we should demonstrate we have a
> strong will for introducing proper Unicode characters in Romanian? At least as
> strong as MS and Apple do have. If you do not push for changes - they most
> likely will not happen at all. If MS and Apple would not push for commabelow
> layouts (leaving FOSS aside) - people would never switch to using proper
> characters, don't you think?

MS and Apple haven't introduced the "commabelow" diacritics because of strong will, quite the opposite, they simply don't care much about the Romanian market and they took the path of least resistance. We needed a new official standard and a law to force MS to change the keyboard layouts and the diacritics. And Fedora (which is an experimental distribution by the way) has pushed for UTF-8 locales when most of the software was already UTF-8 clean (the QT/GTK+ toolkits were already fixed beforehand). It is quite the opposite with "commabelow" diacritics, ~90% of the existing sofware installation cannot render them. 

As for the MS and Apple push, let's consider a situation with +80% Linux users in Romania, that will have certainly made it easy for us to change the default layout back in 2004. But we didn't, because +80% of Romanian users have big problems reading them and we have very little influence regarding this issue. We have included them and we made them available to users that know the difference, but as the above referenced Ubuntu bug shows, the Linux users started to look for the comma layouts in 2007, which is quite late...


> Regarding web pages - I guess at that point majority of serious Web masters in
> Romania are aware of the problem (again, I have some parallels in Russia) - and
> they either provide proper conversion (warnings?) or ban diacritics altogether.
> Am I wrong?
 
Yet another complex issue. My opinion is that most Romanian webmasters are not yet aware of the "commabelow" diacritics. We only have five characters with diacritics and just three of them are outside ISO-8859-1, so most Romanian webmasters do not use diacritics at all, some of them use the ones that are part of ISO-8859-1 (â and î) and some use the whole set with "cedilla" from ISO-8859-2 (ăâîşţ). Also take the low percentage of Romanian pages with diacritics into consideration when interpreting the Google results above.

Sergey, if there is need for more info, I'll gladly dive even deeper into this, but I hope that you'll take into consideration the will expressed here by two of the contributors to the Romanian layouts (Marius Andreiana and myself) which is to revert this hasted patch. Bluntly put: please don't ruin our work this way... Although it wasn't such a big contribution, it was built with much care, think about the amount of testing needed to discover a bug such as this one that involves using dead keys in one of Xorg's graphical toolkits: http://bugzilla.gnome.org/show_bug.cgi?id=162845 . 

We have fought for Romanian diacritics in the world of free software for a long time and won some battles over the years. However, this struggle for "commabelow" diacritics needs good timing, it cannot be done as soon as possible. If we change the default layout too early, we will lose too many users. I've seen it happening some years ago when the "winkeys" layout got to be the default layout in XFree86 because of a similar incident in the development of the old "xkb" component involving an outsider that shall remain nameless. Romanian XFree86 users were angry and some of them didn't have the patience to learn how to switch to a sane layout, so they just dropped using diacritics altogether. That's not what we want.

Believe me, it will still be early 6, 9 or 12 months from now, we will still lose some of the users, but we need to minimise losses while maximising the positive impact of Xorg users using "commabelow" diacritics. Just like in any battle, you don't engage the opposite side as soon as possible when you only have a tiny fraction of their forces. You wait for the best moment to throw the bulk of your troops into battle. Granted, you need some active avant-garde in order to assess the difficulties, which is what these early adopters of commabelow diacritics are doing for all of us, we thank them. I hope they know where to thank for the "commabelow" layouts and for the fonts with all the necessary glyphs.

However, the *default* layout is not for what's correct from an idealistically point of view, it is for what works best for now and in the near future. I hope that summarizes the issue for everyone.

(In reply to comment #50)
> Sergey, if there is need for more info, I'll gladly dive even deeper into this,
I think I got enough infomation to understand both positions at this point.

> Believe me, it will still be early 6, 9 or 12 months from now, 
That is exactly what bothers me. Even _if_ (I am considering that possibility still) I would revert the patch for the oncoming release (due in ~1 month) - the next release (late Spring) would be again "still early". And the next one. And the next one... This is going to be endless story, isn't it? And I do not see any real criteria how we could determine "it is a good time to change the default one". Could you offer that kind of criteria? Something simple, not too vague if possible.

> However, the *default* layout is not for what's correct from an idealistically
> point of view, it is for what works best for now and in the near future.
My example with UTF-8 and Fedora worked against your paradigm - and it worked ok IMHO! It was correct from "idealistical POV" - and it did not work best (for that moment) - 8-bit encodings were way better supported by the existing software. Would you call that decision of RedHat premature?

(In reply to comment #51)
> This is going to be endless story, isn't it? And I do not
> see any real criteria how we could determine "it is a good time to change the
> default one". Could you offer that kind of criteria? Something simple, not too
> vague if possible.

Our criteria is that a significant number of Romanian users have no major issues displaying the "commabelow" diacritics. Currently ~90% have big problems with these diacritics. "Significant" would mean for us ~25% of the Romanian users with no major problems and ~25% with the means to solve the problem by themselves (e.g. by installing the aforementioned hotfix in XP). We have fair reasons to believe this is going to happen in 9 to 12 months, so we are targeting a release 6-9 months from now, but we take care to not miss distributions with greater release cycles like Debian. It's sad that localizers with experience and great responsibility planned this with great care and then some hasted newcomer manages to blow it all away...


> > However, the *default* layout is not for what's correct from an idealistically
> > point of view, it is for what works best for now and in the near future.
> My example with UTF-8 and Fedora worked against your paradigm - and it worked
> ok IMHO! It was correct from "idealistical POV" - and it did not work best (for
> that moment) - 8-bit encodings were way better supported by the existing
> software. Would you call that decision of RedHat premature?

When the experimental distribution Fedora started to use UTF-8 locales by default, most of the free software distributed with it worked just fine in an UTF-8 environment (all GTK+, GNOME, QT & KDE software and then some). IMO it's not a fair comparison at all, because ~90% of the Romanian users currently have big problems with the "commabelow" diacritics.

> themselves (e.g. by installing the aforementioned hotfix in XP). We have fair
> reasons to believe this is going to happen in 9 to 12 months, so we are
> targeting a release 6-9 months from now, but we take care to not miss
Ok, so if the January release 1.2 would have old default, the May release 1.3 would have the new one. Would this be a compromized solution acceptable by anyone as "lesser evil"?

> When the experimental distribution Fedora started to use UTF-8 locales by
> default, most of the free software distributed with it worked just fine 
Not exactly. Huge amount of non-graphical console software (essential for admins) had serious issues... Ncurses sucked around UTF-8.

> UTF-8 environment (all GTK+, GNOME, QT & KDE software and then some). IMO it's
> not a fair comparison at all, because ~90% of the Romanian users currently have
> big problems with the "commabelow" diacritics.
As we agreed, it is only the case when careless and ignorant(!) Romania web designer prepares pages using non-default layout (while he still has an option do use the old one).

(In reply to comment #53)
> Ok, so if the January release 1.2 would have old default, the May release 1.3
> would have the new one. Would this be a compromized solution acceptable by
> anyone as "lesser evil"?

Indeed, on the "diacritice" list we were also discussing the 1.3 May release as the most appropriate for changing the default layout to a variant that generates "commabelow" diacritics, with 1.4 as a "worst case scenario" in September.


> > When the experimental distribution Fedora started to use UTF-8 locales by
> > default, most of the free software distributed with it worked just fine 
> Not exactly. Huge amount of non-graphical console software (essential for
> admins) had serious issues... Ncurses sucked around UTF-8.

Yes, most of the text-mode programs had problems, but the regular Linux user uses very few text-mode apps (if any). An admin usually knows how to work around such problems.

 
> As we agreed, it is only the case when careless and ignorant(!) Romania web
> designer prepares pages using non-default layout (while he still has an option
> do use the old one).
 
Actually, it's not only about the web content... I was thinking of documents, mails and instant messages too. Also, not all web content is produced by web designers. Comments in forums, contributions in wikis etc. are written by all kind of users and most of them use the default settings.

Alexandru, what about you?

(In reply to comment #55)
> Alexandru, what about you?
> 

To better understand the people that won't see the ș and ț characters, I've set up a little script here: 

http://aldracu.iglu.cz/test.php?url=http://ro.wikipedia.org/wiki/Sibiu

It shows a rather big wikipedia article with many ș and ț in it. The original article has cedillas. Please have a look at it in Windows XP. See how the text looks when ș and ț aren't available to display. If you got no XP available, just try to imagine a text with squares instead of ș and ț.

We now know that:
- romanian readers can read without diacritics
- few people use now diacritics
- there is more content on the net without diacritics than with diacritics
- there is a small number of situations where the lack of diacritics leads to confusion

Given this knowledge, and looking at the text without ș and ț, can someone honestly state that the text is unreadable and readers will give up reading it?

Now try to switch that page's encoding from UTF-8 to ISO-8859-1. Aaargh, right? That's what I'd call unreadable. Still, as you said that huge problem was accepted when red hat hit the switch.

Yes, right now a big percentege of people can see the two letters that do not belong to the romanian alphabet. I don't see a choice here. We don't have those letters, so I'm saying stop fooling people with resemblance. Microsoft fixed it, Apple fixed it, Linux is a pioneer imho, why is Linux behind ?

(In reply to comment #56)
There are so many exaggerations and emotional statements in the above post, I wouldn't know where to start... And frankly I'm sick and tired of arguing with you Alexandru, you'll always find some silly reasons to start it all again. 

But to point out something that Sergey also knows well, this is not about the pioneering of Linux, xkeyboard-config is a rather OS-agnostic database of keyboard configurations for X Window systems, targeting XFree86 and Xorg which are used in a dozen other operating systems besides Linux.

I can't help myself to also say that I find it embarrassing that the first three arguments on your side are:
  - Romanian readers can read without diacritics
  - few people use now diacritics
  - there is more content on the net without diacritics than with diacritics.

Lads please calm down. The discussion is getting too emotional - which is not productive. What I offer is not reverting the patch but for one release (in January), making the following changes (from the current CVS):

basic -> comma
cedilla -> basic (default one)
std -> std_comma
std_cedilla -> std_cedilla (remains the same)

In May, "comma" would become "basic" again (and default), "basic" would be renamed back to "cedilla" (leaving std_comma and std_cedilla intact).

I guess this solution (while perfect for none of you guys) is the best compromize agreement we could reach.

Alexander, I think your argument about "Linux is a pioneer" is not exactly applicable in the technical discussion, is it? Aside from being too politically charged, it misses the several points. First of all, XKeyboardConfig/XOrg is not Linux software. Second - it does not matter who is the first, what matters is that we know what is right (and it is a very good thing that everyone on this thread agrees that comma is the WTG) - and we all are trying to do the right thing at the right time, aren't we?

I agree to this solution. Maybe std_comma should be renamed to standard in may too, but I don't think it's a must.

(In reply to comment #58)
> Lads please calm down. The discussion is getting too emotional - which is not
> productive. What I offer is not reverting the patch but for one release (in
> January), making the following changes (from the current CVS):
> 
> basic -> comma
> cedilla -> basic (default one)
> std -> std_comma
> std_cedilla -> std_cedilla (remains the same)

This will get us into the situation that existed before the patch, but with different names for some of the variants, which will break setups in which people use the "std" or "academic" variants. Some of them have these variants set up in the xorg.conf file or in other ways. They are a minority, but why should we break their settings?

There was also discussion on the "diacritice" list in the enormous thread about xkeyboard-config regarding the names of the variants, especially about the descriptive names visible to regular users as exported from the base.xml file. We were not happy with them but agreed that we better change them when we change the default layout to a "commabelow" one, in order to minimize confusion and to annoy users as little as possible. Of course, taking into consideration the inner ways of xkeyboard-config.

Sergey, please let me implement at the appropriate time in xkeyboard-config what we have discussed on the most important Romanian l10n list. Alexandru, although a member of that list, completely disregarded our discussions and promoted his own ideas behind our backs, without any open discussions whatsoever. I also think that because I've been maintaining these layouts for some years, I should have been at least asked about this patch beforehand. These are the reasons why I insist into reverting this patch.

> This will get us into the situation that existed before the patch, 
Exactly! That is why I hoped you'd like it!

> but with
> different names for some of the variants, which will break setups in which
> people use the "std" or "academic" variants. 
Not an issue. We'll have the compatibility rules. As long as people do not use XkbSymbols directly (which is not recommended anyway) - they will be ok. I just like the new names more - they look more consistent. 

With xkeyboard-config, the policy is as follows: we're doing our best to keep RMLVO-based configurations working (that is why we have compatibility rules). But we take freedom to break KcCGST-based as we go.

> Some of them have these variants
> set up in the xorg.conf file or in other ways. They are a minority, but why
> should we break their settings?
For consistency sake?

> We were not happy with them but agreed that we better change them when we
> change the default layout to a "commabelow" one, in order to minimize confusion
> and to annoy users as little as possible. 
We can do it gradually. Especially if std_comma and std_cedilla are going to be changed only once (for the January release).

> Sergey, please let me implement at the appropriate time in xkeyboard-config
> what we have discussed on the most important Romanian l10n list. 
Does this mean you are not happy to setup the hard date of switch in May? If so, I do not like your approach...

(In reply to comment #61)
> > Sergey, please let me implement at the appropriate time in xkeyboard-config
> > what we have discussed on the most important Romanian l10n list. 
> Does this mean you are not happy to setup the hard date of switch in May? If
> so, I do not like your approach...

As said before, I think that 1.3 seems most appropriate at the moment but I would like to keep the possibility to postpone the change of the default layout to 1.4 as a "worst case scenario" if things go much worse than predicted. However, I was just browsing the browser/OS statistics for the last 12 months of some of the major generalist Romanian web sites[1][2] and the signs are encouraging. Autumn 2008 seems to be a good hit for the first distributions that will implement the 1.3 version.

In regards to the proposed renaming, I think it's not worth to make this change for consistency sake now, even if the number of affected users is very low. We'll break things when we change the default layout anyway, so why not make these changes in the same release? The current names were already "normalized" once, when you changed "standard" to "std" and "ms" to "winkeys" some time ago.

 1. http://stat.trafic.ro/stat/expresro/sistem/#sistemdeoperare
 2. http://stat.trafic.ro/stat/revistapreseiro/sistem/#sistemdeoperare

> As said before, I think that 1.3 seems most appropriate at the moment but I
> would like to keep the possibility to postpone the change of the default layout
> to 1.4 as a "worst case scenario" if things go much worse than predicted.
Do you mean - MS would pull out Vista from the market? Or release some Service Pack removing comma below chars? Ok, in that highly unlikely case we'll reconsider our decision. Do you foresee anything else?

> encouraging. Autumn 2008 seems to be a good hit for the first distributions
> that will implement the 1.3 version.
Good! So, if you like, I can put it "1.3 will have that switch unless something serious and groundbreaking would make that switch problematic". I really mean - some Force Majeure event.

> for consistency sake now, even if the number of affected users is very low.
And - what's important, that change is independent from the change of the default layout. I cannot imagine a person using two Romanian layouts together.
Frankly speaking, I really LIKE breaking compatibility on XkbSymbols level. I want to push people NOT using XkbSymbols at all. And since we maintain compatibility on XkbLayout level - I see no problem here.

> We'll break things when we change the default layout anyway, so why not make
> these changes in the same release? 
Just because they are pretty much independent.

> The current names were already "normalized"
> once, when you changed "standard" to "std" and "ms" to "winkeys" some time ago.
I do not like the "academic" name, actually. It is out of style. So I'd prefer to get rid of it ASAP. The matrix 2x2 (comma vs cedilla, with and without "std") looks much more logical to me.

For the fun of it, here's one such scenario: MS acknowledges Vista as a flop by marketing a Windows version based on XP SP3 named XP Second Edition, leaving Vista a dead end, like Windows ME was a few years ago. But I don't want to think much about "worst case" scenarios, we should think positively and hope that the current trend regarding the spread of fonts with "commabelow" diacritics will continue and accelerate, so that the addition of the Linux users that will use a default "ro" layout with "commabelow" diacritics would be a catalyst and not an inhibitor for the use of "commabelow" diacritics.

In regards to the new variant names, I see your point, but frankly I don't agree, I'm rather on the conservative side. Fewer changes mean fewer chances to break something, even when it's a matter of deprecated interfaces. Less is more. But it's your playground, so whatever... Name the variants to your liking, but don't forget to reverse everything else that this patch introduced, not only for the reasons enumerated at comment #60, but also because of the pleonastic term "cedillabelow" and the immature shouted comment.

The "academic" variant was indeed one of the complaints when we discussed the names on the "diacritice" list but the consensus was that we'll change the names when we change the default layout. And people were much more interested in the descriptive names that are exposed from the XML file, so I would suggest for now something like this for the variants in the XML file:

 * Commabelow
 * Standard (Cedilla)
 * Standard (Comma)
 * Winkeys

FYI there were at least two other issues raised in that discussion in regards to the descriptive names. One is that people would rather have a "Romanian" label and not "Romania". The other is that people would like a more descriptive label for the default entry, they find that the current one is not descriptive enough and would like something like "Romanian programmers (cedilla)". If you combine the two suggestions, the result would be:

 * Romanian Programmers (Cedilla)
 * Romanian Programmers (Commabelow)
 * Romanian Standard (Cedilla)
 * Romanian Standard (Commabelow)
 * Romanian Winkeys

In case you are wondering, the "Programmers" layout is also included in the new Romanian standard SR13992:2004 (a feat I'm particularly proud of) because of the lack of Romanian hardware keyboards and the predilection of most X users for this layout when using US hardware keyboards.

> more. But it's your playground, so whatever... 
Good, so I will play;)

>  * Commabelow
>  * Standard (Cedilla)
>  * Standard (Comma)
>  * Winkeys
Good! Makes sense.

> to the descriptive names. One is that people would rather have a "Romanian"
> label and not "Romania". 
This is against the rules. The schema is <Country Name> - <Variant Description>.  BTW "Romanian" is ambiguous because it is not clear whether it is related to the country or to the language.

> label for the default entry, they find that the current one is not descriptive
> enough and would like something like "Romanian programmers (cedilla)". If you
> combine the two suggestions, the result would be:
Well, there is an issue with default variants - they only have the country name, according to the rules. May be, the rule has to be changed - I'll think of it one day...

Anyway, I've committed the temporary solution. Lads please check it - whether it is correct for the January release, according to our agreement.

(In reply to comment #65)
[snip]
> Anyway, I've committed the temporary solution. Lads please check it - whether
> it is correct for the January release, according to our agreement.

Thank you Sergey. I'm happy with it with two observations:

 1. The comments in the symbols/ro file need to be changed to reflect the new names and the naming of the diacritical marks in the labels needs to be technical correct and in accordance to their names in xkeyboard-config.pot ("cedilla below" should be "cedilla", "comma below" should be "commabelow"). I'll add a unified patch in a few minutes to fix these problems.

 2.  I've tested the new symbols/ro file in my setup and the "winkeys" variant doesn't work (I use a stable Debian etch system with xkeyboard-config 0.9). I see the problem is "kpdl(comma)" instead of "keypad(comma)" in CVS, is this normal?

Created attachment 13430 [details] [review]
Fixes comments and labels

Please see comment #66 for details.

>  1. The comments in the symbols/ro file need to be changed to reflect the new
> names and the naming of the diacritical marks in the labels needs to be
> technical correct and in accordance to their names in xkeyboard-config.pot
> ("cedilla below" should be "cedilla", "comma below" should be "commabelow").
> I'll add a unified patch in a few minutes to fix these problems.
No problem! Committed.

>  2.  I've tested the new symbols/ro file in my setup and the "winkeys" variant
> doesn't work (I use a stable Debian etch system with xkeyboard-config 0.9). I
> see the problem is "kpdl(comma)" instead of "keypad(comma)" in CVS, is this
> normal?
It is actually kpdl(comma), at least in CVS. So it should be fine.

Thank you Sergey, I'll get back to this bug with a patch to change the default layout, most probably in time for the 1.3 release of xkeyboard-config in May, as we discussed.

Alexandru, I am sorry we couldn't finally agree on this issue. I just want to say that I count the Pidgin incident as an accident and this one as a coincidence and I hope there will never be a third problem of this sort between us. I don't need this sort of time-killer demotivating conflicts.

In regards to xkeyboard-config in general I think that, in case of patches to existing layouts from new contributors, the last contributor should be consulted before applying the patch, especially if he/she acted as a maintainer for some time and fixed problems with the layout. This could be as simple as adding his/her email address to the bug report asking for an opinion.

(In reply to comment #69)
> Thank you Sergey, I'll get back to this bug with a patch to change the default
> layout, most probably in time for the 1.3 release of xkeyboard-config in May,
> as we discussed.
> 
> Alexandru, I am sorry we couldn't finally agree on this issue. I just want to
> say that I count the Pidgin incident as an accident and this one as a
> coincidence and I hope there will never be a third problem of this sort between
> us. I don't need this sort of time-killer demotivating conflicts.
> 

I thought we reached an agreement, Sergey suggested to apply the reverse patch for now and switch to comma default in May "unless something
serious and groundbreaking would make that switch problematic. I really mean -
some Force Majeure event.". I agreed to that appreciating that a hard date was set. 

Is what I'm saying correct Sergey ?

> I thought we reached an agreement, Sergey suggested to apply the reverse patch
> for now and switch to comma default in May "unless something
> serious and groundbreaking would make that switch problematic. I really mean -
> some Force Majeure event.". I agreed to that appreciating that a hard date was
> set. 
> 
> Is what I'm saying correct Sergey ?
Yes that's what I plan to do. I am going to change the default variant in May - unless Iget some _very_ serious and convincing arguments that it would be a bad idea. For a moment I am considering May as a hard date.

(In reply to comment #70)
> I thought we reached an agreement [snip]

Sorry, my bad... Good to see you have finally agreed to delay this change, it certainly was a long journey.

Sergey, I hope my suggestion in regards to consulting the last contributor/maintainer when newcomers send patches to existing layouts will be taken into consideration.

> Sergey, I hope my suggestion in regards to consulting the last
> contributor/maintainer when newcomers send patches to existing layouts will be
> taken into consideration.
Yes, that was a valid point. And I am looking forward to hearing from you in early May.

Mass reopen.  The "LATER" resolution is lame, I'm deleting it.  Consider LATER to have arrived.

Since they removed LATER, the bug will be in NEEDINFO till May (which is only 2 months away BTW:)

Lads, don't you think it is about time to start working on the patch to change the default variant? Or at least start discussing it (if there are objections)?

(In reply to comment #76)
> Lads, don't you think it is about time to start working on the patch to change
> the default variant? Or at least start discussing it (if there are objections)?
> 

I'll get back with a patch in a few days. It's a matter of minutes to create a diff, but I'm currently away from home in an Easter vacation. Is it possible to know when exactly in May are you going to release version 1.3 of xkeyboard-config? Thanks.

> Is it possible to
> know when exactly in May are you going to release version 1.3 of
> xkeyboard-config? Thanks.

http://www.freedesktop.org/wiki/Software/XKeyboardConfig/ReleaseSchedule

Thanks. I see the freeze begins on 13.05. Although I'm currently away from home I'll build a patch this week as I have access to a Linux machine with X.org.

Hello,


Since we are now in May 2008; Ubuntu 8.04 was released and Fedora 9 is just a few days away. Someone *please* update us on the state of this bug - I'll try to sum up:

- do we have keyboard arrangements corresponding to the "primary" and "secondary" layouts in the Romanian National Standard SR 13392:2004 ?

- does the keyboard generate Unicode characters, not ISO8859-2, for Romanian language ?

- are the s/S and t/T with marks below the correct comma versions, not the incorrect cedilla versions ? (for the non-Romanian readers: there is no such thing as "cedilla-below" in Romanian language, it's purely a very old bug);

- do we have fonts to correctly represent the comma glyphs, both in text mode (default font) and in X mode ?


Thanks a lot,
Răzvan

(In reply to comment #80)
> Since we are now in May 2008; Ubuntu 8.04 was released and Fedora 9 is just a
> few days away. Someone *please* update us on the state of this bug - I'll try
> to sum up:

You are asking in the wrong place, this bug is about changing the default variant in the Romanian layout of xkeyboard-config. Please join the "diacritice" mailing list[1] to discuss the other issues (most of them already solved).

 1. http://groups.google.ro/group/diacritice

Created attachment 16351 [details] [review]
Changes the default variant in the Romanian layout

Here's the awaited patch that changes the default variant in the Romanian layout to one that generates the technically correct commabelow diacritics. Please commit it time for the 1.3 release of xkeyboard-config.

Thank you.

Created attachment 16354 [details] [review]
A better version of the previous patch

This one updates the comments also in accordance with the new changes.

Everybody happy with that patch?

Thanks !

From my point of view, I think it's OK. Please include it ASAP, so it will land in all Linux distributions that uses that code. It will be a lot of practical testing then, we will know if something breaks ;-)

Răzvan

(In reply to comment #85)
> It will be a lot of practical testing then, we will know if something breaks ;-)

Răzvan, we need to find out if this patch breaks something *before* releasing a new version of xkeyboard-config that includes it. We absolutely do not want to get a broken Romanian keyboard layout in released distributions...

On my side I have tested the patch to the "symbols/ro" file in my Debian stable system (all variants) and have carefully checked the names in the "rules/base.xml.in" for a perfect match. But I would be most happy if someone else would double-check this. Alex?

Mişu, of course we all want those programs tested *before* general use. However:

I think your patch it's 100% OK, strictly speaking from the keyboard point of view. The difference will be at application level. By using the new Romanian keyboard, people using Romanian diacritics will start to insert comma-below characters in their documents, translations, webpages, etc.

After a while we will see some incompatibilities/bugs, but not regarding the code itself. There will be cases where people don't use fonts with comma-below characters, we will see mixed-typed documents, webpages and interface translations, etc. But this have to do with the applications and people's works, not the code.

IMHO, it's the *best way* to make a large number of people aware of the problem, in order to start correcting their works from cedilla-below characters to comma-below ones.

It will also break some Windows compatibility (in pre-Vista versions), but the important fact is that *a patch do exist* for Windows XP and previous. For Windows users, it is important to let people know about Cristian Secara's keyboard driver and about Microsoft EUupdate corrected fonts. It's more a matter of "national politics" in IT than coding.

People on the TIC-Lobby mailinglist at Googlegroups (whics mainly include influential decision-makers in Romanian IT) are very much aware of this and *prepared* for it. Both Cristian Secara and me explained the matter in detail.

So *please* let them include the patch in distros, to let a large number of people to test in in real-life situations. If not, we will not go over this "fear-to-change" phase forever...

Thanks a lot to you, Alex and all Romanian developers/translators for your *very good* work !


Regards,
Răzvan

(In reply to comment #86)
> (In reply to comment #85)
> > It will be a lot of practical testing then, we will know if something breaks ;-)
> 
> Răzvan, we need to find out if this patch breaks something *before* releasing
> a new version of xkeyboard-config that includes it. We absolutely do not want
> to get a broken Romanian keyboard layout in released distributions...
> 
> On my side I have tested the patch to the "symbols/ro" file in my Debian stable
> system (all variants) and have carefully checked the names in the
> "rules/base.xml.in" for a perfect match. But I would be most happy if someone
> else would double-check this. Alex?
> 

I applied the patch, everything works fine, Sergey please go ahead with this.

Thanks lads, I'll commit it tonight or tomorrow.

committed!

Thank you Sergey, it is a great moment for the Romanian layout in xkeyboard-config. We now join the fight for a perfect Romanian in the computer world and I hope that the users who get burned by compatibility issues will not be so many...

Alex, if you're satisfied with the resolution, I think it's time to close this bug.

Problem fixed here, will check how it turns out in the distros.

Hello,


Please, does someone knows when is this patch expected to land in popular distributions like Fedora, Ubuntu, Debian, Red Hat, CentOS, SuSE ? I mean in .rpm and .deb packages crafted for those distros...

In Fedora (this is the distro that I use), this is a precondition to start inserting correct characters in documents (we still miss correct comma-below glyphs in many fonts, but working on it)...

Thanks and regards,
Răzvan

(In reply to comment #93)
> 
> In Fedora (this is the distro that I use), this is a precondition to start
> inserting correct characters in documents (we still miss correct comma-below
> glyphs in many fonts, but working on it)...
 
I just want to point out this change of the default variant is NOT a prerequisite for using correct characters in Romanian documents. There has been at least one variant with commabelow diacritics in the Romanian layout for the past 8 (eight) years.

That's right, Mişu, but please think about the regular, *non-technical* user that is using a Linux workstation for day-to-day office work. What she/he does?

In Gnome:

- go to System menu -> Administration -> Language and set it to „Romanian”;

- go to System menu -> Administration -> Keyboard and set it to „Romanian” (or, if she/he is lucky enough, has there two variants, „Romanian primary” and „Romanian secondary”. IMHO, „secondary” should be the default).

That's *all* ! No command line, no manual patches, no setkbd commands...

Then she/he expects to have:

- correct keyboard layout (according to Romanian official standard)
- correct fonts (comma-below)
- correct Romanian conventions for numbers, dates, currency, etc.

His expectations are *independent of the application* he uses or the fact he uses that workstation in text mode (default runlevel 3) or X mode (default runlevel 5).

IMHO, that's the point where we should arrive - software that „just works” as expected.

This is a precondition for having a continuous stream of *correctly generated* Romanian documents, webpages, etc. (because the non-technical office worker is the main source of such documents).


Thanks a lot,
Răzvan

(In reply to comment #93)
> Hello,
> 
> 
> Please, does someone knows when is this patch expected to land in popular
> distributions like Fedora, Ubuntu, Debian, Red Hat, CentOS, SuSE ? I mean in
> .rpm and .deb packages crafted for those distros...
> 

Usually X.org packages are not changed between distribution releases. With the case of Fedora, expect the package in Fedora 10, this autumn.

Thank you, Alex !

IMHO, I should not consider this a RFE, but the fix of a longstanding bug. So I'll be happy to see that in all distros ASAP. But I don't decide this.

Related to this, please see also my comment in Red Hat's bugzilla, bug 337271, comment 19.


Regards,
Răzvan

The next Debian stable release (aka Lenny) will have this (it already does, actually, so if you're using testing, then you're already set for using the new default).

Still, there is an upgrade/transition issue from the old default to the new one (apparently): http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=485702

(In reply to comment #98)
> Still, there is an upgrade/transition issue from the old default to the new one
> (apparently): http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=485702

I am basing this statement on the assumption[*] that the old variant was named "standard", while the new one is called "std". Is this assumption[*] correct?



If I am right and the name changed, we (all distros) need a migration path which must be implemented by the distribution - not a very nice thing since upstream should have made sure there is a smooth upgrade path, IMO.




[*] more than an assumption, actually; I remember setting this variant as default in xorg, and it worked; it stopped working when the new xkeyboard-config (1.3) was installed instead.

Hello,

I would like to draw attention to an issue related
to the latest change in xkeyboard-config. This issue
was detected[1] in Debian in the corresponding package
xkb-data[2]. The issue is that the layouts are not
loaded properly and more details to be found at the
links listed below. Another discussion sheds more
light on the issue [3].

[1] http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=490013
[2] http://packages.qa.debian.org/x/xkeyboard-config.html
[3] http://lists.debian.org/debian-l10n-romanian/2008/07/msg00167.html

Thanks,
Igor Stirbu

(In reply to comment #100)

> links listed below. Another discussion sheds more
> light on the issue [3].

> [3] http://lists.debian.org/debian-l10n-romanian/2008/07/msg00167.html

Since that discussion is in Romanian, it isn't too helpful without a translation.

At the end of the thread apparently there is a conclusion:
http://lists.debian.org/debian-l10n-romanian/2008/07/msg00171.html



>> > $ setxkbmap -v ro std
>> > Warning! Multiple definitions of keyboard layout
>> >         Using command line, ignoring X server
>> >         Trying to build keymap using the following components:
>> >         keycodes:   xfree86+aliases(qwertz)
>> >         types:      complete
>> >         compat:     complete
>> >         symbols:    pc+ro(std_cedilla)
>> >         geometry:   pc(pc105)
>> >
>> >
>> > De ce încarcă 'std_cedilla', pentru că începe cu 'std'?

Translation: Why does it try to load 'std_cedilla'? Is it because it starts with 'std'?

>> 
>> Foarte ciudat. Am rulat comanda ta și mi-au apărut diacritice
>> cu sedilă iar dacă rulez în acest context:

T: Very weird. I ran your command and I got diacritics with cedilla, while running in this context:

>> 
>> setxkbmap -model pc105 -layout ro,us,ru -variant std,,phonetic
>> 
>> mi le ia cu virgulă. Asta e legat de configurarea greșită a fișierului
>> pentru limba română din xkb-data. setxkbmap își face bine treaba.

yelds comma diacritics. This is related to the wrong configuration of the xkb-data file for Romanian. setxkbmap works correctly.

> 
> Da ai dreptate:

Yes, you're right:

> $ setxkbmap -v 10 -layout ro,de -variant std,
> Setting verbose level to 10
> locale is C
> Warning! Multiple definitions of keyboard layout
> 	 Using command line, ignoring X server
> 	 Warning! Multiple definitions of layout variant
> 		  Using command line, ignoring X server
> 		  Applied rules from xorg:
> 		  model:      pc105
> 		  layout:     ro,de
> 		  variant:    std,
> 		  Trying to build keymap using the following components:
> 		  keycodes:   xfree86+aliases(qwertz)
> 		  types:      complete
> 		  compat:     complete
> 		  symbols:    pc+ro(std)+de:2
> 		  geometry:   pc(pc105)
> 
> Dar dacă dacă nu specific decât un layout:

If I specify a single layout:

> $ setxkbmap -v 10 -layout ro -variant std
> Setting verbose level to 10
> locale is C
> Warning! Multiple definitions of keyboard layout
> 	 Using command line, ignoring X server
> 	 Warning! Multiple definitions of layout variant
> 		  Using command line, ignoring X server
> 		  Applied rules from xorg:
> 		  model:      pc105
> 		  layout:     ro
> 		  variant:    std
> 		  Trying to build keymap using the following components:
> 		  keycodes:   xfree86+aliases(qwertz)
> 		  types:      complete
> 		  compat:     complete
> 		  symbols:    pc+ro(std_cedilla)
> 		  geometry:   pc(pc105)
> 
> E posibil problema să fie altundeva. Fișierul 
> /usr/share/X11/xkb/rules/xorg pare suspect, dar nu sunt 100% sigur.

It could be that the problem lies elsewhere. The file /usr/share/X11/xkb/rules/xorg seems suspect, but I am not 100% sure.

(In reply to comment #100)
> ...The issue is that the layouts are not
> loaded properly and more details to be found at the
> links listed below. 

Igor, I think you should open a new bug report. This one's closed and your problem is not related to the default layout in the Romanian layout, but rather to using three layouts in a particular order, two of them with a non-default variant. I say this because it seems that everything is fine if you change the order.

And also, what is the "xkb plugin"? I've tried all the setxkbmap commands from 
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=490013 and they work for me.

(In reply to comment #102)
> (In reply to comment #100)
> > ...The issue is that the layouts are not
> > loaded properly and more details to be found at the
> > links listed below. 
> 
> Igor, I think you should open a new bug report. This one's closed and your

Might be a good thing to do but for now I consider that
it's very close related to the issues discussed in this
thread/bug and the last change related to romanian layout
is the clue. Is the discussion prohibited even if the bug
is closed/fixed?

> problem is not related to the default layout in the Romanian layout, but rather
> to using three layouts in a particular order, two of them with a non-default
> variant. I say this because it seems that everything is fine if you change the
> order.

I see a problem if I have to use the parameters in a special
order. It's a hack (to change the order of layouts) and not a
proper solution though I can cope with if for a while.

> And also, what is the "xkb plugin"? I've tried all the setxkbmap commands from 
> http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=490013 and they work for me.

Initially I thought that the bug was in the xfce4-xkb-plugin that
refused to load after the upgrade to version 1.3 of xkb-data (aka
xkeyboard-config). After discussing the issue with the Debian
maintainer of xkb-data, the bug was moved to xkb-data.

Please look closer at the commands and their outputs in the bug report
and d-l10n-ro mailing list. I have almost zero knowledge in
configuring rules and stuff for xkeyboard-config and can't really
pin point the bug. What I can say that there are different ouputs
for similar input and that's an issue.

Thanks for your time,
Igor

> After discussing the issue with the Debian
> maintainer of xkb-data, the bug was moved to xkb-data.

Sorry, the correct version is:

After discussing the issue with the Debian
maintainer of xfce4, the bug was moved to xkb-data.

(In reply to comment #103)
> (In reply to comment #102)
> > (In reply to comment #100)
> > > ...The issue is that the layouts are not
> > > loaded properly and more details to be found at the
> > > links listed below. 
> > 
> > Igor, I think you should open a new bug report. This one's closed and your
> 
> Might be a good thing to do but for now I consider that
> it's very close related to the issues discussed in this
> thread/bug and the last change related to romanian layout
> is the clue. Is the discussion prohibited even if the bug
> is closed/fixed?

<snip>

The issue with this thread is that it has gone over 100 comments. Any new person that might want to assist will probably not be able to follow the discussion.

In the course of the hundred comments, I believe that there is now a better view of the issue you are describing. What you need is to write a concise summary of the problem as it stands now and file it under a separate bug report.

As a maintainer, I wholeheartedly agree. Please file separate bug. Even if that problem was triggered by the fix of this bug.