Bug 27472 - Add symbols to Arabic (ara) keyboard layout
Summary: Add symbols to Arabic (ara) keyboard layout
Status: RESOLVED FIXED
Alias: None
Product: xkeyboard-config
Classification: Unclassified
Component: General (show other bugs)
Version: unspecified
Hardware: Other All
: medium normal
Assignee: xkb
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2010-04-05 11:38 UTC by Benjamin Geer
Modified: 2012-11-19 23:13 UTC (History)
2 users (show)

See Also:
i915 platform:
i915 features:


Attachments
patch to add symbols to /usr/share/X11/xkb/symbols/ara (2.44 KB, patch)
2010-04-05 11:38 UTC, Benjamin Geer
Details | Splinter Review
patch to add symbols to /usr/share/X11/xkb/symbols/ara, version 2 (2.61 KB, patch)
2010-05-06 00:35 UTC, Benjamin Geer
Details | Splinter Review
Patch to remove "level3(ralt_switch)" line (701 bytes, patch)
2012-11-18 14:38 UTC, Khaled Hosny
Details | Splinter Review

Description Benjamin Geer 2010-04-05 11:38:56 UTC
Created attachment 34682 [details] [review]
patch to add symbols to /usr/share/X11/xkb/symbols/ara

The attached patch adds some symbols to the Arabic (ara) layout.

First, the patch adds four symbols that are sometimes used to transliterate non-Arabic words into Arabic:

U+067E (پ) is used to represent IPA /p/.

U+0686 (چ) is used to represent IPA /tʃ/ or /j/.

U+06A4 (ڤ) is used to represent IPA /v/.

U+06AF (گ) is used in Persian to represent IPA /ɡ/, and can be used in Arabic for the same purpose.  There are other ways to transliterate this sound in Arabic; I am following the suggestion on http://www.alfagih.net/site/?p=2723 to promote the use of گ.

The patch also adds guillemets, U+00AB («) and U+00BB (»), which are often used as quotation marks in Arabic texts.

In order to make this patch minimally intrusive, I have used AltGr for all these additional symbols:

پ = AltGr + ب
چ = AltGr + ج
ڤ = AltGr + ف
گ = AltGr + ك

The guillemets (guillemotleft and guillemotright) are located on the Z and X keys, using AltGr, as in the "latin" layout, but they're reversed, like the parentheses (parenleft and parenright) in the "ara" layout, so that they're oriented correctly in right-to-left contexts.
Comment 1 Sergey V. Udaltsov 2010-05-05 16:35:47 UTC
You are adding some more symbols on 3rd level. Should you also include "level3(ralt_switch)" - as we typically do for variants with >2 levels?
Comment 2 Benjamin Geer 2010-05-06 00:35:25 UTC
Created attachment 35451 [details] [review]
patch to add symbols to /usr/share/X11/xkb/symbols/ara, version 2

added "level3(ralt_switch)"
Comment 3 Benjamin Geer 2010-05-06 00:38:00 UTC
Thanks, Sergey.  I've included "level3(ralt_switch)" and attached an updated patch.
Comment 4 Sergey V. Udaltsov 2010-05-06 13:34:33 UTC
That looks fine. Thanks, committed!
Comment 5 alsadi 2011-12-17 02:56:23 UTC
those patches broke changing layout with right alt+shift

we use alt+shift almost every minute, and I as a native Arabic speaker almost never used those non-native Arabic letters (written in native Arabic script)


I want to ask you if those letters are known to users and if they are part of arabic layout in any other operating system

qwerty (Arabic qwerty) is a de-facto standard so if we added secret combinations how uses would know about them and use them

I suggest you either use gucharmap or add a macro to your editor, or use usual shift combinations
Comment 6 Benjamin Geer 2011-12-17 03:31:04 UTC
I don't use right alt+shift to change keyboard layouts. I use the "Windows" key for that. This is easy to configure.

Yes, Arab users know these letters. In Egypt, where I live, one often sees the چ and the ڤ with three dots in advertising, product names, etc. "Vodafone" in Arabic is written ڤودافون:

http://www.vodafone.com.eg/vodafoneportalWeb/ar/P613722281289132343406

In Egypt, "garage" is written جراچ:

http://www.almasry-alyoum.com/article2.aspx?ArticleID=46504&IssueID=570

If you want to read something by a native Arabic speaker (a Saudi journalist) who uses and advocates the گ, check the link I already posted:

http://www.alfagih.net/site/?p=2723

What other operating systems do is irrelevant. Free software should be better than proprietary software, and this means offering features that proprietary software doesn't offer.
Comment 7 Daniel Stone 2011-12-18 06:52:38 UTC
(In reply to comment #6)
> What other operating systems do is irrelevant. Free software should be better
> than proprietary software, and this means offering features that proprietary
> software doesn't offer.

Actually, it's massively relevant.  'Better' in this case doesn't mean 'if you start from zero knowledge of a keyboard then this might subjectively be an improvement'.  If that was the case, we wouldn't default to QWERTY.  The best -- and I do mean absolute best -- thing we can do is, by default, to match exactly what Windows and OS X do, so users aren't surprised (and not in a good way) that their keyboard doesn't work how they expect it to.

Now, if you want to create a variant that does something different/better, then sure, knock yourself out.

But diverging from other OSes in surprising ways is simply not an option.  Sorry.
Comment 8 Benjamin Geer 2011-12-18 12:05:12 UTC
The Arabic keyboard layout already wasn't the same as the Windows Arabic layout anyway, as I recall. Neither are many of the other X keyboard layouts. For example, another X layout I use is USA International, which has no counterpart in Windows.

The additional characters added by this patch shouldn't affect any user who isn't interested in them. Alsadi's objection that he uses right alt+shift to switch keyboard layouts is simply bizarre, because the Arabic layout already used right alt+shift for several characters (the harakat). Users who want the extra characters I added will be happy to find them when they explore what's available in this layout. Others shouldn't notice them or care.
Comment 9 Sergey V. Udaltsov 2011-12-20 15:15:47 UTC
> The Arabic keyboard layout already wasn't the same as the Windows Arabic layout
> anyway, as I recall. 
Could you please point to the differences? Perhaps the layout has to be polished and "aligned".

> Neither are many of the other X keyboard layouts. For
> example, another X layout I use is USA International, which has no counterpart
> in Windows.
But the default US layout in xk-c is the same as the one on Windows.

> The additional characters added by this patch shouldn't affect any user who
> isn't interested in them. Alsadi's objection that he uses right alt+shift to
> switch keyboard layouts is simply bizarre, because the Arabic layout already
> used right alt+shift for several characters (the harakat). Users who want the
> extra characters I added will be happy to find them when they explore what's
> available in this layout. Others shouldn't notice them or care.
It is not exactly true. The layout can be "forgiving". For example, is something is missing on the higher shift levels, the mappings would be borrowed from the lower levels. And people may forget to release the Shift (or RAlt) key - getting the results they expected, instead of the new chars.

As Daniel said, the default layout should be as compatible as possible, even if it is clear that technically (usability-wise) it is not perfect. But some new sensible variants are always welcome.
Comment 10 Benjamin Geer 2011-12-21 08:20:04 UTC
OK, there are many differences between the Windows Arabic layout and the X Arabic layout. I don't have Windows, but the layout can be examined here:

http://msdn.microsoft.com/en-us/goglobal/bb964651The Windows layout has 

The Windows Arabic (102) layout has the ذ (U+0630) character on the middle row, just to the left of the Enter key. In X, this character is on the top row, to the the left of the 1 key.

If you press Shift, the Windows layout gives you ِ (U+0650) and ٍ (U+064D) on the third and fourth keys from the left after the Left Shift key on the bottom row, while the X layout gives you these characters on the middle row, on the first two keys to the right of the Shift Lock key.

In the Windows layout, the right and left curly brackets (U+007D and U+007B) are accessed with the Shift key, using the two keys furthest to the right on the row above the middle row, just to the left of the Enter key. In the X layout, these are located on the fourth and fifth keys from the left after the Left Shift key on the bottom row.

I could go on and on, but I think this is a pointless exercise. I've been using the X Arabic layout for years, and I'm used to it. It would greatly inconvenience me and other users to have to get used to the Windows layout instead. I don't see why the expectations of Windows users are more important than the expectations of long-time Linux users.

Moreover, the Mac OS X Arabic layout is completely different from both the Windows and X layouts. Should we be trying to accommodate the expectations of Mac users, too?

Finally, I was mistaken when I said above that the X Arabic layout has always provided the harakat using alt-shift; actually it provides them just using Shift. However, I don't see why the patch that was applied should affect the use of alt-shift to change keyboard layouts, since the characters added are typed just using alt, not alt-shift.

Sergey, I don't understand your point about a layout being "forgiving." No keyboard layout can accommodate all the mistakes that anyone could possibly make.

Finally, the Windows Arabic layout provides a few characters using right-alt. If it's OK for them, I don't see why it's not OK for us.
Comment 11 Benjamin Geer 2011-12-21 08:20:59 UTC
Corrected link to the Windows Arabic keyboard layout:
Comment 12 Benjamin Geer 2011-12-21 08:21:21 UTC
Corrected link to the Windows Arabic keyboard layout:

http://msdn.microsoft.com/en-us/goglobal/bb964651
Comment 13 Sergey V. Udaltsov 2011-12-24 13:04:14 UTC
I guess you really touched a very interesting issue.
Would it be possible for you to create 2 variants that would mimic exactly (or at least as close as possible) the Windows approach and MacOS X approach - so the 3rd one, the X one, could use any innovations that we might find useful? There is a separate question which one should be the default one - but we should ideally provide all 3 anyway. What do you think?
Comment 14 Benjamin Geer 2011-12-24 16:22:46 UTC
Sergey, I agree with you. This is a good idea, and I would be happy to do it, but unfortunately I don't know when I'm going to have time to do it.
Comment 15 Sergey V. Udaltsov 2011-12-24 16:25:24 UTC
Let that bug hang around then ;)
Comment 16 Muhammad Saied 2012-11-17 18:28:52 UTC
Windows users use Arabic 101 layout NOT 102. The 101 layout is identical to X layout. This batch broke the right alt key and now it's not usable at all. I don't think that any Arabic user is familiar with level3 because it's not being used on any other system, further more the symbols added to the layout are NOT Arabic: they are used in some texts, but Arabic Language Academies consider using them wrong in Standard Arabic and discourage it.

100% of Linux Arab users are overriding this patch by modifying the ara file locally, none of them consider this "improvement".

Q: If these symbols are popular in Egypt, Why did you modify the generic layout instead of creating ar_EG layout?
Comment 17 Benjamin Geer 2012-11-17 18:37:21 UTC
"I don't think that any Arabic user is familiar with level3 because it's not being used on any other system"

This is not any other system. All systems are different and require some adjustment. Many Arabic users use Mac OS X, which is completely different from Windows. They will have to make some adjustments when switching to Linux/Unix in any case. Level3 will be the least of their worries.

"the symbols added to the layout are NOT Arabic: they are used in some texts, but Arabic Language Academies consider using them wrong in Standard Arabic and discourage it."

Very few people care what Arabic language academies say. If they did, we wouldn't see these symbols constantly in real Arabic texts. Most users care more about reality than about the fantasies of language academies.

"100% of Linux Arab users are overriding this patch by modifying the ara file locally, none of them consider this "improvement"."

I don't see how you could possibly know what 100% of Arab users are doing. Show me a survey based on a statistically random sample of users.

"If these symbols are popular in Egypt, Why did you modify the generic layout instead of creating ar_EG layout?"

Because they're not just used in Egypt. See the Saudi link I already posted.
Comment 18 Muhammad Saied 2012-11-17 21:00:08 UTC
>> I don't see how you could possibly know what 100% of Arab users are doing. 
>> Show me a survey based on a statistically random sample of users.

You can see that all of the Arabic distrobutions are overriding this patch to get the old behavior back. Users of other distrobutions modify the file by hand, there are also shell scripts to do it automatically for beginners. I can make you a survey if you insists.

>> Because they're not just used in Egypt. See the Saudi link I already posted.

That's only one person's blog, and if you notice the article suggestes using the letter گ to readers, which means that it is not popular as you say.

>> Very few people care what Arabic language academies say. If they did, we >>wouldn't see these symbols constantly in real Arabic texts.

So now we should listen to individual opinions and ignore the academies?
Please show me a survey based on a statistically random sample of *real* Arabic texts to proof popularity of those symbols.

Letters of a language don't change over time, don't you think?
Comment 19 Benjamin Geer 2012-11-17 21:51:54 UTC
The use of AltGr to produce additional symbols is standard in the X Window System. Most keyboard layouts use it. Clearly the users of those other layouts (many of whom are probably also familiar with Windows) have no problem with this, and don't feel that it "breaks" the Right-Alt key. I don't see why the situation should be any different for Arabic. If the users of all those other keyboard layouts can adapt to the X Window System, it seems to me that Arabic users can adapt, too.

If you're using Right-Alt-Shift to switch keyboard layouts, there's no reason you can't use some other key or key combination to do that. I use the Left-Win key, for example. There are plenty of options, and this is very easy to configure. However, there is currently no other way to type these additional characters, which are widely used in Arabic. I think a small inconvenience is well worth the ability to type widely used characters that would otherwise be impossible to type.

Your argument is a bit like saying that since Windows users are used to locking their session using Ctrl-Alt-Del, Unix should behave in exactly the same way. It doesn't. But people adapt. We are not robots. We can learn new habits.

> So now we should listen to individual opinions and ignore the academies?

If you study the history of the Arabic language, you will find that the academies have always been ignored. All languages change and evolve. Alphabets change and evolve, too. This is a normal, universal phenomenon. Everyday usage, not academics, determine what is standard in every language.

If Vodaphone decided to spell their brand name with a ڤ, it's certainly because 300 million Arabs recognize this symbol and are familiar with it. Marketing is based on surveys and reflects the preferences of consumers.

If every garage in Egypt has a sign in front of it that says جراچ, it's because 80 million Egyptians recognize this symbol and are familiar with it.

Just walk around Cairo and you'll see hundreds of billboards with these letters on them. That's real standardization. The addition of these letters to the Arabic keyboard layout simply reflects this widespread reality.

If you refuse to accept the consensus of 300 million Arabs, then you're the one who is trying to impose your "individual opinion" on everyone else.

As for گ, I acknowledge that it's not widely used, but once we've added a third level, it costs nothing to add this symbol. Those who don't want it don't have to use it. However, the other symbols I added are extremely popular among Arabs, and I think it would be irresponsible not to provide a way to type them.
Comment 20 Benjamin Geer 2012-11-17 21:54:27 UTC
And by the way, the standard Ubuntu distribution contains this keyboard layout. I'd bet that's the distribution that most Arab Linux users actually use.
Comment 21 Benjamin Geer 2012-11-17 22:09:06 UTC
And here's an Arabic textbook that explains how these symbols are commonly used in Arabic:

http://books.google.com/books?id=svelPctsVNEC&lpg=PR5&ots=EJpOpKybXI&dq=arabic%20letter%20%22three%20dots%22%20%22foreign%20words%22&lr&pg=PA49#v=onepage&q&f=false
Comment 22 Benjamin Geer 2012-11-17 22:47:59 UTC
Here's another academic reference (see the bottom of the page):

http://books.google.com/books?id=8E0Rr1xY4TQC&pg=PP392#v=onepage&q&f=false

Note that the author points out that these letters are used in the Gulf.
Comment 23 Muhammad Saied 2012-11-17 23:21:06 UTC
> Alphabets change and evolve, too. This is a normal, universal phenomenon.

When this happens we get a different language, give me an example of a language that has changed its letters or added to them. All of the layouts that uses level3 uses it to access letters of its own alphabet.

Yes, we recognize those letters but this does not mean that they belong to the Arabic alphabet. This is why we discourage their usage.

If you want these letters then create ar_EG.

>And by the way, the standard Ubuntu distribution contains this keyboard layout. 
>I'd bet that's the distribution that most Arab Linux users actually use.

They modify the file, forums are full of QAs about it. And there are also Arabic distrobutions such as Ojuba which fix this by default. Why would they change it if it is so helpful?
Comment 24 Benjamin Geer 2012-11-18 00:20:49 UTC
> > Alphabets change and evolve, too. This is a normal, universal phenomenon.
>
> When this happens we get a different language, give me an example of a language
> that has changed its letters or added to them.

English: http://en.wikipedia.org/wiki/Long_s

French: http://en.wikipedia.org/wiki/Use_of_the_circumflex_in_French#First_usages

German: http://en.wikipedia.org/wiki/%C3%9F

> All of the layouts that uses level3 uses it to access letters of its own
> alphabet.

These letters *are* part of Arabic. See the two academic sources I cited.

> Why would they change it if it is so helpful?

Maybe because they're language purists. The goal of language purists is not to be helpful, but to impose a particular view of language on everyone else:

http://en.wikipedia.org/wiki/Linguistic_prescription
Comment 25 Khaled Hosny 2012-11-18 13:23:47 UTC
(In reply to comment #16)
> Arabic Language Academies consider using them wrong in Standard Arabic and discourage it.

That is a strong accusation you have to back with evidence, I have in my hands the book published by the Academy of the Arabic Language in Cairo summarizing its first 50 years of activity, which *recommends* the use of several extended Arabic characters for use in transliteration, it even goes further and introduces few of its own. The Arabic translation of Scientific American makes an extensive use of extended Arabic characters (http://www.oloommagazine.com), and many well respected members of Arabic Language Academies work on these translations. So the “un-Arabic” claim is dubious at best, so lets try to fix the real issue not some imaginary language purity (even though we are discussing the writing system in the first place!)
Comment 26 Khaled Hosny 2012-11-18 13:59:32 UTC
The book I’m referring to, مجمع اللغة العربية – مجموعة القرارات العلمية في خمسين عاما، p.196–216. Unfortunately it is not available from Cairo Academy website (http://www.arabicacademy.org.eg).
Comment 27 Khaled Hosny 2012-11-18 14:38:50 UTC
Created attachment 70216 [details] [review]
Patch to remove "level3(ralt_switch)" line

Now I think the problematic part is the "level3(ralt_switch)" line, with this line even if I configure another key as a 3rd level switcher right alt will be still used as a 3rd level chooser as well with no way to override it. Given this is the default Arabic layout, I think it is a good compromise to drop that line so that people who care about the 3rd level keys can configure a switcher, and people who don’t will not be affected.

With the attached patch one can make, say, right ctrl 3rd level switcher and still use right alt+shift to switch layout. Hopefully it can make it to 2.7 release.
Comment 28 Muhammad Saied 2012-11-18 16:43:36 UTC
(In reply to comment #25)

> That is a strong accusation you have to back with evidence,

You can read the recommendations of the Cairo Conference for the 1991 here:
http://www.majma.org.jo/majma/index.php/2009-02-10-09-36-00/625-mag40-9.html

#2 is the one relevant to this issue.

There is also the 4th lecture of the 12th cultural season of the Jordan Academy of Arabic (1994) p93-p120, I have the printed book but I don't think it's available on the web. 

I also have the Science and Art terms published by the Egyptian Academy (2002), and none of these letters is used in it.
Comment 29 Khaled Hosny 2012-11-18 17:14:35 UTC
(In reply to comment #28)
> (In reply to comment #25)
> 
> > That is a strong accusation you have to back with evidence,
> 
> You can read the recommendations of the Cairo Conference for the 1991 here:
> http://www.majma.org.jo/majma/index.php/2009-02-10-09-36-00/625-mag40-9.html
> 
> #2 is the one relevant to this issue.

This is discussing the issue of writing non-standard Arabic dialects, a totally different and irrelevant issue.

This is also getting really off-topic. The issue here is that users who want to be able to use right alt+shift keys to switch layout cannot do so now, so lets please focus on this and not the fine details of Arabic orthography and writing system.
Comment 30 Khaled Hosny 2012-11-19 17:22:30 UTC
(In reply to comment #27)
> Hopefully it can make it to 2.7 release.

Oops, I got the dates wrong! 2.7 was released in September already.
Comment 31 Benjamin Geer 2012-11-19 23:07:04 UTC
Thanks, Khaled, that looks like a good solution to me. And clearly I should give the language academies more credit. :)
Comment 32 Sergey V. Udaltsov 2012-11-19 23:13:50 UTC
cool. it seems everybody's happy. committed!


Use of freedesktop.org services, including Bugzilla, is subject to our Code of Conduct. How we collect and use information is described in our Privacy Policy.