Bug 926

[ I mailed Ivan Pascal about the change to the xkb files ]

  Hi,

>> As you can see this change breaks among others the metacity window 
>> manager.  I'm not too familiar with XKB, could you suggest a fix for 
>> this?  I was thinking of adding back the line
>> 
>>    modifier_map Mod1   { Alt_L, Alt_R };
>> 
>> in the pc/pc(basic) section, which works for me, but I fear it may break 
>> something else.  Could you comment?


OK.  Let me explain.
It was a fix for some problem arised after addind muti-layout capability.
In such multi-layout you can combine (in differnt XKB groups), say, 'us' keymap
where right Alt key is just Alt and 'cz' keymap where this key is used as
AltGr i.e. for acces to third-fourth symbol of the key.  Or you can put 'ru'
keymap (where right Alt is Alt) above 'de' keymap where this key is AltGr.

But AltGr and Alt control differnt modifiers (Mod4 and Mod1 respectively) and
it means that such combination puts two differnt modifiers on the same key.

The roots of the problems are...
* XKB and core protocol have different approaches to modifiers.  In the core
protocol a modifier bit is attached to a keycode i.e. to a whole key.  Whereas
in XKB a modifier is controlled by a single keysym i.e. the same key can
set/reset different modifiers in dependence on other modifiers state or
the current XKB group (layout).

This differnce itself is not a big problem.  At least XKB approach is more
flexible and it sets modifiers well and an application gets their state
correctly.  But you know a part of modifiers are unnamed and an application
need to know what each Mod* means in each keymap.
And here we face a serious problem...

* All toolkits don't use directly XKB functions and...
* Xlib doesn't provide any public (core protocol) functions to find the mapping
between modifier bits and keysyms (like GetModifier(keysym) or
GetKeysym(modifier)).  If such mapping functions existed they could be replaced
with XKB's ones in Xlib invisibly for an application and the differnce in
approaches would not matter.

Instead Xlib provides a 'raw data' such as core protocol tables of 'modifiers
to keycodes' binding and 'keycodes to keysyms' table (the keymap itself)
and makes a toolkit to guess about Mod's binding in own way.  And the worst
thing is that diferent toolkits make is in different ways.

In the core protocol one-layout maps it was simple.  There was one Mod on
each modifier key and one keysym (at least one _modifier_ keysym) tied to
this key.  An application found modifier-keycode and keycode-modifier_keysym
pairs and made modifier-modifier_keysym mapping.

But in the multi-layout cases I said above we have ambiguity.  The right Alt
key has two Mods bound and at least two keysyms that are modifier ones.
If a toolkit asked XKB about mapping it would get clear answer (two separated
pairs).  But actually it tries to guess about this mapping from core protocol
tables where such mess can't be resolved.

As I said different toolkits in such cases come to differnt conclusions.
Some of them thinks that Alt is active only when both Mods (Mod1 and Mod4)
are set. Some other become crazy and think that Alt modifier is set always
even if none modifier key is pressed.  (You guess in this case you press the
letter key 'F' but an application instead of printing the letter opens 'File'
menu which hotkey is Alt+F).  And in the rest Alt key doesn't work anyhow.

To avoid these problems I removed all modifiers (unnamed Mods) from real keys
and made some 'fake keys'.  I took keycodes (scan-codes) that are never produced
by hardware keyboard and therefore I could use them as I like.
Using these never_can_be_pressed keys I made simple one_keycode-one_modifier
and one_keycode-one_keysym pairs to help application to guess about Mods
meaning (Mod* - keysym relations) even they use dumb core protocol tables.
The problem with "many modifiers and many keysyms on the same key" mess was
solved.

But it led to the problem you talk about.

The thing is that Alt+TAB feature (copied from MS Windows) doesn't fit well
X Window architecture.  In X's a window manager can't get key press/release
events from any key because they are passed to focused window only.
Of course there is GrabKey mechanism but it contrary passes events to the
grabing application but not to the focused window.  There even is a way to
repeat the grabed even for the focused window but it is too comples and need
the grabing application makes UnGrab and grab again for each such event.

Therefore no one WM (AFAIK) grabs modifier keys.  Instead a WM garbs 'TAB with
Alt modifier' combination only (in such case TAB alone or with any other
modifier arrives to the focised window).  But the problem is that when it
gets such grabed event it doesn't know what key had set this Alt modifier.
In the same time it must to know when this _unknown_ key will be released.

For it the WM searches all (some WMs even search the first one they can find)
keys (keycodes !) that have Mod1 modifier bound, grab them all and wait
some release event.  It need not to say that if modifiers are bound to
fake keys only the WM will never get such release event.

To solve (or rather to moderate) this problem I returned Mod1 (and some other
modifiers) to the real key left Alt.  This key usualy has not other meaning
(unless it is Meta key) and it should not cause any conflict between modifiers.
But the right Alt key (as I said) can be used for two (or even more) modifiers
in different layouts and I didn't dare to tie Mod1 to it.

The another one problem is that 'XKB language' doesn't have any 'if-then'
construction and I can't imagine how to make it guess when Mod1 can be put
onto this key (when a single layout is used or all of layouts use right Alt
in the same way) and when is shouldn't be done.

I see two ways only.  The first one is to return Mod1 to right Alt key but
don't put any other modifiers there.  But in this case in some multi layout
combinations this key will have one modifier but a few modifier keysyms.
Thus we can do it and ... wait new complains that something in some toolkits
doesn't work as expected.

And the second one is to make special XKB option that does only one thing -
maps Mod1 to right Alt key.  In this case all users who want right alt key
works as Alt have to specify this option explicitly in their configs.
I guess they will not be happy to learn such detail but it is most correct
solution I can offer now.

-- Ivan U. Pascal | e-mail: pascal@tsu.ru Administrator of | Tomsk State
University University Network | Tomsk, Russia 

[ More explanation from Ivan ]

   Hi,


>> Is it okay if I add your explanation to the Xorg bugzilla report?


Yes, of course.
But I forget one note about "WMs that provide Alt+TAB feature".

XKB could help them in this task.  Using XKB you can order a notification
about any modifier state changes.  Therefore if such WM used XKB features
it could just request special notification "Mod1 state is changed" and wait
this event instead of unreliable guessing what physical key controls this
modifier.


>>> > But AltGr and Alt control differnt modifiers (Mod4 and Mod1 respectively) and
>>> > it means that such combination puts two differnt modifiers on the same key.
>
>> 
>> And is this not possible? How did it work/not work before?


Before the mult-layout support all keymaps were staticaly composed and every
modifier key in them has only one modifier attached and one keysym that can
be treated as modifier keysym.  E.g. the right Alt key had Alt_R keysym and
Mod1 modifier or Mode_switch and Mod3 modifier but there was no one keymap
where this key has both keysyms or both modifiers.


>> Would it be possible for the application to test if XKB is in use and in 
>> that case use the XKB functions for mapping modifiers to keysyms, and if 
>> not, just look it up in the core modifier map?  Is this a 'good' solution?


I think so.  But most of tookits don't rely on XKB.  If we take it as
a solution we need to fix all toolkits.


>> How does this actually work? What modifier state is set when you press a 
>> key that generates a keysym which has two modifiers bound?


No, keysym never has two modifiers bound.  (Strictly speaking it CAN control
more than one modifier but nobody use it.  And it is reasonable.  There are
too few modifiers and the attaching two modifiers to one keysym just redices
the number of differen modifier we can use for different purposes).

OK.  I'll try to explain it with more details.
In the core protocol a modifier bit is attached to keycode (scan code).
Even if this physical key has no one keysym (symbol) assigned anyway when you
press this key the xserver sets this modifier in the 'keyboard state' and
reports this state in every key press/release event.  I.e. keysyms are not
needed to set/reset a modifier bit.  You can leave Shift keys without any
keysym but attach Shift modifier to them.  For usual text input it is enough.

But since most of modifiers (except Caps, Shift and Control) are unnamed,
an application needs somehow know what modifier means what.  If you attach,
say, Mod3 to left Windows key but don't assign any keysym to it the modifier
will be set at this key press but the application will not guess does it
mean Alt, Meta or any other modifier.  (I don't take Mod1/Alt modifier because
some toolkits have hardcoded relation "Mod1 means Alt"  :) .

The core protocol provides two tables:
* the keymap itself where each row match one keycode (scan code) and has
  two-four (actually up to 256) columns and each column can be filled with
  some keysym (symbol).  You can see this table using 'xmodmap -pk'.
* the modifier map where each row match one modifier and columns conatins
  keycodes of keys the modifier is attached to.  It has more then one column
  because the same modifier can be attached to two (or more) keys.  E.g.
  the Shift modifier usualy is attached to Left Shift and Right Shift keys.
  You can get this table running xmodmap without arguments.

When an application wants to know what unnamed modifier means Alt it ...
* searches Alt_L and Alt_R keysyms in the first table and gets the row number
  (that is keycode) where the keysym is found, and then ...
* searches this keycode in the second table and gets the row number that there
  means the modifier bit number.

The differences among differnt toolkits are that ...
some toolkits searches only first occurence of the interesting keysyms but
other searches all, some toolkits sums (with bitwise OR) all found modifiers
but others use the first or the last modifier found.

XKB uses another approach.  In XKB each keysym can have 'an action' bound
to it and this action is being performed in the server.  Actions can change
the 'keyboard state' (modifiers set, current layout and other 'control flags'),
emulate mouse events, emit some special XKB events and so on.

One of such actions is 'SetModifier(modifier)' that sets (and resets) the
specified modifier in the 'keyboard state'.  As I said an action is tied
to keysym, i.e. to a separate symbol.  In opposite to core protocol XKB doesn't
need a modifier bit be attach to something (keycode or keysym).  The modifier
that should be set with some keysym is explicitly specified in the SetModifier
action arguments.  (Strictly speaking XKB uses own internal modifiers with
clear names like Alt, Meta, LevelThree, etc.  But for reporting them to an
application in key events, XKB needs additional mapping of these internal
'virtual' modifiers to 'real' core protocols modifiers.  And for this mapping
XKB uses procedure like core protocols modifier attaching.  But it makes things
more complicated.  Therefore lets think that usual modifiers are specified
directly in the SetModifier arguments).

Thus with XKB we can put two such keysms on the same key, for example
Right Alt key   
keycode    first column         ...    third column
113        Alt_R                       ISO_Level3_Shift
           action= SetMod(Mod1)        action=SetMod(Mod4)
... and none Mods is attached anywhere.

If the current layout (XKB group) is 1 and I press this key the Alt_R keysym
is choosen and its action sets Mod1.  If the current layout is 2 (and points 
to third-fourth columns) the same key press chooses ISO_Level3_Shift and its
action sets Mod4.

But if an application using core protocols procedure (described above) tries
to find what Mod* means Alt, it finds nothing if we don't attach Mod1 to
this (or at least left Alt) key or will be confused if we attach both modifiers
(Mod1 and Mod4) to this Right Alt key.


>>  In other words, when does Alt_R set the Mod1 state and when does it set
>> the Mod4 state?


See above.  If you mean keysym Alt_R it is wrong.  Alt_R keysym always sets
Mod1 only.  But right Alt key (whole key) can have two or more keysyms that
are able to set own modifiers.  And what modifier will be set depends on
what keysym will be chosen (in dependence of the state of other modifiers
and the current layout).


>>> > For it the WM searches all (some WMs even search the first one they can find)
>>> > keys (keycodes !) that have Mod1 modifier bound, grab them all and wait
>>> > some release event.  It need not to say that if modifiers are bound to
>>> > fake keys only the WM will never get such release event.
>
>> 
>> So this is the core of the problem; there is no way to map from a 
>> modifier to the keysyms that it is bound to.


I would say that here we have another problem.
As I said it is possible to make clear one-to-one mapping between modifier
keysyms and modifier bits.  And give this mapping to an application in
traditional core protocol tables.

But for "Alt+TAB feature" an WM needs to know not this mapping only.
It needs to know exactly the key (because it can grab namely key not a keysym)
that had set this modifier.  But actually there is not such information.
And the WM does a trick.  It thinks that it is the key needed modifier is
bound to.  This binding is not needed for all other features (simple text
input or hot-keys catching).  And all other features don't need the modifier
be bound to all keys that sets this modifier (as I said XKB can set any
modifier without such Mod_to_keycode binding).


>>> > I see two ways only.  The first one is to return Mod1 to right Alt key but
>>> > don't put any other modifiers there.  But in this case in some multi layout
>>> > combinations this key will have one modifier but a few modifier keysyms.
>>> > Thus we can do it and ... wait new complains that something in some toolkits
>>> > doesn't work as expected.
>
>> 
>> What kind of problems would this give, more specifically?


Farnkly speaking - don't know.  But I didn't think that two modifiers on the
same key can cause problems (that appear differently in differnt toolkits)
until got reports about such problems.  And now I just fear to make such mix.
I know that it is ambiguity for applications and I think it _can cause_
a problem in some cases in some toolkits.


>> Yeah, we shouldn't have to burden the user with these details.  In GNOME 
>> there is a keyboard config tool to setup XKB, what if this tool could be 
>> extended to figure out from the layouts the user chooses how to 
>> configure Alt_R?  If there are no conflicts in the chosen maps, it would 
>> add and Alt_R option that would make Alt_R work correctly.


Yes.  I forget to mention that there is one more solution - to 'clean' a keymap
with some 'side software'.  But of course I don't like the solution that
solve the problem for Gnome users only ( and in Gnome versions later than xx).

IMHO, purists in the free software world all too often confuse "correct
solution" with "people will actually use the software" when they are entirely
incompatible.  Everyone and his brother (including me, someone who has been
writing software for half his lifetime) expects Alt+Tab to toggle windows
without further ado.

*** Bug 1405 has been marked as a duplicate of this bug. ***

*** Bug 587 has been marked as a duplicate of this bug. ***

sergey, this is generally fixed now, right?

Yes IIRC in the current xkeyboard-config it should be ok now.
Kristian, could you please confirm?

(In reply to comment #6)
> sergey, this is generally fixed now, right?

Bugzilla Upgrade Mass Bug Change

NEEDSINFO state was removed in Bugzilla 3.x, reopening any bugs previously listed as NEEDSINFO.

  - benjsc
    fd.o Wrangler

No answer for 2 years...