Bug 92818

It's correct per the type, AFAICT:
    type "CTRL+ALT" {
        modifiers = Control+Alt+Shift+LevelThree;
        map[None] = Level1;
        map[Shift] = Level2;
        map[LevelThree] = Level3;
        map[Shift+LevelThree] = Level4;
        map[Control+Alt] = Level5;
        preserve[Shift] = Shift;
        preserve[Shift+LevelThree] = Shift;
        level_name[Level1] = "Base";
        level_name[Level2] = "Shift";
        level_name[Level3] = "Alt Base";
        level_name[Level4] = "Shift Alt";
        level_name[Level5] = "Ctrl+Alt";
    };

The type - if you check the key definitions - is used to implement VT switching, as Ctrl+Alt+F1 will generate XF86_Switch_VT_1. So that accounts for all of them: Mod1 being Alt, and Mod5 being LevelThree/AltGr.

What seems incorrect about it ... ?

(In reply to Daniel Stone from comment #1)
> It's correct per the type, AFAICT:
>     type "CTRL+ALT" {
>         modifiers = Control+Alt+Shift+LevelThree;
>         map[None] = Level1;
>         map[Shift] = Level2;
>         map[LevelThree] = Level3;
>         map[Shift+LevelThree] = Level4;
>         map[Control+Alt] = Level5;
>         preserve[Shift] = Shift;
>         preserve[Shift+LevelThree] = Shift;
>         level_name[Level1] = "Base";
>         level_name[Level2] = "Shift";
>         level_name[Level3] = "Alt Base";
>         level_name[Level4] = "Shift Alt";
>         level_name[Level5] = "Ctrl+Alt";
>     };
> 
> The type - if you check the key definitions - is used to implement VT
> switching, as Ctrl+Alt+F1 will generate XF86_Switch_VT_1. So that accounts
> for all of them: Mod1 being Alt, and Mod5 being LevelThree/AltGr.

Hmm, right, VT switching...

> 
> What seems incorrect about it ... ?

I guess only my expectancies about xkb_state_key_get_consumed_mods() being behaviorally compatible with gdk_keymap_translate_keyboard_state() "consumed_modifiers" argument :).

There, we only mark the modifiers that would end up producing a different keysym if toggled. However ctrl+f1 alone doesn't produce a different keysym, nor alt+f1 (and ctrl+alt+f1 is unseen by clients). But we've got all those bits in the mask that end up ignored, and we see <any-of-these-mods>+f1 seen as f1.

I guess the odd thing is that here you need 2 modifiers enabled to get a different keysym, whereas in eg. ALPHABETIC you only need 1 bit, I can't tell if that's also the case anywhere else though.

All in all, feel free to disregard this as NOTABUG, I see some sense with the "vt switching" piece of the puzzle I was missing, even though it's not the behavior I was expecting from these functions. I might also mimick the "enable/disable modifier bits and see if the resulting keysym is different" business we do on x11.

(In reply to Carlos Garnacho Parro from comment #2)
> I guess only my expectancies about xkb_state_key_get_consumed_mods() being
> behaviorally compatible with gdk_keymap_translate_keyboard_state()
> "consumed_modifiers" argument :).
> 
> There, we only mark the modifiers that would end up producing a different
> keysym if toggled. However ctrl+f1 alone doesn't produce a different keysym,
> nor alt+f1 (and ctrl+alt+f1 is unseen by clients). But we've got all those
> bits in the mask that end up ignored, and we see <any-of-these-mods>+f1 seen
> as f1.

Oh, hmm ... interesting.

> I guess the odd thing is that here you need 2 modifiers enabled to get a
> different keysym, whereas in eg. ALPHABETIC you only need 1 bit, I can't
> tell if that's also the case anywhere else though.
> 
> All in all, feel free to disregard this as NOTABUG, I see some sense with
> the "vt switching" piece of the puzzle I was missing, even though it's not
> the behavior I was expecting from these functions. I might also mimick the
> "enable/disable modifier bits and see if the resulting keysym is different"
> business we do on x11.

I hadn't realised GTK+ behaved that way. Ran, any thoughts?

(In reply to Daniel Stone from comment #3)
> (In reply to Carlos Garnacho Parro from comment #2)
> > I guess only my expectancies about xkb_state_key_get_consumed_mods() being
> > behaviorally compatible with gdk_keymap_translate_keyboard_state()
> > "consumed_modifiers" argument :).
> > 
> > There, we only mark the modifiers that would end up producing a different
> > keysym if toggled. However ctrl+f1 alone doesn't produce a different keysym,
> > nor alt+f1 (and ctrl+alt+f1 is unseen by clients). But we've got all those
> > bits in the mask that end up ignored, and we see <any-of-these-mods>+f1 seen
> > as f1.
> 
> Oh, hmm ... interesting.

FWIW, the place where this happens is:
https://git.gnome.org/browse/gtk+/tree/gtk/gtkkeyhash.c#n442

This is the code that looks up actions associated to keybindings (eg. Ctrl-+, Ctrl-F1). There we have these checks:

  (state & ~consumed_modifiers ...) == (modifiers & ~consumed_modifiers ...)

that effectively disable all the consumed modifier bits from the checks (the example in gtk+ docs being Ctrl-+ being actually Ctrl-Shift-= in american keyboards, so Shift must be ignored for matching). 

consumed_modifiers comes more or less straight from our gdk_keymap_translate_keyboard_state() function (goes through the _gtk_translate_keyboard_accel_state() wrapper, but it doesn't do much on top that affects here).

The short story is: our current behaviour is what the XKB standard requires, and what Xlib implements, and what the existing keymaps expect; hence I don't think we should change it (unless the above statements are not correct in some way). The standard itself does not really expand on the issue, but it does say that the modifiers from the type's mask are consumed even if they would not affect the result:

http://www.x.org/releases/X11R7.7/doc/kbproto/xkbproto.html#Key_Types

There is some semi-related discussion in another bug (a bit long):

https://github.com/xkbcommon/libxkbcommon/issues/17

And a commit it links to:

https://github.com/xkbcommon/libxkbcommon/commit/5417440970f15673fbc128108245ad0697fcadc0

GTK X11 has its own custom interpretation of "consumed", which I'm sure took a lot of time to get to play well with the implementation of keyboard shortcuts. See for example here:

https://github.com/GNOME/gtk/blob/cdd3e05fba8992746bcd80b286137d3bfc258d7c/gdk/x11/gdkkeys-x11.c#L1128

In order for the wayland backend to behave the same, you will almost certainly have to bend what xkbcommon gives you to your will.

If testing against a scratch xkb_state with different modifier masks works, then I don't see a problem with that (but there surely are edge cases...). What you can do though, is only iterate over the modifiers which are in the consumed mask returned by xkbcommon (or maybe all subsets of them?); this way you don't test unrelated modifiers, and automatically respect the "preserve" clauses of the type.

Closing in favor of https://github.com/xkbcommon/libxkbcommon/issues/17.