I work in game development and I am looking to test some of my games on Wayland. I currently have the UE4Editor running on Fedora 22 GNOME Shell Wayland without X11. This is related to the following issues: https://bugs.freedesktop.org/show_bug.cgi?id=84014 https://bugs.freedesktop.org/show_bug.cgi?id=85715 Firstly I suggest you read the UE4 source first (it is now free if you accept the EULA) to see how it is implementing mouse input. IDEA: 1. Raw motion: I propose a new interface wl_input which will receive generic libInput events allowing the application to receive and then process these libinput events directly. It will only receive these events if the compositor allows; either because the window has focus or the device is a global device. These could also be sensor devices or any other HID device. This will include a new libinput relative axis event which will support the need for a "relative pointer". There would be an option for opening the default device for a specific seat which would mostly be the same mouse as the wl_pointer (but should not always be assumed). This would allow axis events from an arbitrary list of devices to be mapped into a game input, allowing a user to bind a mouse to look and thumbstick to move, all from the same api. The game would always still receive the normal wl_pointer motion events if the wl_pointer would be inside the input area allowing it to draw on screen cursors in the correct place. This will then be extended to support joystick devices and 6dof, and hmds all as a generic libinput device. The axis events will transmit all information about dead-zones, sensitivity, normalization and DPI should any of this information change in-fight by WM settings. 2. Locking/Confinement: If the application opens the default device and it is the same device as the wl_pointer (as in moving the wl_input device will move the wl_cursor) then it is up to the compositor to alert the user etc. based on sandbox settings and then the confinement can happen. It is as if the application has become fullscreen, outside the window area the cursor will be hidden. If the user sets the default device to a different mouse or a gamepad axis then the game will not receive a lock/confinement as it does not need one. Applications should not assume the pointer can ever become confined or locked by any direct action it takes itself or that a user wants to play a game using the default mouse. I may want any device to be picked as my default 2D relative movement device, including a second mouse I may have plugged in. REASONING: The main sticking point is the idea of a "relative pointer". Firstly a "relative pointer" does not exist. Pointer != mouse != cursor. A mouse is a relative device, the information retrieved is an axis delta and not an absolute position. It may or may not have acceleration applied at various levels. When used for input in a game this is just another axis. It is not a pointer. A pointer is an absolute device with an input area. What the device is pointing at in the input area is a fixed absolute position with no acceleration. You cannot lock, grab or restrict pointers because there is no feedback to the device to prevent its movement. What we are currently doing is mapping a mouse into a pointer with an infinite input area and calling this point the "cursor". You can move or hide and lock the cursor because the mouse is a relative device. But when the cursor is also used for a pointer type device you can no-longer do this. I think it is a mistake to base off the current idea SDL has of a "relative mouse mode" because this is an abstraction of many different operating systems and does not map directly to the wayland idea of a pointer. If an application is intending to receive relative motion events it should be using a completely separate interface than the wl_pointer as it could be a completely different device such as an onboard accelerometer. The wl_pointer co-ordinates should still be wherever the point is pointing at. Most games currently use Windows and Windows mostly use Raw HID input events received via a WM_INPUT event to handle the "high precision", "raw" mouse. They do not use anything related to "relative" mouse at all. UE4 for example still expects that the mouse will be visible and move around as normal when in this mode and requests the lock/capture separately. This api is fairly analogous to libinput and can support any HID device. Secondly some devices such as gamepads, joysticks, HMDs, 6DOF input device, mice may also be used by many applications and need to be associated with a certain seat and window/surface focus. When swapping between games for example, the gamepad should only input in the current focus. Adding in all these different cases into the wayland protocol is not viable. I will try to work out a small bit of code and submit it but I am already working on too many projects and I may not have time.
I came up with a quick usage scenario that may help to understand some of the ideas: A user has a mouse, keyboard and PS4 controller plugged in. The PS4 controller has a trackpad with a single click interface and the user may have set the compositor to use the PS4 trackpad and the mouse to the wl_pointer. The compositor can tell the user is using the PS4 controller based on sensor (gyro/accelerometer) data from the controller and sets the controller as "default" for relative/game input. The user may be sat some distance away from the screen unable to use the mouse and keyboard. The user opens a game which requests the "default" input device from Wayland the game does not care what it is just that it can be used to play games or get relative 2 axis motion. Wayland either sends a pointer to libinput events or a fd to a marshaled evdev device. The game can automatically map the first axis of the PS4 controller which the user may have set up to be the right thumb stick as the axis it uses for "mouse look". Or if the application is fully aware it can just switch to gamepad mode. The compositor does not need to confine or lock the pointer as the trackpad is a different device to the gamepad. When the user uses the PS4 trackpad they move the pointer outside of the window and click on another window such as a music application. Change the song, then return to the game window. The user may then press the home button on the PS4 controller and the application becomes fullscreen or works as a task switcher. Any subsequent or long presses of the home button will open exposay or some special full-screen compositor overlay interface. The user moves back to the desk and puts the controller down and moves the mouse. The compositor then switches the default input device back to mouse and either the game can run as keyboard and mouse mapped to gamepad axis and buttons (bindings taken care of in the WM settings) or if programmed correctly can switch back to keyboard and mouse mode. If you check out something like Steam OS it would be quite nice if it could use this functionality directly in the compositor and set up all of the input device ids and mappings directly. Rather than having Steam as a separate application running on-top of GNOME and then have game interpret the evdev data by hoping it uses the correct mapping information from SDL. The other case is where a user is using a "magic mouse" or mouse which has a built in touch area. The user sets up the compositor to switch the wl_pointer to the trackpad when the game requests the "default" input device. The game does not need pointer lock or confinement because the user can use the trackpad to move the pointer but the mouse will be used in game. Also the relative events no-longer come from the same device as the wl_pointer.
-- GitLab Migration Automatic Message -- This bug has been migrated to freedesktop.org's GitLab instance and has been closed from further activity. You can subscribe and participate further through the new bug through this link to our GitLab instance: https://gitlab.freedesktop.org/wayland/wayland/issues/20.
Use of freedesktop.org services, including Bugzilla, is subject to our Code of Conduct. How we collect and use information is described in our Privacy Policy.