Bug 100740

Summary: Blender viewport "goes crazy" when mouse reaches viewport window edge in fly mode (Shift+F) or moving object (G)
Product: Wayland Reporter: Source0f1 <jesse.dubord>
Component: XWaylandAssignee: Wayland bug list <wayland-bugs>
Status: RESOLVED MOVED QA Contact: Xorg Project Team <xorg-team>
Severity: normal    
Priority: medium CC: jadahl, jesse.dubord
Version: unspecified   
Hardware: x86-64 (AMD64)   
OS: Linux (All)   
Whiteboard:
i915 platform: i915 features:

Description Source0f1 2017-04-20 16:55:28 UTC
So, 3 issues:
(1) Using fly-mode in viewport (Shift+f) causes the camera to move sporadically.
(2) Moving an object (g) and pushing the mouse past the edge of the viewport window causes the object to move sporadically.
(3) Screen flickers pretty bad during both instances (1) and (2).

DETAILS
What happens in Blender 3D editor is, sometimes you can "fly" through your 3D scene by entering fly mode (Shift+f in viewport window). This gives you a first-person camera view and controls similar to conventional first-person shooter PC games: you can use the mouse and W, S, D and A to move through the scene.

Problem is, running Blender 2.78c under Wayland produces an issue when in this fly mode: the viewport screen goes ALL over the place. It seems to be involved with moving the mouse, specifically.

Under X window system, when you're in fly mode or moving a 3D object/mesh in the 3D viewport window, once the mouse reaches the edge of the viewport screen, it will "roll over" to the other side and allow the user to continue moving the object. The mouse will travel across the viewport window, again, and once it reaches the edge of the viewport window, it will roll over to the opposite edge of the viewport.

This issue seems to come up for moving objects as well. Once you press G to move an object, and move it along a certain axis, once the mouse reaches the edge of the Viewport window, the select object's position goes ALL over the place.

The screen also flickers pretty regularly during fly mode (Shift + f) or moving an object (G) in viewport.

I checked with the Blender dev's, and they said it's not a bug with the program.

SYSTEM SPECS
Running:
Blender 2.78c
Ubuntu GNOME 17.04 64-bt
GNOME 3.24.0
Wayland 1.7? (whichever version is in the Ubuntu repositories for their 17.04/Zesty release)
Linux kernel 4.10.0-19-generic

Feel free to email me and let me know if I can provide greater details. Thanks!
Comment 1 Jonas Ådahl 2017-04-21 03:14:21 UTC
Does blender have a Wayland backend, or is it still using X11 via Xwayland? If the latter, I suspect this is either a shortcoming of the Xwayland pointer warp emulation that might be improved by the wine-driven pointer warp emulation patches. Or possibly a bug in mutter, as you seem to use GNOME. If its the former, it is more likely a bug in Blender. Anyway, the answer to the question would make it possible to determine where to move this bug, as it doesn't have anything to do with "wayland" the IPC library where it is filed under now.
Comment 2 Source0f1 2017-04-24 16:35:53 UTC
I forwarded your question to Blender's Stack Exchange page (see link: https://blender.stackexchange.com/questions/78414/does-blender-have-a-wayland-backend). Once I get an answer, I'll forward the answer here and you can direct the bug wherever it's appropriate.

Appreciate the reply and the help!
Comment 3 Olivier Fourdan 2017-04-25 08:18:06 UTC
FWIW Blender uses Xwayland and this definitely looks like it has to do with pointer warp emulation (can be reproduced at will)

fly-mode behaves as if the pointer was over-reactive.
Comment 4 Olivier Fourdan 2017-04-25 08:22:11 UTC
Same behavior under weston, the objects always move toward the same direction regardless of where the pointer is moved, it seems (like top even when moving the pointer toward to bottom, and right even when moving the pointer toward the left)

=> Moving to Xwayland
Comment 5 Jonas Ådahl 2017-04-25 08:36:13 UTC
(In reply to Olivier Fourdan from comment #4)
> Same behavior under weston, the objects always move toward the same
> direction regardless of where the pointer is moved, it seems (like top even
> when moving the pointer toward to bottom, and right even when moving the
> pointer toward the left)

Weston, last time I checked, did not have the adaptations needed for Xwayland. So far I only know of mutter that is forgiving enough to let Xwayland pretend to warp the pointer.

> 
> => Moving to Xwayland

It's possible that they might be fixed by the https://lists.freedesktop.org/archives/xorg-devel/2016-November/051819.html series. Note to self, I should look at that again.
Comment 6 Olivier Fourdan 2017-04-25 08:41:29 UTC
Carlos' Wine patches [1] series crash Xwayland with blender:

(EE) 
(EE) Backtrace:
(EE) 0: /home/ofourdan/local/bin/Xwayland (OsSigHandler+0x29) [0x4765f9]
(EE) 1: /lib64/libpthread.so.0 (__restore_rt+0x0) [0x7f7b95e265bf]
(EE) 2: /home/ofourdan/local/bin/Xwayland (xwl_cursor_warped_to+0xc0) [0x4244e0]
(EE) 3: /home/ofourdan/local/bin/Xwayland (ProcXIWarpPointer+0x1e5) [0x558b95]
(EE) 4: /home/ofourdan/local/bin/Xwayland (Dispatch+0x2a6) [0x43b8d6]
(EE) 5: /home/ofourdan/local/bin/Xwayland (dix_main+0x3d8) [0x43f8c8]
(EE) 6: /lib64/libc.so.6 (__libc_start_main+0xf1) [0x7f7b95a6f401]
(EE) 7: /home/ofourdan/local/bin/Xwayland (_start+0x2a) [0x42323a]
(EE) 8: ? (?+0x2a) [0x2a]
(EE) 
(EE) Segmentation fault at address 0x28
(EE) 
Fatal server error:
(EE) Caught signal 11 (Segmentation fault). Server aborting
(EE) 

With gdb:

gdb) cont
Continuing.

Thread 1 "Xwayland" received signal SIGSEGV, Segmentation fault.
xwl_cursor_warped_to (device=<optimized out>, screen=<optimized out>, client=0x1ff7920, window=0x0, sprite=0x1c60b40, x=811, y=476)
    at xwayland.c:183
183	    if (!xwl_window && !window->parent &&
(gdb) bt
#0  xwl_cursor_warped_to (device=<optimized out>, screen=<optimized out>, client=0x1ff7920, window=0x0, sprite=0x1c60b40, x=811, y=476)
    at xwayland.c:183
#1  0x0000000000558b95 in ProcXIWarpPointer (client=0x1ff7920) at xiwarppointer.c:190
#2  0x000000000043b8d6 in Dispatch () at dispatch.c:479
#3  0x000000000043f8c8 in dix_main (argc=10, argv=0x7fff454791b8, envp=<optimized out>) at main.c:287
#4  0x00007f3685f89401 in __libc_start_main (main=0x423200 <main>, argc=10, argv=0x7fff454791b8, init=<optimized out>, fini=<optimized out>, 
    rtld_fini=<optimized out>, stack_end=0x7fff454791a8) at ../csu/libc-start.c:289
#5  0x000000000042323a in _start ()
(gdb) p window
$1 = (WindowPtr) 0x0
(gdb) 


[1] https://patchwork.freedesktop.org/series/15204/
Comment 7 Olivier Fourdan 2017-04-25 08:44:41 UTC
Fixing the crash leads to the same behavior though...

diff --git a/hw/xwayland/xwayland.c b/hw/xwayland/xwayland.c
index 623633d..4c7ab8b 100644
--- a/hw/xwayland/xwayland.c
+++ b/hw/xwayland/xwayland.c
@@ -176,6 +176,9 @@ xwl_cursor_warped_to(DeviceIntPtr device,
     struct xwl_seat *xwl_seat = device->public.devicePrivate;
     struct xwl_window *xwl_window;
 
+    if (!window)
+        return;
+
     if (!xwl_seat)
         xwl_seat = xwl_screen_get_default_seat(xwl_screen);
Comment 8 Olivier Fourdan 2017-04-25 08:51:37 UTC
Tried with gnome-shell/mutter with the Wine series and the fix (comment 7) and this is exactly the same, I reckon the Wine series are unrelated to this issue.

Also, I tried in Xorg (just to make sure, I am really not a Blender expert!) and the behavior is indeed entirely different, in fly-mode the view follows the pointer movement whereas in Xwayland it keeps going spinning in the same direction (comment 4) and very fast.
Comment 9 Olivier Fourdan 2017-04-25 09:23:08 UTC
Carlos' series:

https://patchwork.freedesktop.org/series/15204/

Plus this patch here:

diff --git a/hw/xwayland/xwayland.c b/hw/xwayland/xwayland.c
index 623633d..3b5c9a7 100644
--- a/hw/xwayland/xwayland.c
+++ b/hw/xwayland/xwayland.c
@@ -180,9 +180,10 @@ xwl_cursor_warped_to(DeviceIntPtr device,
         xwl_seat = xwl_screen_get_default_seat(xwl_screen);
 
     xwl_window = xwl_window_from_window(window);
-    if (!xwl_window && !window->parent &&
-        client == wClient(xwl_seat->focus_window->window)) {
-        DebugF("Warp on root window, assuming pointer focus\n");
+    if (window == None ||
+        (!xwl_window && !window->parent &&
+        client == wClient(xwl_seat->focus_window->window))) {
+        DebugF ("Warp on root window, assuming pointer focus\n");
         xwl_window = xwl_seat->focus_window;
     }
     if (!xwl_window)

Fix the issue for me.
Comment 10 Olivier Fourdan 2017-04-25 13:19:46 UTC
Also, note the patch from comment 9 avoids the test "client == wClient(xwl_seat->focus_window->window)" when the window is None, this is because in the case of blender (at least) we have the client being "blender" but the wClient(xwl_seat->focus_window->window) is gnome-shell itself, i.e. the focused window is one of gnome-shell owns.
Comment 11 Source0f1 2017-04-25 16:53:29 UTC
A lot of recent activity trying to troubleshoot this bug. Really appreciate it, guys. :)
Comment 12 Carlos Garnacho Parro 2017-05-28 14:24:51 UTC
FWIW my last patchset [1] makes shift-F work nicely. Moving objects past the edges still seems to behave oddly though, we probably need to translate subwindow coordinates into a toplevel-relative region for the confined pointer.

[1] https://lists.x.org/archives/xorg-devel/2017-May/053788.html
Comment 13 Carlos Garnacho Parro 2017-05-28 17:14:05 UTC
Nah I'm wrong. Blender seems to warp the pointer to the opposite side of the view in "move object" mode (pressing 'g'). Since xwayland warp emulator just kicks in when the pointer is hidden (and it's not the case), the warp gets ignored and Blender acts as if you were hitting the edge on every motion.

I'm not sure how can this be fixed though... Even if we allow warping while the pointer is visible, according to the protocol we can't lock onto a region the pointer is not in. Jonas, any ideas?
Comment 14 Jonas Ådahl 2017-06-08 03:39:53 UTC
(In reply to Carlos Garnacho Parro from comment #13)
> Nah I'm wrong. Blender seems to warp the pointer to the opposite side of the
> view in "move object" mode (pressing 'g'). Since xwayland warp emulator just
> kicks in when the pointer is hidden (and it's not the case), the warp gets
> ignored and Blender acts as if you were hitting the edge on every motion.
> 
> I'm not sure how can this be fixed though... Even if we allow warping while
> the pointer is visible, according to the protocol we can't lock onto a
> region the pointer is not in. Jonas, any ideas?

I think we might have discussed this on IRC (?) about warping after the cursor is hidden, but FWIW, the protocol does allow warping a locked pointer, it just requires the pointer to be within the lock region when locked.
Comment 15 Source0f1 2017-10-15 07:40:26 UTC
Did something change? I just tested Blender, today, on Ubuntu 17.10 beta 2 (ubuntu-modified gnome shell under wayland) and issue #1 appears to be fixed: pressing shift+f to enter fly mode and move around the scene no longer causes the camera to behave sporadically and it moves like it does in Xorg. Issues #2 and #3 still exist, however.

Still, progress is progress!
Comment 16 Jonas Ådahl 2017-10-16 03:38:12 UTC
(In reply to Source0f1 from comment #15)
> Did something change? I just tested Blender, today, on Ubuntu 17.10 beta 2
> (ubuntu-modified gnome shell under wayland) and issue #1 appears to be
> fixed: pressing shift+f to enter fly mode and move around the scene no
> longer causes the camera to behave sporadically and it moves like it does in
> Xorg. Issues #2 and #3 still exist, however.
> 
> Still, progress is progress!

Given the methods we have under Wayland (pointer locking and confinement instead of clients directly controlling the pointer cursor) we have implemented various support for emulating the X11 ways of implementing those features (locking/confinement). What Blender does however is taking control of the mouse cursor (without hiding it) and then moving it around on the screen (you'll see that it suddenly warps when hitting some edge in the UI).

This is the known remaining issue, and we cannot really solve it without going back to giving clients arbitrary control of the pointer cursor. Therefore we opened a bug on Blender about this issue, arguing that the issue would best be mitigated in Blender.

The bug was swiftly closed as wont-fix however, and the recommended work around is to disable the warp functionality. See https://developer.blender.org/T53004 for more details.
Comment 17 Source0f1 2017-10-16 04:58:34 UTC
(In reply to Jonas Ådahl from comment #16)
> (In reply to Source0f1 from comment #15)
> > Did something change? I just tested Blender, today, on Ubuntu 17.10 beta 2
> > (ubuntu-modified gnome shell under wayland) and issue #1 appears to be
> > fixed: pressing shift+f to enter fly mode and move around the scene no
> > longer causes the camera to behave sporadically and it moves like it does in
> > Xorg. Issues #2 and #3 still exist, however.
> > 
> > Still, progress is progress!
> 
> Given the methods we have under Wayland (pointer locking and confinement
> instead of clients directly controlling the pointer cursor) we have
> implemented various support for emulating the X11 ways of implementing those
> features (locking/confinement). What Blender does however is taking control
> of the mouse cursor (without hiding it) and then moving it around on the
> screen (you'll see that it suddenly warps when hitting some edge in the UI).
> 
> This is the known remaining issue, and we cannot really solve it without
> going back to giving clients arbitrary control of the pointer cursor.
> Therefore we opened a bug on Blender about this issue, arguing that the
> issue would best be mitigated in Blender.
> 
> The bug was swiftly closed as wont-fix however, and the recommended work
> around is to disable the warp functionality. See
> https://developer.blender.org/T53004 for more details.

Thanks for the update. I also wrote a comment in the Blender bug you linked, hoping greater collaboration could be done to find a more seamless solution than "turn off a setting in User Preferences to restore basic functionality for all Wayland/XWayland users". With Wayland/Xwayland becoming default for many, that would seem like a sore inconvenience for Blender users, everywhere.
Comment 18 Campbell Barton 2017-10-24 13:00:56 UTC
Hi, I help maintain Blender's X11 support,
even if this technically a bug in Blender it would be nice if it could be made to work.

I checked on a fix based on information from this thread,
to use `XGrabPointer` and constrain to a window.

However warping the cursor was still ignored (both in weston and gnome-shell).

Patch linked here:
https://developer.blender.org/T53004#467156

Any hints on how warping could be supported?

software versions latest at time of writing

- gnome-shell: 3.26.1+6+g78d58deb5-1
- wayland 1.14
- xorg-server-xwayland 1.19.5
Comment 19 Campbell Barton 2017-10-25 08:08:04 UTC
Looked into this further.

It seems to this can be made to work by hiding the cursor, moving it, then showing it again.

If supporting XWayland means developers have to spend time adding such hacks to their applications, I'm not sure what the point of such rules are.

XWayland should handle this internally.
Comment 20 GitLab Migration User 2019-05-10 15:52:53 UTC
-- GitLab Migration Automatic Message --

This bug has been migrated to freedesktop.org's GitLab instance and has been closed from further activity.

You can subscribe and participate further through the new bug through this link to our GitLab instance: https://gitlab.freedesktop.org/xorg/xserver/issues/702.

Use of freedesktop.org services, including Bugzilla, is subject to our Code of Conduct. How we collect and use information is described in our Privacy Policy.