Bug 89999 - Add support for 3-finger dragging
Summary: Add support for 3-finger dragging
Status: RESOLVED WONTFIX
Alias: None
Product: Wayland
Classification: Unclassified
Component: libinput (show other bugs)
Version: unspecified
Hardware: Other All
: medium normal
Assignee: Wayland bug list
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2015-04-13 01:59 UTC by Peter Hutterer
Modified: 2017-03-13 04:11 UTC (History)
6 users (show)

See Also:
i915 platform:
i915 features:


Attachments

Description Peter Hutterer 2015-04-13 01:59:49 UTC
Goal: three finger drag (without physical click) should be a drag-n-drop operation. This is available on macs with their touchpads and could/should be available on all devices that support three fingers.

Might be possible on synaptics pads with 2 finger + BTN_TOOL_TRIPLETAP.

Sequence:
* three fingers down
* exceed motion threshold or timeout -> button 1 down
* movement -> motion event
* transition to less than 3 fingers -> button 1 up

Note that this is different to swipe gestures, in this approach we send normal motion events. May put a requirement on timeout-only, since a motion event can look like a swipe until it's too late to change our mind.

The question is: do we add this as 3 finger gesture to the gesture API and let the caller deal with it, or emulate it in libinput directly? I'm tending towards the latter if we can make it work properly.
Comment 1 Hans de Goede 2015-04-13 07:42:45 UTC
Hi,

Adding this should be relatively easy with my gestures branch in essence this just replaces 3 finger swipes with drag+drop emulation. We do need a config option to choice between one or the other as I do not see a
way to do both. Which makes me think / believe that we should really leave this up to the toolkit, as that will know which of the 2 behaviors makes more sense when there is a 3 finger gesture.

Regards,

Hans
Comment 2 Peter Hutterer 2015-04-13 23:15:39 UTC
couple of question here, can we do something with a timeout? i.e. three fingers down + timeout X switches to drag motion, otherwise the swipe triggers.

we don't have any movement restrictions on swipe motions atm, right? anything is a swipe once 3 fingers are down, so maybe we should add restrictions to horizontal/vertical swiping here?

once of the drawbacks of having this via gestures if of course that we need both the compositor (for cursor movement) and the toolkits to support this. that's the upside too though I guess, it makes context-specific interaction more doable than emulating it in libinput.
Comment 3 Hans de Goede 2015-04-14 07:03:42 UTC
(In reply to Peter Hutterer from comment #2)
> couple of question here, can we do something with a timeout? i.e. three
> fingers down + timeout X switches to drag motion, otherwise the swipe
> triggers.

Yes that is possible, do we want to handle this in gestures.c or in tap.c though, I've a feeling it belongs in tap.c. because we do not want the following sequence to trigger it:

Finger 1 down
Finger 1 moves over tap threshold
Fingers 2 + 3 down
Timeout

Where as having it in the gesture code would lead to the timeout eventually activating 3 finger mode, and then if more time is passed before moving, switch to 3fg drag-n-drop. I guess one could argue that is desirable behavior, and in that case this is easy to implement in the gesture code, esp. since the tap statemachine already is too complicated IMHO.

> we don't have any movement restrictions on swipe motions atm, right?
> anything is a swipe once 3 fingers are down, so maybe we should add
> restrictions to horizontal/vertical swiping here?

I've deliberately not added any limits as I've seen e.g. usb-touchpads with firmware handled gestures which have diagonal swipes, so I would rather leave any direction locking up to the user.

> once of the drawbacks of having this via gestures if of course that we need
> both the compositor (for cursor movement) and the toolkits to support this.
> that's the upside too though I guess, it makes context-specific interaction
> more doable than emulating it in libinput.

Ack.
Comment 4 Peter Hutterer 2015-04-14 07:42:21 UTC
(In reply to Hans de Goede from comment #3)
> (In reply to Peter Hutterer from comment #2)
> > couple of question here, can we do something with a timeout? i.e. three
> > fingers down + timeout X switches to drag motion, otherwise the swipe
> > triggers.
> 
> Yes that is possible, do we want to handle this in gestures.c or in tap.c
> though, I've a feeling it belongs in tap.c. because we do not want the
> following sequence to trigger it:
> 
> Finger 1 down
> Finger 1 moves over tap threshold
> Fingers 2 + 3 down
> Timeout
> 
> Where as having it in the gesture code would lead to the timeout eventually
> activating 3 finger mode, and then if more time is passed before moving,
> switch to 3fg drag-n-drop. I guess one could argue that is desirable
> behavior, and in that case this is easy to implement in the gesture code,
> esp. since the tap statemachine already is too complicated IMHO.

I'd say that's definitely desirable behaviour, because you'd likely move the pointer somewhere, then just put the extra fingers down to drag without lifting the first finger first.

> > we don't have any movement restrictions on swipe motions atm, right?
> > anything is a swipe once 3 fingers are down, so maybe we should add
> > restrictions to horizontal/vertical swiping here?
> 
> I've deliberately not added any limits as I've seen e.g. usb-touchpads with
> firmware handled gestures which have diagonal swipes, so I would rather
> leave any direction locking up to the user.

right, makes sense.
Comment 5 Hans de Goede 2015-04-14 09:56:53 UTC
(In reply to Peter Hutterer from comment #4)
> (In reply to Hans de Goede from comment #3)
> > Finger 1 down
> > Finger 1 moves over tap threshold
> > Fingers 2 + 3 down
> > Timeout
> > 
> > Where as having it in the gesture code would lead to the timeout eventually
> > activating 3 finger mode, and then if more time is passed before moving,
> > switch to 3fg drag-n-drop. I guess one could argue that is desirable
> > behavior, and in that case this is easy to implement in the gesture code,
> > esp. since the tap statemachine already is too complicated IMHO.
> 
> I'd say that's definitely desirable behaviour, because you'd likely move the
> pointer somewhere, then just put the extra fingers down to drag without
> lifting the first finger first.

Ok, I was actually thinking the same (that this is actually desirable) but I wanted to double check with you, ok in that case it should be easy to add support for this using an initial timeout to choice between this and a 3fg swipe. I do believe that this should be only enabled when tapping is enabled.
Comment 6 Sui Chen 2015-04-14 15:46:54 UTC
(In reply to Hans de Goede from comment #5)
> (In reply to Peter Hutterer from comment #4)
> > (In reply to Hans de Goede from comment #3)
> > > Finger 1 down
> > > Finger 1 moves over tap threshold
> > > Fingers 2 + 3 down
> > > Timeout
> > >  (omitted)
> > 
> > I'd say that's definitely desirable behaviour,  ... (omitted)
> 
> Ok, I was actually thinking the same (that this is actually desirable) but I
> wanted to double check with you, ok in that case it should be easy to add
> support for this using an initial timeout to choice between this and a 3fg
> swipe. I do believe that this should be only enabled when tapping is enabled.

Hi,
I am playing with emulating 3-finger drag in the Synaptics driver these days (now I think I should look into libinput), and I agree it's convenient to allow the user to put the extra fingers down to drag.

I think there is another thing that may need to be taken care of: the 3-finger drag should also support "locked dragging". 
Reason is it's easy to run out of space on a trackpad with 3 fingers. If the user wants to do a long distance drag (like drawing a window to another monitor), s/he has to either do a drag with a high initial speed, or keep swinging back and forth to take advantage of the acceleration mechanism. That is easy to do with 1 finger dragging, but with 3 fingers there is greater friction so it's harder. Because of this, a locked dragging is more convenient for long-distance 3-finger drags.

IMHO it may look like the following in the program:
1. The user is already in a 3-finger drag.
2. The user lifts 3 fingers from the trackpad because s/he is running out of space. The countdown starts.
3. Any time before the countdown reaches zero, the user may move the cursor with 1 or 2 or 3 fingers (that means 2-finger scroll is ignored).
4. If the user puts 3 fingers back onto the trackpad to continue the 3-finger drag any time before the countdown reaches zero, the countdown resets.
5. If the countdown reaches zero the drag is stopped. (It may also be stopped by other events like pressing the physical button on the trackpad)

Thanks,
Sui
Comment 7 Peter Hutterer 2015-04-16 05:59:41 UTC
Carlos, any comments from the GTK/Compositor side of things?
Comment 8 Carlos Garnacho Parro 2015-04-16 11:51:02 UTC
(In reply to Peter Hutterer from comment #7)
> Carlos, any comments from the GTK/Compositor side of things?

If this gesture is meant to only start DnD (as in the user-visible distinct operation), I've got the feeling it will be better to get the compositor/client involved. Otherwise, it will be really hard to tell whether DnD can be started from the pointer position, you might as well be selecting text, clicking a button, doing nothing... 

One thing I wonder though is how would this work best if 3fg pointer movement is only meant to happen during this operation, and remain static otherwise. I'm maybe overcomplicating things in my head though.

If the gesture is rather seen as a shortcut to "button 1 drags", then I guess you have as much information as you need to do that, it would be pointless to punt this any further.
Comment 9 Peter Hutterer 2015-04-16 12:14:25 UTC
short answer: we can make it whatever we want. on the mac it seems to be just a shortcut to button 1 drags. a compositor integration would enable us to be more context-specific here, e.g. make a three finger drag a "move window" drag. wehther we want that is another question, so I pretty much defer to you. we can implement either in libinput.
Comment 10 Peter Hutterer 2015-07-14 02:57:32 UTC
After some more discussion with carlos over email we decided to push this into the compositor. This way gesture support is controlled in one place and a bit more flexible.
Comment 11 Caibin Chen 2016-01-06 05:37:56 UTC
Sorry for commenting on a closed bug.

I hacked tap.c to add 3fg dragging support: https://github.com/tigersoldier/libinput/tree/wip/three-fingers-dragging

I agree doing this in compositor is more flexible. OS X seems to treat 3fg dragging and 1fg click dragging differently. One difference is that 3fg dragging on selected text won't trigger dnd. It unselects the text and restart a new selection.

However, I wonder how dragging lock can be implemented at compositor level. A simple timeout won't detect any taps between two 3fg drags.
Comment 12 Peter Hutterer 2016-01-06 06:23:46 UTC
a 3fg drag is effectively a 3 finger swipe gesture. we already provide that from libinput, so it's up to the higher levels to convert that accordingly. At least on OS X you don't tap before a 3fg drag, just the gesture itself triggers the drag (iirc).
Comment 13 Caibin Chen 2016-01-06 06:52:32 UTC
It's more than 3fg swipe if you put dragging lock into consideration. Is there an easy way to implement dragging lock on top of 3fg swipe? In my mind the high level has to check not only the interval between two swipes, but also if there are any other touchpad actions between these two swipes.
Comment 14 Peter Hutterer 2016-01-07 23:17:06 UTC
I think we can get away with effectively putting a timeout on the action and then simply canceling drag lock on timeout, or any other event coming from the touchpad. Any action that should cancel it (button click, pointer motion, scroll events) produces an event, any internal action would cause the timeout, thus cancelling it in the caller.
Comment 15 Sever Oraz 2016-07-03 18:58:30 UTC
What happened to this bug? I am very interested in 3fgd and all the comments seemed to point to a success in implementation until I read the status is resolved wontfix :(
Comment 16 Peter Hutterer 2016-07-03 22:55:13 UTC
See comment #10 for the summary, we're pushing this to the compositor.
Comment 17 Sever Oraz 2016-07-04 13:00:03 UTC
(In reply to Peter Hutterer from comment #16)
> See comment #10 for the summary, we're pushing this to the compositor.

Great, is there a ticket or something where we can track progress of this feature?
Comment 18 Peter Hutterer 2016-07-05 00:34:05 UTC
I filed one for mutter now, but for other compositors it'll have to be filed in their respective bug trackers. Please do so if you use kwin, enlightment, etc.

mutter bug: https://bugzilla.gnome.org/show_bug.cgi?id=768421
Comment 19 Schlomo Schapiro 2017-03-10 08:36:42 UTC
Hi everybody, please reconsider the decision in comment #10 to push three-finger-drag into the compositor.

The reason is that the user who initially asked for this has a very simple request: Get the same behavior as on Mac OS X. There you can do three-finger-drag to select text, do drag and drop and move around things on the screen.

I have the feeling that the discussion here steered off that goal into drag and drop only, see the original request by Sen Chui at https://lists.freedesktop.org/archives/xorg/2015-April/057286.html). I think that the main use case is much more than just drag and drop and that text selection is probably the primary use case for most people. This is due to the fact that three-finger-drag allows selecting text without any interruption in the flow whereas double-tap-drag requires you to lift off your finger from the touchpad.

So, if to go back to the original goal to have an alternative to double-tap-drag wouldn't libinput and the corresponding Xorg input driver the right place for that?

Another thing is that now in 2017 most users still use Xorg (e.g. on Ubuntu 16.04 LTS) so that IMHO it is worth the effort to help those users now.
Comment 20 Peter Hutterer 2017-03-13 04:11:51 UTC
from the input driver's perspective, a three-finger drag looks identical to a three-finger swipe gesture, without context of the underlying widget we don't know which one is appropriate. pushing it into libinput means it's less accessible, less configurable and less context-aware.

libinput provides sufficient information for a compositor to convert a gesture into a drag&drop operation. on the X side this conversion would have to be in xf86-input-libinput, though that too lacks the context awareness. 

Either way, this is not something to be fixed in libinput, sorry.


Use of freedesktop.org services, including Bugzilla, is subject to our Code of Conduct. How we collect and use information is described in our Privacy Policy.