Bug 89683 - Two finger scrolling isn't smooth
Summary: Two finger scrolling isn't smooth
Status: RESOLVED FIXED
Alias: None
Product: Wayland
Classification: Unclassified
Component: libinput (show other bugs)
Version: unspecified
Hardware: x86-64 (AMD64) Linux (All)
: medium normal
Assignee: Wayland bug list
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2015-03-19 19:48 UTC by Velimir Lisec
Modified: 2015-08-06 04:28 UTC (History)
3 users (show)

See Also:
i915 platform:
i915 features:


Attachments
evemu-record of 2fg scroll down gesture (44.15 KB, text/plain)
2015-03-19 19:48 UTC, Velimir Lisec
Details
2 finger scroll down gesture (183.04 KB, text/plain)
2015-04-22 12:23 UTC, Velimir Lisec
Details
2 finger scroll up gesture (97.80 KB, text/plain)
2015-04-22 12:23 UTC, Velimir Lisec
Details
2fg scroll left (55.14 KB, text/plain)
2015-04-22 12:24 UTC, Velimir Lisec
Details
2fg on the touchpad, but only one is moving (193.57 KB, text/plain)
2015-04-22 12:26 UTC, Velimir Lisec
Details
[PATCH] touchpad: Only use slot 0 deltas for 2fg scrolling on semi-mt touchpads (1.47 KB, patch)
2015-04-24 14:51 UTC, Hans de Goede
Details | Splinter Review
2fg scroll recording with semi-mt kernel fix (77.13 KB, text/plain)
2015-04-25 14:25 UTC, Velimir Lisec
Details

Description Velimir Lisec 2015-03-19 19:48:35 UTC
Created attachment 114480 [details]
evemu-record of 2fg scroll down gesture

I've tried running latest fedora rawhide image from a live USB on a Dell Inspiron n5110 and noticed that two finger scrolling on a touchpad isn't smooth like it is with synaptics driver. In applications that support smooth scrolling (gedit for example) scrolling is jumpy. Jumps aren't constant size, some are bigger, some are smaller. If i enable edge scrolling through xinput, scrolling behaves like it does under synaptics (it's smooth).

I've attached evemu-record of 2fg scroll event.
Comment 1 Peter Hutterer 2015-04-22 05:45:08 UTC
This is weird. In the recording, your fingers move in opposite directions for a number of events. In libinput, we average the movement of all fingers so the movement cancels out. The jumps come from when the fingers move in the same direction.

I put printf statements in libinput before replaying and this is the output:
---- tp_gesture_post_twofinger_scroll:123
Touch 0 0.00/10.42
Touch 1 0.00/-10.42
Delta 0.00/0.00
----
Delta after filter: 0.00/0.00
---- tp_gesture_post_twofinger_scroll:123
Touch 0 0.00/9.92
Touch 1 0.00/-9.92
Delta 0.00/0.00
----
Delta after filter: 0.00/0.00
---- tp_gesture_post_twofinger_scroll:123
Touch 0 0.00/9.92
Touch 1 0.00/-9.92
Delta 0.00/0.00
----
Delta after filter: 0.00/0.00
---- tp_gesture_post_twofinger_scroll:123
Touch 0 0.00/9.43
Touch 1 0.00/-9.43
Delta 0.00/0.00
----
Delta after filter: 0.00/0.00
---- tp_gesture_post_twofinger_scroll:123
Touch 0 0.00/8.43
Touch 1 0.00/19.35
Delta 0.00/13.89
----
Delta after filter: 0.00/5.62
---- tp_gesture_post_twofinger_scroll:123
Touch 0 0.00/7.94
Touch 1 0.00/49.61
Delta 0.00/28.78
----
Delta after filter: 0.00/13.06


That's the first couple of events, up to probably E: 0.389343 in the recording. This repeats itself, with the jumps in between where the delta goes to something normal. The movements aren't always identical, there is a later set like this:
---- tp_gesture_post_twofinger_scroll:123
Touch 0 0.00/9.43
Touch 1 0.00/-7.44
Delta 0.00/0.99
----
Delta after filter: 0.00/0.35
---- tp_gesture_post_twofinger_scroll:123
Touch 0 0.00/8.43
Touch 1 0.00/-8.43
Delta 0.00/0.00
----
Delta after filter: 0.00/0.00
---- tp_gesture_post_twofinger_scroll:123
Touch 0 0.00/7.44
Touch 1 0.00/-7.44
Delta 0.00/0.00

This looks very odd. Unless you're doing this intentionally this would point at some hw or firmware issue. Or a kernel bug. Can you confirm that you're moving the fingers in the same direction here?
Comment 2 Velimir Lisec 2015-04-22 05:51:28 UTC
I'm moving both fingers in the same direction, and at constant speed.
Comment 3 Peter Hutterer 2015-04-22 11:24:37 UTC
can you attach two or three more recordings please, I need to make sure that this really is the issue and it wasn't just something else.
Comment 4 Velimir Lisec 2015-04-22 12:23:01 UTC
Created attachment 115264 [details]
2 finger scroll down gesture
Comment 5 Velimir Lisec 2015-04-22 12:23:34 UTC
Created attachment 115265 [details]
2 finger scroll up gesture
Comment 6 Velimir Lisec 2015-04-22 12:24:19 UTC
Created attachment 115266 [details]
2fg scroll left
Comment 7 Velimir Lisec 2015-04-22 12:26:44 UTC
Created attachment 115267 [details]
2fg on the touchpad, but only one is moving
Comment 8 Peter Hutterer 2015-04-23 00:46:01 UTC
Related question: did it work before and do you know which one the most recent kernel version was that worked? If you use the synaptics xorg driver does it work better then?
Comment 9 Peter Hutterer 2015-04-23 05:59:51 UTC
ok, had a look through the recordings and they indeed show the same symptoms. in between normal-ish events there are events that cancel each other out. That's most pronounced in the last recording.

This is related to the device being a semi-mt. Those devices only provide a bounding box and if you check the center of the bounding box you can see exactly those jumps:

Touch 0: 0/222
Touch 1: 300/258
Bounding box: 0/222... 300/258
			center: 150/240
Touch 0: 0/226
Touch 1: 300/254
Bounding box: 0/226... 300/254
			center: 150/240
Touch 0: 0/231
Touch 1: 300/249
Bounding box: 0/231... 300/249
			center: 150/240
Touch 0: 0/235
Touch 1: 300/245
Bounding box: 0/235... 300/245
			center: 150/240
Touch 0: 0/239
Touch 1: 300/241
Bounding box: 0/239... 300/241
			center: 150/240
Touch 0: 0/243
Touch 1: 300/237
Bounding box: 0/237... 300/243
			center: 150/240
Touch 0: 0/247
Touch 1: 300/293
Bounding box: 0/247... 300/293
			center: 150/270
Touch 0: 0/251
Touch 1: 300/289
Bounding box: 0/251... 300/289
			center: 150/270

I honestly don't know how to fix this though.
Comment 10 Velimir Lisec 2015-04-23 07:29:35 UTC
(In reply to Peter Hutterer from comment #8)
> Related question: did it work before and do you know which one the most
> recent kernel version was that worked? If you use the synaptics xorg driver
> does it work better then?

It didn't work before. Whenever i tried using libinput I always had bad scrolling. It works great under synaptics driver. Scrolling is done pixel by pixel. 

I get precise recordings of finger movements from ABS_MT_SLOT 0, but SLOT 1 returns it's previous position minus movement from SLOT 0. Those two cancel out in libinput and touchpad isn't scrolling. But sometimes SLOT 1 gives correct recording of a finger position. That's when the jump happens. I guess synaptics driver only tracks SLOT 0 position for scrolling.
Comment 11 Hans de Goede 2015-04-23 09:58:34 UTC
Hi Velimir and Peter,

I've been looking into this today, mostly looking at the alps kernel code / the alps ps2 protocol. This problem affects alps v3 - v5 devices. Including the Rushmore (v3) touchpad on my Dell Latitude. As well as the
Dolphin (v5) touchpad on Velimir's inspiron.

The interesting thing is that both touchpads give us the same raw data at the ps/2 level (in slightly different formats): 1 more or less accurate touch, and a bitmap with columns and rows in which 1 or more fingers are seen resuting in a crude (low res) bounding box, but the kernel uses 2 different strategies for translating this into touch data.

The kernel code for v3 and v4 tablets simply takes 2 corners of the bounding box and reports those as touch1 and touch2, resulting in 2 touches with a crude resolution, but non of the "interesting" moving back / forth (saw tooth pattern) coordinates for the second touch.

The kernel code for v5 tablets takes the single accurate touch info, and calculates the distance to the center of the bounding box, and then puts the 2nd touch mirrored to the center. Since when the fingers only move a bit the bounding box does not move at all (because it has a low resolution) this results in one touch's coordinates  moving up and the other moving down, resulting in the saw tooth pattern we are seeing on
Velimir's inspiron.

Neither strategy is really good, the v3-v4 strategy throws away the accurate touch info we've for the first touch down, the v5 strategy has accurate touch info in touch1, but touch2 does the saw tooth pattern thing.

One thing we can do is stop using the "pointer emulation" code for mt devices in the kernel for these devices and report the accurate touch data for the first touch through ABS_X and ABS_Y events, while reporting the crude coordinates from the v3 and v4 strategy in ABS_MT_POSITION events. This way userspace has all info and can act on it.

Alternatively we can do a mix between the 2 strategies where we always use the accurate coordinates for touch1 and when a 2 touch sequence starts find which corner is closest to the accurate coordinate and use the crude coordinates of the opposite corner for touch2 for the duration of the 2 touch sequence
(we cannot pick the corner each event because that will result in weird jumps when the fingers are close together).

Either way will also need some libinput adjustments, and either way will make 2 fg scrolling on v3 / v4 devices much better with synaptics directly because synaptics uses the ABS_X and ABS_Y events which will now be accurate on v3 / v4 devices just like they already are on v5 devices.

Peter, which solution do you think is best ?

Regards,

Hans
Comment 12 Peter Hutterer 2015-04-24 01:42:36 UTC
If I read this correctly, the first suggestion would result in ABS_X carrying different information to ABS_MT_POSITION_X of either touch. I don't think that's a good idea but that's mostly gut feeling, I'm sure if that would break anything.

I think keeping the accurate touch info for touch 1 is better, so your second approach. What type of libinput changes do we need though? it looks like changing that in the kernel would give us sufficient information in libinput to just keep doing what we're doing right now.
Comment 13 Hans de Goede 2015-04-24 06:04:26 UTC
Hi,

(In reply to Peter Hutterer from comment #12)
> I think keeping the accurate touch info for touch 1 is better, so your
> second approach.

Ok, I will go work on the kernel side of that then.

> What type of libinput changes do we need though? it looks
> like changing that in the kernel would give us sufficient information in
> libinput to just keep doing what we're doing right now.

Even when touch1 is accurate then touch2 will still jump from one location to the next because it is still low resolution. Since we average the movement of both fingers for 2g scrolling this means that scrolling will be smooth until the 2nd touch jumps and then the scrolling will jump too, move smooth again, jump at the next transition, etc. So for 2fg scrolling on semi-mt touchpads libinput should only look at slot 0.

Regards,

Hans
Comment 14 Hans de Goede 2015-04-24 14:50:52 UTC
Hi,

Can you try building libinput with the attached patch? That should fix the problems you are seeing.

Also I've been working on some related kernel improvements, if you've experience with building kernels from source, can you please try the input-work branch of this git repo:

http://git.linuxtv.org/cgit.cgi/hgoede/gspca.git/log/?h=input-work

Thanks & Regards,

Hans
Comment 15 Hans de Goede 2015-04-24 14:51:26 UTC
Created attachment 115305 [details] [review]
[PATCH] touchpad: Only use slot 0 deltas for 2fg scrolling on semi-mt touchpads
Comment 16 Velimir Lisec 2015-04-24 16:17:01 UTC
(In reply to Hans de Goede from comment #14)
> Hi,
> 
> Can you try building libinput with the attached patch? That should fix the
> problems you are seeing.
> 
> Also I've been working on some related kernel improvements, if you've
> experience with building kernels from source, can you please try the
> input-work branch of this git repo:
> 
> http://git.linuxtv.org/cgit.cgi/hgoede/gspca.git/log/?h=input-work
> 
> Thanks & Regards,
> 
> Hans

Libinput with the patch applied is working great. Scrolling behaves just like it does under synaptics xorg driver. 

I have no experience with building kernels from source, but I'd like to try anyways. The problem is I can't seem to clone the repo. Here is the error:

Cloning into 'gspca'...
error: Unable to find 7655e594945289b418af39f6669fea4666a7b520 under http://git.linuxtv.org/cgit.cgi/hgoede/gspca.git
Cannot obtain needed object 7655e594945289b418af39f6669fea4666a7b520
while processing commit 858904e33a17953df6debd5907e991ee9e74ed21.
error: Fetch failed.
Comment 17 Hans de Goede 2015-04-24 16:35:58 UTC
Hi,

(In reply to Velimir Lisec from comment #16)
> Libinput with the patch applied is working great. Scrolling behaves just
> like it does under synaptics xorg driver. 

Thanks, good to hear.

> I have no experience with building kernels from source, but I'd like to try
> anyways. The problem is I can't seem to clone the repo. Here is the error:
> 
> Cloning into 'gspca'...
> error: Unable to find 7655e594945289b418af39f6669fea4666a7b520 under
> http://git.linuxtv.org/cgit.cgi/hgoede/gspca.git
> Cannot obtain needed object 7655e594945289b418af39f6669fea4666a7b520
> while processing commit 858904e33a17953df6debd5907e991ee9e74ed21.
> error: Fetch failed.

Ah yes the url provided on the web interface is not correct, I had forgotten about that, the correct url to clone is: git://linuxtv.org/hgoede/gspca.git

Regards,

Hans
Comment 18 Velimir Lisec 2015-04-25 14:23:22 UTC
I've built the input-work branch. There is no more saw tooth pattern appearing on slot 1. Without the libinput patch scrolling works better than it worked with the old kernel, but just as Hans said it's smooth at first, then it jumps, smooth again, then it jumps and so on. With libinput patch scrolling is always smooth. I've attached evemu-recording of 2 finger scrolling with the custom kernel.

One more thing that's bugging me is now that scrolling works great, I've noticed that there's a delay between me starting a scroll gesture and the scrolling actually happening on the screen. That's mostly noticeable when slow scrolling. I've tried measuring the distance required for me to move the fingers on the touchpad until scrolling starts, and it's somewhere between 3 and 5 mm. But after scrolling starts, any movement on the touchpad will result in scroll gesture. I have another problem with tapping, but I'll report separate bug for that.
Comment 19 Velimir Lisec 2015-04-25 14:25:36 UTC
Created attachment 115322 [details]
2fg scroll recording with semi-mt kernel fix
Comment 20 Peter Hutterer 2015-04-27 01:24:52 UTC
commit 07b20dcce6379bf21dc578858bce78026c25d729
Author: Hans de Goede <hdegoede@redhat.com>
Date:   Fri Apr 24 16:49:36 2015 +0200

    touchpad: Only use slot 0 deltas for 2fg scrolling on semi-mt touchpads


Hans, I'm leaving this bug open for the kernel behaviour, if you want to track this elsewhere feel free to close the bug.
Comment 21 Peter Hutterer 2015-08-06 04:28:53 UTC
afaict, this should be fixed with the series of kernel commits up to dccf1dd845823


Use of freedesktop.org services, including Bugzilla, is subject to our Code of Conduct. How we collect and use information is described in our Privacy Policy.