Bug 99212 - Button areas on Lenovo Thinkpad W540
Summary: Button areas on Lenovo Thinkpad W540
Status: RESOLVED FIXED
Alias: None
Product: Wayland
Classification: Unclassified
Component: libinput (show other bugs)
Version: unspecified
Hardware: Other All
: medium enhancement
Assignee: Wayland bug list
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2016-12-28 18:45 UTC by Markus Schmidt
Modified: 2018-01-09 03:52 UTC (History)
2 users (show)

See Also:
i915 platform:
i915 features:


Attachments
Hand position on Thinkpad W540 ready to do a left click with pre-configured button areas (84.00 KB, image/jpeg)
2016-12-28 18:46 UTC, Markus Schmidt
Details
Hand position on Thinkpad W540 ready to do a left click with custom button areas (90.17 KB, image/jpeg)
2016-12-28 18:47 UTC, Markus Schmidt
Details
Custom button areas for comfortably using a off-center clickpad (53.61 KB, image/jpeg)
2016-12-28 18:48 UTC, Markus Schmidt
Details

Description Markus Schmidt 2016-12-28 18:45:57 UTC
Dear libinput devs,

I'm regarding this one:

"For every feature, we try to understand why it is necessary, in which use-cases and how it can best be resolved. So our request to you is: please don't file a bug report for "implement the evdev driver's option Foo" or "allow this option to be set to value X". Explain what you are doing, where you need it done and let's see what the best solution is."

My problem is my lousy notebook, a Lenovo W540. The worst about it is its "clickpad" which is just a huge, rattly button with touch capabilities. Using it with the button areas pre-defined in libinput the hand position is simply not feasible since the clickpad is off-center to the left. See hand_position_bad.jpg for a picture of my hand ready to do a left click with the thumb. Please note that the image doesn't show the extreme angle my wrist snaps off while sitting in front of the computer.

With X and my latest driver (a self-compiled combination of evdev and synaptics) I was able to set very custom button areas as shown in clickpad.jpg. With this configuration my hand was resting as shown in hand_position_good.jpg, a position ready to do a left click again. As you can see in clickpad.jpg the areas for switching between left (yellow), middle (red) and right (green) click was just a question of a few millimeters which was very comfortable and restful. The gray area was behaving like a normal touchpad, only the coloured areas were configured as buttons without touch movement.

Using libinput with predefined areas my wrist is aking after a few minutes. Using the touch sensitive area below the middle button area one always has to lift the thumb in order to switch between button and touch zones. I think if we have this kind of "advanced" input hardware yet (buttons are just freely configurable areas on a touchpad) we probably should make use of its capabilities for custom use cases.

Best Markus
Comment 1 Markus Schmidt 2016-12-28 18:46:56 UTC
Created attachment 128673 [details]
Hand position on Thinkpad W540 ready to do a left click with pre-configured button areas
Comment 2 Markus Schmidt 2016-12-28 18:47:30 UTC
Created attachment 128674 [details]
Hand position on Thinkpad W540 ready to do a left click with custom button areas
Comment 3 Markus Schmidt 2016-12-28 18:48:47 UTC
Created attachment 128675 [details]
Custom button areas for comfortably using a off-center clickpad
Comment 4 Peter Hutterer 2017-01-04 07:22:35 UTC
sorry, I'm a bit confused. If I read this right, you want the button area to be like in attachment 128675 [details], i.e. the height reduced with the ability to left-click below the actual buttons?

which brings up one question: do you have the touchpad enabled or disabled? reason is that the buttons change size when disabled.

fwiw, I have a T440 here which has the same touchpad hardware but is (almost) centered on the laptop, so at least testing is easy. What I found is that putting a finger down in the button area, then moving into the bottom area and clicking does not trigger a left click, the click only works on an immediate press, not when there's movement. If you are on the edge of that boundary (and you may be because of the touchpad's location) you'd be triggering that, forcing you to press the button differently.

I wonder if fixing that would be sufficient? I'd be quite similar to what you're doing here anyway
Comment 5 Markus Schmidt 2017-01-04 10:25:11 UTC
Hey Peter,

thanks a lot for taking care of this report.

"which brings up one question: do you have the touchpad enabled or disabled?"

Touchpad is enabled. Reason is that there are other people using this notebook from time to time, inexperienced using the trackpoint.

"What I found is that putting a finger down in the button area, then moving into the bottom area and clicking does not trigger a left click, the click only works on an immediate press, not when there's movement."

To me it seems that there's the button area and there is the touchpad area. Switching from button area to touchpad area while the finger is touching the surface prevents from the default touchpad behavior and is simply doing nothing while swiping/clicking. The other way around (swiping from touch to button) the button area seems to be ignored and is used like the touchpad area. Both behaviors will last until one lifts the finger for a short amount of time after the movement which seems to "reset" the zones.

To me this absolutely makes sense regarding average clickpad usage. You don't want your pointers movement to stop just because you reach the button area while swiping. Otherwise shouldn't an unintentional swipe while in button area make the pointer move. So everything fine so far.

"I wonder if fixing that would be sufficient? I'd be quite similar to what you're doing here anyway"

Using the touchpad area as left click button unfortunately is not the same as having it configured like a "real" button since it allows movement of the pointer. So moving the pointer with the trackpoint where the click shall appear and then moving the thumb to the left click position will move the pointer a few pixels, too. This wouldn't be of a bigger problem for average desktop use but me as a graphic designer is depending on pixel-precise movement/clicks.

So far, all the best for your 2017

Markus
Comment 6 Markus Schmidt 2017-01-04 10:27:37 UTC
Sorry for not answering the first question:

"If I read this right, you want the button area to be like in attachment 128675 [details], i.e. the height reduced with the ability to left-click below the actual buttons?"

Exactly. I loved the way it was configurable freely because one is able to configure it depending on his hardware/wrist/sitting position/thum size.
Comment 7 Peter Hutterer 2017-01-06 05:20:26 UTC
(In reply to Markus Schmidt from comment #5)
> To me it seems that there's the button area and there is the touchpad area.
> Switching from button area to touchpad area while the finger is touching the
> surface prevents from the default touchpad behavior and is simply doing
> nothing while swiping/clicking. The other way around (swiping from touch to
> button) the button area seems to be ignored and is used like the touchpad
> area. Both behaviors will last until one lifts the finger for a short amount
> of time after the movement which seems to "reset" the zones.

yep, it is intended behaviour to ignore the buttons when moving up - this
increases the effective size of the touchpad area. I didn't realise that
physical clicks were ignored when coming out of the top button area though.

> Using the touchpad area as left click button unfortunately is not the same
> as having it configured like a "real" button since it allows movement of the
> pointer. So moving the pointer with the trackpoint where the click shall
> appear and then moving the thumb to the left click position will move the
> pointer a few pixels, too. This wouldn't be of a bigger problem for average
> desktop use but me as a graphic designer is depending on pixel-precise
> movement/clicks.

hmm, this shouldn't happen, palm detection is active while the trackpoint is
in use but I suspect your problem is that the click comes after the palm
detection expired (the timeout is quite short).

Exposing the full button configuration is something I'm very hesitant to
expose, especially for what is now 3 year old hardware. Plus, options fix it
on exactly one machine and then spread like an STD across forums, half the
time without the users knowing what they actually set. So I'm looking for
something that improves the touchpad for every user with a simple update.

there are a few options we have here: we could make the buttons bigger while
the trackpoint is in use and for longer afterwards. This would avoid the
issue with pointer movement but may still have the left button out of reach
for you. And from what I gather, the issue isn't so much that you're missing
the middle/right button areas but rather that you can't left-click (without
pointer movement) below those buttons - that's really what you need, right?

If so, then extending the timeout to include most of the northern part of the touchpad for palm detection for longer should fix that. It's not the same as your configuration (buttons would remain the same size) but you can now click below them to trigger left-clicks.
Comment 8 Markus Schmidt 2017-01-13 14:51:13 UTC
Hey Peter,

(In reply to Peter Hutterer from comment #7)
> Exposing the full button configuration is something I'm very hesitant to
> expose, especially for what is now 3 year old hardware. Plus, options fix it
> on exactly one machine and then spread like an STD across forums, half the
> time without the users knowing what they actually set. So I'm looking for
> something that improves the touchpad for every user with a simple update.

From my understanding as UX/UI dev both (extreme) ways is a can of worms:

* I'm not aware of any slightly complex interaction scheme where "one button fits all" i.e. "one config fits all" works. One might make things usable for lots of people in a "WFM" manner but it always stays behind users expectations or technical capabilities.

* OTOH handing out every configurable value might lead to lots of support but raises the need for a) mature manuals and/or b) a more or less self-explaining user interface.

I guess the paragraph I quoted in the initial report doesn't come out of nothing so it seems you were already facing lots of requests about at least some configuration. I fully understand your reservation against letting users fuck up their input devices by altering some random values in a text file - but.

What about a helpful, easy-to-use, restriction-aware _graphical_ user interface for configuring libinput? I don't mean a couple of meaningless faders or numerical values but something like I did with e.g. xplanetFX ( https://youtu.be/x9SyYLu_M6M ) or WifiRadar ( http://bit.ly/2irXgvY )

I would love to offer my help in this topic and to build a GUI for configuring the users-most-wanted settings of libinput. This way

* total noobs would still be able to use libinput the OSFA-way
* pro users with the need of having a custom configuration could easily tweak some of the most demanded values via a (restricted) GUI without the jeopardy of making their input devices unusable (preventing from lots of support requests) and
* hardcore users would still be able to fuck up their systems intendedly by tweaking a text file

> there are a few options we have here: we could make the buttons bigger while
> the trackpoint is in use and for longer afterwards. This would avoid the
> issue with pointer movement but may still have the left button out of reach
> for you. And from what I gather, the issue isn't so much that you're missing
> the middle/right button areas but rather that you can't left-click (without
> pointer movement) below those buttons - that's really what you need, right?
> 
> If so, then extending the timeout to include most of the northern part of
> the touchpad for palm detection for longer should fix that. It's not the
> same as your configuration (buttons would remain the same size) but you can
> now click below them to trigger left-clicks.

After using libinput for a couple of weeks now I still find myself e.g. closing tabs in my browser while I only wanted to scroll down (I'm using firegestures), hammering on my clickpad cause it doesn't react (note to myself: LIFT THE THUMB FIRST, IDIOT!) or opening menus on my desktop instead of switching to the next one. So trying to fiddle with timeouts or other settings might make it a bit more usable to myself but - everyone has different fingers/hands/wrists/touch-/clickpads - so possibly it makes things worse for others. So for my clickpad the button zones don't fit the marked ones on the hardware, and if they would they would still be quite uncomfortable for my work.

So I would like to finish with a possibly controversial statement - what I love *most* on Linux is the fact that normally everything is tweak-able to the users needs. You don't like something? Open a terminal and change the configuration. I really don't understand e.g. the Gnome3 way - strip off every single button/menu entry/config to force the user to use the software *their* way. I love to have (hidden if you like) choices.

Thanks for still taking care of my needs, best Markus
Comment 9 Peter Hutterer 2017-01-16 06:45:34 UTC
(In reply to Markus Schmidt from comment #8)
> From my understanding as UX/UI dev both (extreme) ways is a can of worms:

[...]
yeah, I agree. Right now, I quick grep counts 16 different options already, not counting the ones we adjust based on the device (i.e. where the device has specific behaviour because it wouldn't fit the standard behaviour but we don't expose a config for that).

> What about a helpful, easy-to-use, restriction-aware _graphical_ user
> interface for configuring libinput? I don't mean a couple of meaningless
> faders or numerical values but something like I did with e.g. xplanetFX (
> https://youtu.be/x9SyYLu_M6M ) or WifiRadar ( http://bit.ly/2irXgvY )

both of those are end-user applications. libinput is a library used by the compositor and users don't have direct access to it. having a GUI to libinput would be akin to having a GUI for glibc - it just doesn't make sense this way. the GUI for libinput is the gnome control panel, or the KDE settings panel, or ...

The problem is: unless you go full "yes to anything", you always have some limit of where you draw the line in further configuration options. And each option increases the required test scenarios (and guess who has to write those tests :) but right now we're just arguing about where the line is drawn and that's just a subjective argument, as maintainer I just have to say no to a lot of people ("yes is forever...")

> * pro users with the need of having a custom configuration could easily
> tweak some of the most demanded values via a (restricted) GUI without the
> jeopardy of making their input devices unusable (preventing from lots of
> support requests) and
> * hardcore users would still be able to fuck up their systems intendedly by
> tweaking a text file

problem is: you can't separate things this way with libinput, as said above the configuration is controlled by the compositor (or the xorg driver)

> So I would like to finish with a possibly controversial statement - what I
> love *most* on Linux is the fact that normally everything is tweak-able to
> the users needs. You don't like something? Open a terminal and change the
> configuration. I really don't understand e.g. the Gnome3 way - strip off
> every single button/menu entry/config to force the user to use the software
> *their* way. I love to have (hidden if you like) choices.

simple answer: more options == less testing == less stable code == more time spent bugfixing after releases
other answer: more options == more effort to maintain == less time for actual development

In your case: this touchpad was present in a single series from early 2014 (with a refresh in Oct 2014) and got replaced because of user complaints in the 2015 models. No device since had this touchpad and no device will have it. So my motivation to put a new option in that I'll have to support forever for a device that's 3 years old and won't come back is... limited ;)

I'll fix the bug in that you can't click unless you lift the finger, once that one's fixed we can look at what else is needed. Meanwhile: let's not have more philosophical discussions on options here, you can move that to the wayland-devel list if you want. Here it's just me replying to it, which could get boring after a while.
Comment 10 Markus Schmidt 2017-01-16 09:52:25 UTC
Hey Peter,

thanks a lot for your explanations.

(In reply to Peter Hutterer from comment #9)
> both of those are end-user applications. libinput is a library used by the
> compositor and users don't have direct access to it. having a GUI to
> libinput would be akin to having a GUI for glibc - it just doesn't make
> sense this way. the GUI for libinput is the gnome control panel, or the KDE
> settings panel, or ...

Probably you missed my point. You say a GUI for libinput is senseless just to point to the different DE's control panels as GUI afterwards. So my point was that even if there were some faders to e.g. set the touchpads sensitivity (in whichever environment) in the past it always was far from offering a UI for controlling input devices behavior. the different Xorg drivers have lots of settings and even if there have been a couple of projects trying do provide some deeper control it always was just a tiny fraction of the possibilities. So the idea was to have a graphical interface developed in direct cooperation with the libs dev to reflect the whole power/settings of the driver without using meaningless numerical values as an interface while preventing the user to shoot oneself in the foot by using their text based config files as a playground.

> The problem is: unless you go full "yes to anything", you always have some
> limit of where you draw the line in further configuration options. And each
> option increases the required test scenarios (and guess who has to write
> those tests :) but right now we're just arguing about where the line is
> drawn and that's just a subjective argument, as maintainer I just have to
> say no to a lot of people ("yes is forever...")

I fully understand your point.

> problem is: you can't separate things this way with libinput, as said above
> the configuration is controlled by the compositor (or the xorg driver)

libinput already separates things - there's the plug-and-play-like behavior ("wow, it just ... works!") and there are some configuration values for deeper control or special use cases.

For Xorg the configuration is just a simple text file or some console-driven control interface like synclient which suits the pro- to hardcore users. I'm not familiar with the way wayland handles things.

From my POV a UI shouldn't care about what it controls or manipulates in the end - a config file, an API, a command line tool or whatever. xplanetFX for example generates different config files for xplanet and is doing some graphical manipulation on the renderings via imagemagick afterwards. If I would switch over to GIMP for the graphical part the UI of the program simply wouldn't care since the interface between UI and back-end is just a config file.

> simple answer: more options == less testing == less stable code == more time
> spent bugfixing after releases
> other answer: more options == more effort to maintain == less time for
> actual development

Exactly.

> In your case: this touchpad was present in a single series from early 2014
> (with a refresh in Oct 2014) and got replaced because of user complaints in
> the 2015 models. No device since had this touchpad and no device will have
> it. So my motivation to put a new option in that I'll have to support
> forever for a device that's 3 years old and won't come back is... limited ;)

There are lots of external clickpad-like devices available, Macbooks have this kind of control, too. But as I said I completely understand your point. So to me it seems that a single person obviously is not enough to bring full support and control to all input devices. If you manage to bring some "one-size-fits-all" solution which makes most of the users happy most of the time it still is a master stroke.

> I'll fix the bug in that you can't click unless you lift the finger, once
> that one's fixed we can look at what else is needed.

Great, thanks a lot! I will love to test it and if it still doesn't fit my needs I have to switch back to the self-compiled evdev/synaptics cocktail in hope that I make it work again.

> Meanwhile: let's not
> have more philosophical discussions on options here, you can move that to
> the wayland-devel list if you want. Here it's just me replying to it, which
> could get boring after a while.

No, I don't want to steal anyones time discussing my personal opinions, better invested in coding the world a better place .) Since you are the dev you decide what to build into your software and what not. If I disagree I'm free to switch to another software or to build my own - that's what "freedom" in FOSS really means. So no need to go on with this discussion at all.

Thanks a lot for your time and your efforts,

best Markus
Comment 11 Peter Hutterer 2017-01-17 06:12:13 UTC
very simple diff to test please:

diff --git a/src/evdev-mt-touchpad-buttons.c b/src/evdev-mt-touchpad-buttons.c
index f4fe6b7..aaffb7b 100644
--- a/src/evdev-mt-touchpad-buttons.c
+++ b/src/evdev-mt-touchpad-buttons.c
@@ -301,7 +301,7 @@ tp_button_top_handle_event(struct tp_dispatch *tp,
                                            event);
                break;
        case BUTTON_EVENT_IN_AREA:
-               tp_button_set_state(tp, t, BUTTON_STATE_TOP_TO_IGNORE, event);
+               tp_button_set_state(tp, t, BUTTON_STATE_AREA, event);
                break;
        case BUTTON_EVENT_UP:
                tp_button_set_state(tp, t, BUTTON_STATE_NONE, event);

that should detect any physical button presses as button left now (if outside the top area) I'll prepare the proper patch if you're reasonably happy with this.
Comment 12 Markus Schmidt 2017-01-17 17:02:40 UTC
hey Peter,

thanks a lot for the patch!

Everything went fine cloning, patching and building libinput. Do you have any advice how to install and use it in Xorg on a debian-based system with xserver-xorg-input-libinput installed?

Best Markus
Comment 13 Peter Hutterer 2017-01-17 21:16:59 UTC
for a patch this simple, you can just overwrite the system copy and restart X/Wayland. Alternatively, you could install it into a custom prefix and change the /usr/lib/libinput.so* symlinks to the new version. Or build a new debian package with the patch included and install that, that way downgrading is easiest.

I'd go with the simple make install one for this patch here, afterwards you can force-reinstall the debian package to overwrite the files with the package ones again.
https://wayland.freedesktop.org/libinput/doc/latest/building_libinput.html
Comment 14 Peter Hutterer 2017-01-25 02:18:34 UTC
ping?
Comment 15 Markus Schmidt 2017-01-25 13:25:41 UTC
Hey Peter,

sorry for not reporting back earlier. I tested the patch but the behavior is still awkward somehow. Additionally - as I said - my fingers don't get used to the pre-defined button areas regarding e.g. the border between middle and right button, I still often find myself closing tabs, switching desktops and more without intend. Conclusion: I really need to configure my button areas manually.

So I'm sorry for the noise but it seems libinput doesn't fit my needs. However, keep up your great work for FOSS community,

best Markus
Comment 16 Peter Hutterer 2017-01-26 05:29:58 UTC
wait, the border between middle and right button should be exactly where you painted it, the only change the patch does is making it possible to click anywhere for a left button click.
Comment 17 Markus Schmidt 2017-01-27 01:01:37 UTC
Hey Peter,

the image is a drawing. The actual border between middle and right button is nearby but not exactly at that position. It took some time to fiddle with the button area settings to make it fit my fingers exactly.

Regarding the patch - my thumb rests on middle button position normally. As soon as I move it down into the touchpad area (while not lifting it, so in fact it is sliding over the surface) and do a click there it is still recognized as middle mouse click.

Best Markus
Comment 18 Peter Hutterer 2017-01-30 00:08:59 UTC
(In reply to Markus Schmidt from comment #17)
> the image is a drawing. The actual border between middle and right button is
> nearby but not exactly at that position. It took some time to fiddle with
> the button area settings to make it fit my fingers exactly.

indeed, verified. code triggers at 42mm and 58mm but both are well within the middle button range. Feel free to submit a patch to change this to 40/60mm instead, that seems like a more accurate range. This may have recently changed, I only submitted the resolution corrections to systemd quite late.

> Regarding the patch - my thumb rests on middle button position normally. As
> soon as I move it down into the touchpad area (while not lifting it, so in
> fact it is sliding over the surface) and do a click there it is still
> recognized as middle mouse click.

right, the patch *should* eventually convert this into a left button click but you'll have to move 10mm down from the top of the pad, so about 5mm below the red markings. Previous recordings when the support was written showed that this size is needed to capture the top button presses.
Comment 19 Peter Hutterer 2017-02-15 06:56:33 UTC
commit cbe9a3bfc3c70992dc08934973a378da4f5314e5
Author: Peter Hutterer <peter.hutterer@who-t.net>
Date:   Wed Feb 8 09:53:44 2017 +1000

    touchpad: expand top middle button to cover 40mm to 60mm
Comment 20 Peter Hutterer 2018-01-09 03:52:16 UTC
simply said: I'm closing this. The middle button was expanded ~1 year ago, I'm not going to merge configurable button areas for the 540, it's 4 years old by now, sorry.


Use of freedesktop.org services, including Bugzilla, is subject to our Code of Conduct. How we collect and use information is described in our Privacy Policy.