Bug 82742 - Touch doesn't work after calibration
Summary: Touch doesn't work after calibration
Status: RESOLVED FIXED
Alias: None
Product: Wayland
Classification: Unclassified
Component: weston (show other bugs)
Version: 1.5.0
Hardware: Other Linux (All)
: medium critical
Assignee: Wayland bug list
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2014-08-18 03:45 UTC by fcooper
Modified: 2014-12-09 02:22 UTC (History)
3 users (show)

See Also:
i915 platform:
i915 features:


Attachments
Touch test using Weston patch and libinput work in progress tree. (1.46 MB, text/plain)
2014-09-02 15:05 UTC, fcooper
Details
Touch test using Weston patch and libinput work in progress tree. Before calibration (1.46 MB, video/quicktime)
2014-09-02 15:06 UTC, fcooper
Details
Touch test using Weston patch and libinput work in progress tree. After calibration (1.79 MB, video/quicktime)
2014-09-02 15:07 UTC, fcooper
Details

Description fcooper 2014-08-18 03:45:00 UTC
Environment:
Embedded Linux Device
Resistive Touch Screen
Latest Weston/Wayland and Libinput

Summary:

Calibrated values are based on pixels and are used to compensate for differences in touch screens.

Libinput receives raw touch values. On an resistive touchscreen these are based on ADC values with a range (ex 0 - 4095). Libinput then uses the calibrated values to adjust the raw touch values. These adjusted values are then passed to Weston which eventually converts the adjusted values to actually screen coordinates/pixels. However, when calculating the screen coordinates these adjusted values results in very large values (greater the screen dimension) or negative values to be returned. When Weston sends these coordinates values to the rest of the system no interaction with views can take place since the incorrect touch coordinates are beyond the screen thus no interaction can ever take place.

So before any raw values are adjusted via calibration they should first be converted to screen coordinates / pixels. This is how Weston without libinput handles things and touch works fine.

More details can be found in this mailing list thread:
http://lists.freedesktop.org/archives/wayland-devel/2014-August/016565.html
Comment 1 Peter Hutterer 2014-08-21 02:48:23 UTC
So, I read through this multiple times (and the mailing list post) and wrote a few tests for libinput and there are a couple of comments I have:

libinput's calibration for multi-touch events didn't exist. I've got patches pending for that. Not sure if that affects your device though, because for those the matrix currently has no effect whatsoever.

But I'm wondering if you're using the calibration matrix as intended. What values do you have for the matrix? The matrix is designed to calibrate the device within it's own range so that the 0 value corresponds to what should be the device's 0 value on the screen (not in pixels, in device units). e.g. if a sensor is bigger than the screen, the 0 value may be offset by a bit.

Looking at weston's code it seems to do the right thing. In the non-transformed non-scaled case, x/y simply become x + screen->x and y + screen->y. That seems correct.
Comment 2 fcooper 2014-08-21 02:48:27 UTC
I will be out of the office Aug 20th and I will be back in the office Aug 21st.
Comment 3 fcooper 2014-08-21 13:41:07 UTC
Hi Peter,

The device I am currently working on is a single touch resistive touch device. Also the way I am calibrating the device (using the calibration matrix) is by running the weston-calibrator and after touching the screen it spits out the calibration values. I then take those files and place them in a udev rule:
ATTRS{name}=="ti-tsc", ENV{WL_CALIBRATION}="<calibration values>"

An example of the calibration values I received was
-1.058590 0.047792 808.858215 -0.022940 -1.082486 495.361237

Since using this same udev rule when using Weston without libinput works I'm not sure where I could be incorrectly using this calibration matrix values.
Comment 4 Peter Hutterer 2014-08-22 06:06:01 UTC
(In reply to comment #3)
> An example of the calibration values I received was
> -1.058590 0.047792 808.858215 -0.022940 -1.082486 495.361237

tbh, that matrix looks odd. It has a translation component of 808/495 pixels.
But otherwise, yes, I now see the problem.

Weston takes the device coordinates, scales them to screen coordinates (without the offset), then applies the matrix, then the screen offset/scale (if any). So the calibration matrix is expected to be in pixels.
When I read the source first I didn't see the device-coord-to-screen scaling which happens right when the event comes in.

libinput takes the device coordinates, applies the matrix, then scales to screen coordinates, then applies the screen offset/scale (if any). So the calibration matrix is expected to be in device coordinates.

We should be able to apply the matrix during _get_x_transformed() after scaling to screen coordinates. That way we can keep having the matrix in pixels. Which is not ideal, but doesn't break with the transition to libinput as default backend. Jonas, any comments?

I'm still concerned about the translation component though, that just doesn't seem right.
Comment 5 Peter Hutterer 2014-08-25 08:11:10 UTC
ok, now that I've had a bit more sleep, that matrix is ok after all. Ignoring the decimal points, it is probably
[ -1  0 800 ]
[  0 -1 480 ]
[  0  0  1  ]
which is the matrix for a 180 degree rotation on a 800x480 display. So I guess that matches your device, the rest is just measuring error.

The main problem is that the translation component of the matrix is expected to be in pixels. Which libinput doesn't really have access to, or at least not when needed. So we'd have to break libinput API or break how WL_CALIBRATION works, neither option is nice.
Comment 6 Peter Hutterer 2014-08-27 07:38:21 UTC
Do me a favour and test the branch here please: https://github.com/whot/libinput/tree/wip/calibration

And the weston patch here: http://lists.freedesktop.org/archives/wayland-devel/2014-August/016884.html

Those two together should restore calibration functionality with the original settings. Let me know how you go please.
Comment 7 fcooper 2014-08-28 12:06:39 UTC
(In reply to comment #6)
> Do me a favour and test the branch here please:
> https://github.com/whot/libinput/tree/wip/calibration
> 
> And the weston patch here:
> http://lists.freedesktop.org/archives/wayland-devel/2014-August/016884.html
> 
> Those two together should restore calibration functionality with the
> original settings. Let me know how you go please.

Hi Peter,

I gave this a shot and unfortunately I couldn't get touch to work at all after calibrating the screen.

Based on your patches I believe this is the proper udev entry to use for my device

ATTRS{name}=="ti-tsc", ENV{LIBINPUT_CALIBRATION_MATRIX}="-1.026360 -0.042350 814.961914 0.016442 -1.164370 496.070984"

I got the values for LIBINPUT_CALIBRATION_MATRIX by first booting up the board with no calibration, running weston-calibrator and then creating the above udev rule.

I'm not familiar with this calibration matrix calculation. Is there a set of values you can provide me that will end up not actually alterating the touch coordinates at all (leave the screen uncalibrated). But atleast we can see if maybe the calibrator app would need some tweaks or if the issue is else where.
I blindly tried using values of all 0's and all 1's but I didn't see any difference.
Comment 8 Peter Hutterer 2014-08-28 20:56:28 UTC
oh, sorry, I wasn't clear enough here. The goal of this patchset is to avoid breakage, so the original WL_CALIBRATION rule you posted in comment 3 should work as-is.
Comment 9 fcooper 2014-08-28 21:48:19 UTC
(In reply to comment #8)
> oh, sorry, I wasn't clear enough here. The goal of this patchset is to avoid
> breakage, so the original WL_CALIBRATION rule you posted in comment 3 should
> work as-is.

Hi Peter,

Ok not a problem. I'll give it another shot. So I'll use ATTRS{name}=="ti-tsc", ENV{WL_CALIBRATION}="<calibration values>" in the udev rule again.

When I tried your patches I initially still used WL_CALIBRATION instead of LIBINPUT_CALIBRATION_MATRIX. However, the calibration still looked off to the point that I believe that it wasn't accepting the calibration value at all. Was that what you were trying to verify? I can double check again since I quickly switched to LIBINPUT_CALIBRATION_MATRIX when I saw the above behavior and looked at some of your patches.
Comment 10 Peter Hutterer 2014-08-28 23:14:11 UTC
There are two separate changes: libinput (LIBINPUT_CALIBRATION_MATRIX) uses a normalized matrix. So the translation component is in device widths/heights. In your case that means the 814 and 496 values both become 1 (or close to 1).

The weston-patch (make sure you applied that one too!) converts the WL_CALIBRATION matrix into that normalized form.

So for your setup: if you apply the libinput patches and the weston patches, your original matrix should work as it did before. If it doesn't, then the patches are incomplete.

If it doesn't work, try setting the LIBINPUT_CALIBRATION_MATRIX matrix to the normalized values and let me know if at least that works fine so we can narrow down where the issues are. From your matrix, I suspect your device has the touchscreen mounted upside-down, right? So the LIBINPUT_CALIBRATION_MATRIX would be "-1 0 1 0 -1 1". That's the basic rotation matrix, it doesn't account for any offsets otherwise.

But anyway, the goal is to have your original WL_CALIBRATION matrix work as-is.
Comment 11 fcooper 2014-09-02 15:04:38 UTC
Hi Peter,

Sorry for the delay. I applied the Weston patch you provided and used your work in progress tree for libinput and it seems that something in the calibration or normalization calculation is still off.

Before, I added the calibration information I am getting the expected result.

However, once I calibrate the screen and add it to the udev rule the calibration is still off. Most noticeably on the y axis since I can not access the bottom half of the screen as if the bottom of the screen is now the middle of the screen. 

I am attaching two videos. One before calibration and another one after calibration. Looking closely at the after calibration video it looks like the calibration on the x axis is correct. Only the y axis is off by 240 (half the screen).
Comment 12 fcooper 2014-09-02 15:05:35 UTC
Created attachment 105614 [details]
Touch test using Weston patch and libinput work in progress tree.
Comment 13 fcooper 2014-09-02 15:06:30 UTC
Created attachment 105615 [details]
Touch test using Weston patch and libinput work in progress tree. Before calibration
Comment 14 fcooper 2014-09-02 15:07:22 UTC
Created attachment 105616 [details]
Touch test using Weston patch and libinput work in progress tree. After calibration
Comment 15 Peter Hutterer 2014-09-03 06:15:52 UTC
Can you post what you have in udev now (probably close to the original values anyway) and what weston says? There should be a "Applying calibration.... " message on startup and that's what weston feeds to libinput.
Comment 16 fcooper 2014-09-03 18:31:45 UTC
Hi Peter,

Current udev rule

ATTRS{name}=="ti-tsc", ENV{WL_CALIBRATION}="-1.010854 -0.100470 826.476990 -0.019684 -0.975587 492.199097"

Output from Weston:
Applying calibration: -1.010854 -0.100470 826.476990 -0.019684 -0.975587 492.199097 (normalized 1.033096 0.615249)
Comment 17 Peter Hutterer 2014-09-03 21:27:51 UTC
Yikes, found the bug. In the weston patch, replace this line:

+       height = device->output->width;

with height = device->output->height; Just a few lines from the beginning of evdev_device_set_calibration. Then let me know how you go. That should give a normalized calibration of roughly 1 and 1
Comment 18 fcooper 2014-09-05 01:01:24 UTC
After changing that line calibration is working perfectly fine. I tested this on my resistive touch screen which originally showed this problem. I will double check things on a cap touch screen but so far everything is working great.

Thank you
Comment 19 Peter Hutterer 2014-09-05 01:26:22 UTC
Updated patch here:
http://lists.freedesktop.org/archives/wayland-devel/2014-September/017103.html

Thanks for the tests!
Comment 20 Peter Hutterer 2014-09-05 01:35:15 UTC
Reassigning this bug to weston. The libinput changes are on git master, so now it's just weston that needs to handle the remaining bit as outlined in the patch linked above.
Comment 21 Pekka Paalanen 2014-09-12 09:53:35 UTC
Committed to Weston as:

commit 3fbba493d5732f9feb5a7956cb98d6eb5f0ff363
Author: Peter Hutterer <peter.hutterer@who-t.net>
Date:   Tue Sep 9 13:02:25 2014 +1000

    libinput: normalize WL_CALIBRATION before passing it to libinput

And is part of Weston 1.5.93.

I assume this is fixed, then.
Comment 22 Denys Dmytriyenko 2014-12-09 02:22:01 UTC
Sorry to revive this old bug, but the videos Franklin posted perfectly illustrate the other issue I'd like to inquire about...

As seen on the videos, weston-simple-touch is limited to a smaller area on the screen, specifically there are "dead" borders on the left and right of the screen, where no touch events are being recorded. It seems like weston-calibrator only calibrates the middle portion of the screen (about 640x480), while the screen is actually wide and has resolution of 800x480.

Is there anything to correct this behavior? I'm using libinput 0.6.0 and wayland
/weston 1.6.0 on fbdev-backend. Thanks.


Use of freedesktop.org services, including Bugzilla, is subject to our Code of Conduct. How we collect and use information is described in our Privacy Policy.