Bug 86110

To ensure the continued backwards-compatibility of the Wayland protocol and libwayland-server,client ABI, we should have some automated testing in place. I very much doubt any developer ever tests mixed versions of different components in purpose.

The components in play:
- Wayland (libwayland-server, libwayland-client, protocols)
- Weston
- an autonomous test app to be written, that excercises as much as possible given the versions available

We would like to test building Weston and app against one version of wayland, and run them against another version. This produces a test matrix on Wayland build version vs. Wayland runtime version.

Only official release tags are the tested versions. Alpha and beta releases should be included if resources allow. In addition, the master version of Wayland and Weston should be included, so we can catch breakage before releases. The rolling tests could run, say, weekly or more often if resources allow.

Weston very often requires the contemporary version of wayland, which is fine, and produces expected build failures. These should be found by testing rather than pre-encoding the assumption. Once a test is confirmed to legitimately fail, it can be marked as known-to-not-work.

We need some way to limit the combinatorial explosion, because both Weston and the app maybe built or ran against different versions of wayland. It is also useless to re-run tests with the exact same setup as before (unless the test app is updated).

The essential idea is, that we would get an automated test bot, that ensures that we do not break the backwards-compatibility of wayland.

There exists
http://upstream.rosalinux.ru/versions/wayland.html
but I think doesn't cover the Wayland-specifics, like different code generation between versions of wayland and how generated code gets built into compositor and app binaries.