Bug 65290

Summary: tests will fail with Automake 1.13
Product: Telepathy Reporter: Simon McVittie <smcv>
Component: gabbleAssignee: Simon McVittie <smcv>
Status: RESOLVED MOVED QA Contact: Telepathy bugs list <telepathy-bugs>
Severity: major    
Priority: high Keywords: patch
Version: unspecified   
Hardware: Other   
OS: All   
Whiteboard:
i915 platform: i915 features:
Bug Depends on: 65131    
Bug Blocks:    
Attachments: Update Wocky to fix a regression caused by #65131
Officially depend on GLib 2.32
.gitignore: automake 1.13 installs test-driver
Adapt run-test.sh to be able to run the tests uninstalled
Use run-test.sh to run uninstalled tests
Remove obsolete GLib < 2.32 code paths
run-test.sh: keep a single Gabble and dbus-daemon during "make check"
[Rakia] Adapt tests to be compatible with Automake 1.13
[Rakia] Adapt tests to be compatible with Automake 1.13
[Haze] Avoid recent libpurple doing "clever" proxy discovery, etc.
[Haze] Adapt tests to be compatible with Automake 1.13
[Gabble] run-test.sh.in: make the indentation make sense

Description Simon McVittie 2013-06-03 10:26:48 UTC
Automake 1.12 introduces a new parallel "test driver" and Automake 1.13 makes it the default.

Unfortunately, this is a compatibility break for projects like telepathy-gabble, telepathy-mission-control, telepathy-haze and telepathy-salut that relied on being able to do obscure tricks with the check-TESTS target and TESTS_ENVIRONMENT. We can't even switch back to the old test driver without a dependency on Automake 1.12.

For telepathy-gabble, the quickest way to get tests passing again seems to be to adapt run-tests.sh so it can be used to run installed tests as well as uninstalled ones, and use that instead of playing with check-TESTS.

Later, we could consider providing our own "test driver". I think we would still have to run the tests serially, because they all listen on the same TCP port, unless we can adapt the framework to have them listen on a dynamic port (and somehow avoid the port used by the test that is meant to connect to a wrong port and fail).
Comment 1 Simon McVittie 2013-06-03 10:39:02 UTC
Created attachment 80200 [details] [review]
Update Wocky to fix a regression caused by #65131

---

Yes I realise this can't really be reviewed... it's mainly a reminder that a couple of patches from Bug #65131 need review first.
Comment 2 Simon McVittie 2013-06-03 10:39:31 UTC
Created attachment 80201 [details] [review]
Officially depend on GLib 2.32

In practice we depend on it anyway, via telepathy-glib 0.19.9.
Also update the telepathy-glib dependency in the .pc files to match
configure.ac.
Comment 3 Simon McVittie 2013-06-03 11:20:02 UTC
Created attachment 80209 [details] [review]
.gitignore: automake 1.13 installs test-driver
Comment 4 Simon McVittie 2013-06-03 11:22:57 UTC
Created attachment 80211 [details] [review]
Adapt run-test.sh to be able to run the tests  uninstalled

When autoreconfiscated with Automake 1.13, the way in which we were
(ab?)using Automake's test driver no longer works. We can't just
switch back to the old serial test driver without a dependency on
at least Automake 1.12, either. However, run-test.sh (which was already
used for installed tests) is quite capable of running uninstalled
tests, with a bit of adjustment:

* if GABBLE_TEST_UNINSTALLED is set, expect GABBLE_ABS_TOP_SRCDIR,
  GABBLE_ABS_TOP_BUILDDIR and optionally GABBLE_TEST_SLEEP in the
  environment
* look for installed or uninstalled files, as appropriate
* use TEST_PYTHON (which might differ from PYTHON)
* use python -u (unbuffered stdout) for better debugging
Comment 5 Simon McVittie 2013-06-03 11:23:12 UTC
Created attachment 80212 [details] [review]
Use run-test.sh to run uninstalled tests
Comment 6 Simon McVittie 2013-06-03 11:56:53 UTC
Created attachment 80221 [details] [review]
Remove obsolete GLib < 2.32 code paths

We now depend on 2.32.

---

Follow-up for Attachment #80201 [details].
Comment 7 Simon McVittie 2013-06-03 13:59:31 UTC
Created attachment 80227 [details] [review]
run-test.sh: keep a single Gabble and dbus-daemon during  "make check"

This cuts "make check" from 5:22 down to 4:58 on my X220.
Comment 8 Simon McVittie 2013-06-03 14:25:40 UTC
Created attachment 80232 [details] [review]
[Rakia] Adapt tests to be compatible with Automake 1.13

Instead of (ab)using the check-TESTS target and the TESTS_ENVIRONMENT
variable, we now run the tests from a shell script similar to the one
in telepathy-gabble.
Comment 9 Simon McVittie 2013-06-04 11:24:46 UTC
Created attachment 80283 [details] [review]
[Rakia] Adapt tests to be compatible with Automake 1.13

---

My previous Rakia patch put run-tests.sh in TESTS, but that means you don't get any feedback when the individual tests are running, which isn't ideal.

One day we should implement an Automake 1.13 "test driver" but for now let's just get the tests passing again.
Comment 10 Simon McVittie 2013-06-04 11:25:26 UTC
Created attachment 80284 [details] [review]
[Haze] Avoid recent libpurple doing "clever" proxy discovery,  etc.

The regression tests are not a full desktop session, and don't have
GSettings or other GNOME features. They also don't have KDE features,
which libpurple also checks.
Comment 11 Simon McVittie 2013-06-04 11:25:49 UTC
Created attachment 80285 [details] [review]
[Haze] Adapt tests to be compatible with Automake 1.13

Instead of (ab)using the check-TESTS target and the TESTS_ENVIRONMENT
variable, we now run the tests from a shell script similar to the one
in telepathy-gabble.
Comment 12 Xavier Claessens 2013-06-11 09:20:15 UTC
Comment on attachment 80201 [details] [review]
Officially depend on GLib 2.32

Review of attachment 80201 [details] [review]:
-----------------------------------------------------------------

ok
Comment 13 Xavier Claessens 2013-06-11 09:20:33 UTC
Comment on attachment 80209 [details] [review]
.gitignore: automake 1.13 installs test-driver

Review of attachment 80209 [details] [review]:
-----------------------------------------------------------------

ok
Comment 14 Xavier Claessens 2013-06-11 09:22:15 UTC
Comment on attachment 80211 [details] [review]
Adapt run-test.sh to be able to run the tests  uninstalled

Review of attachment 80211 [details] [review]:
-----------------------------------------------------------------

ok
Comment 15 Xavier Claessens 2013-06-11 09:22:55 UTC
Comment on attachment 80212 [details] [review]
Use run-test.sh to run uninstalled tests

Review of attachment 80212 [details] [review]:
-----------------------------------------------------------------

ok
Comment 16 Xavier Claessens 2013-06-11 09:23:56 UTC
Comment on attachment 80221 [details] [review]
Remove obsolete GLib < 2.32 code paths

Review of attachment 80221 [details] [review]:
-----------------------------------------------------------------

ok
Comment 17 Xavier Claessens 2013-06-11 09:28:01 UTC
Comment on attachment 80283 [details] [review]
[Rakia] Adapt tests to be compatible with Automake 1.13

Review of attachment 80283 [details] [review]:
-----------------------------------------------------------------

Assuming it is the same than what's in gabble, +1
Comment 18 Xavier Claessens 2013-06-11 09:28:33 UTC
Comment on attachment 80284 [details] [review]
[Haze] Avoid recent libpurple doing "clever" proxy discovery,  etc.

Review of attachment 80284 [details] [review]:
-----------------------------------------------------------------

ok
Comment 19 Xavier Claessens 2013-06-11 09:28:59 UTC
Comment on attachment 80285 [details] [review]
[Haze] Adapt tests to be compatible with Automake 1.13

Review of attachment 80285 [details] [review]:
-----------------------------------------------------------------

Assuming it is the same than gabble, +1
Comment 20 Simon McVittie 2013-06-11 10:06:00 UTC
(In reply to comment #7)
> run-test.sh: keep a single Gabble and dbus-daemon during  "make check"
> 
> This cuts "make check" from 5:22 down to 4:58 on my X220.

< xclaesse> smcv: keep a single Gabble and dbus-daemon during
< xclaesse> "make check" --> why do we want that?
< xclaesse> smcv, that makes tests less isolated, I don't want 
            previous test to be able to influence the next one...
< smcv> xclaesse: it's how the tests work at the moment
< smcv> xclaesse: and saves 20 seconds
< smcv> xclaesse: I was actually surprised how little time it saved. when 
        it's only 20s out of 5min, I'd be OK with dropping that patch - 
        it simplifies things quite a bit actually
< xclaesse> I'm tempted to say that 20s out of 5min is irrelevant, 
            especially since a unit test influancing the next one is 
            potentially days of work to find the problem
< smcv> ok
< smcv> I'll respin the rakia, haze patches to be similar
Comment 21 Simon McVittie 2013-06-11 10:32:40 UTC
Comment on attachment 80201 [details] [review]
Officially depend on GLib 2.32

0.17.6
Comment 22 Simon McVittie 2013-06-11 10:32:53 UTC
Comment on attachment 80209 [details] [review]
.gitignore: automake 1.13 installs test-driver

0.17.6
Comment 23 Simon McVittie 2013-06-11 10:33:06 UTC
Comment on attachment 80211 [details] [review]
Adapt run-test.sh to be able to run the tests  uninstalled

0.17.6
Comment 24 Simon McVittie 2013-06-11 10:33:17 UTC
Comment on attachment 80212 [details] [review]
Use run-test.sh to run uninstalled tests

0.17.6
Comment 25 Simon McVittie 2013-06-11 10:33:30 UTC
Comment on attachment 80221 [details] [review]
Remove obsolete GLib < 2.32 code paths

0.17.6
Comment 26 Simon McVittie 2013-06-11 10:33:46 UTC
Comment on attachment 80227 [details] [review]
run-test.sh: keep a single Gabble and dbus-daemon during  "make check"

rejected
Comment 27 Simon McVittie 2013-06-11 14:00:52 UTC
Created attachment 80689 [details] [review]
[Gabble] run-test.sh.in: make the indentation make sense

The Python invocation is indented, because it's a command-line argument
for the sh invocation. The case shouldn't be.
Comment 28 Simon McVittie 2013-06-25 10:03:10 UTC
(In reply to comment #27)
> [Gabble] run-test.sh.in: make the indentation make sense

Could someone have a quick look at this, please? It's whitespace-only, and I'd like to use Gabble's script as a basis for the other CMs.

If nobody wants to veto it, I'll just apply it.
Comment 29 Simon McVittie 2013-08-07 16:16:32 UTC
Comment on attachment 80689 [details] [review]
[Gabble] run-test.sh.in: make the indentation make sense

Applied based on review from Guillaume
Comment 30 Simon McVittie 2013-08-19 12:58:47 UTC
Comment on attachment 80283 [details] [review]
[Rakia] Adapt tests to be compatible with Automake 1.13

Still not applied, but is obsoleted by Bug #67872
Comment 31 Guillaume Desmottes 2013-12-27 10:33:47 UTC
The two latest patches have been reviewed by Xavier so are good to go I think.
Comment 32 GitLab Migration User 2019-12-03 19:58:50 UTC
-- GitLab Migration Automatic Message --

This bug has been migrated to freedesktop.org's GitLab instance and has been closed from further activity.

You can subscribe and participate further through the new bug through this link to our GitLab instance: https://gitlab.freedesktop.org/telepathy/telepathy-gabble/issues/256.

Use of freedesktop.org services, including Bugzilla, is subject to our Code of Conduct. How we collect and use information is described in our Privacy Policy.