Bug 74626

Summary: Telepathy projects should have GNOME-style installed tests
Product: Telepathy Reporter: Simon McVittie <smcv>
Component: generalAssignee: Simon McVittie <smcv>
Status: RESOLVED FIXED QA Contact: Telepathy bugs list <telepathy-bugs>
Severity: normal    
Priority: medium CC: xclaesse
Version: git master   
Hardware: Other   
OS: All   
Whiteboard:
i915 platform: i915 features:
Bug Depends on: 76688, 76690    
Bug Blocks:    
Attachments: [MC] run-test.sh: output in TAP format, and install an installed-tests wrapper
All C/C++ tests: run with temporary XDG_*_HOME, XDG_RUNTIME_DIR
All C/C++ tests: run with temporary XDG_*_HOME, XDG_RUNTIME_DIR

Description Simon McVittie 2014-02-06 16:17:24 UTC
It would be great if the various Telepathy projects were easy to plug into CI frameworks.

Various GNOME people are promoting <https://wiki.gnome.org/action/show/Initiatives/GnomeGoals/InstalledTests>, which seems as good a way to do this as any.

It would also be nice to hook this into Debian autopkgtest (<http://anonscm.debian.org/gitweb/?p=autopkgtest/autopkgtest.git;a=blob_plain;f=doc/README.package-tests;hb=HEAD>), but it should be easy enough to do that via gnome-desktop-testing-runner, in the same way GLib (Debian: glib2.0) does.
Comment 1 Simon McVittie 2014-02-06 16:18:41 UTC
Created attachment 93542 [details] [review]
[MC] run-test.sh: output in TAP format, and install an  installed-tests wrapper

See:
https://wiki.gnome.org/action/show/Initiatives/GnomeGoals/InstalledTests
https://en.wikipedia.org/wiki/Test_Anything_Protocol

---

As a starting point, here's Mission Control.
Comment 2 Guillaume Desmottes 2014-02-14 14:07:06 UTC
Comment on attachment 93542 [details] [review]
[MC] run-test.sh: output in TAP format, and install an  installed-tests wrapper

Review of attachment 93542 [details] [review]:
-----------------------------------------------------------------

++
Comment 3 Simon McVittie 2014-03-20 12:38:38 UTC
*** Bug 27862 has been marked as a duplicate of this bug. ***
Comment 4 Simon McVittie 2014-03-20 12:39:09 UTC
Comment on attachment 93542 [details] [review]
[MC] run-test.sh: output in TAP format, and install an  installed-tests wrapper

applied in master and next
Comment 5 Simon McVittie 2014-03-20 12:40:34 UTC
telepathy-glib installed tests (also GTest everywhere, as Bug #27862 asked):

http://cgit.freedesktop.org/~smcv/telepathy-glib/log?h=next-tap
Comment 6 Simon McVittie 2014-03-20 12:42:35 UTC
(In reply to comment #5)
> telepathy-glib installed tests

Still to do: the Logger tests aren't installable. They'll need some alterations for that, I suspect.
Comment 7 Simon McVittie 2014-03-20 17:53:29 UTC
MC installed-tests improvements (for master, but should also work for next): http://cgit.freedesktop.org/~smcv/telepathy-mission-control/log/?h=tests
Comment 8 Xavier Claessens 2014-03-22 19:04:06 UTC
 - "run-test.sh: run in a temporary XDG_DATA_HOME etc.": We don't do that when running "make check", if you need it for installed tests we probably need it for make check as well, no? Note that tests/dbus/contacts.c at least already does it from C code. I should push again for my helpers for setting those in glib: https://bugzilla.gnome.org/show_bug.cgi?id=674197.

 - "util-cxx.cpp: convert to GTest": please squash it with the other commit before merging.
Comment 9 Simon McVittie 2014-03-24 12:04:24 UTC
(In reply to comment #8)
>  - "run-test.sh: run in a temporary XDG_DATA_HOME etc.": We don't do that
> when running "make check", if you need it for installed tests we probably
> need it for make check as well, no?

Maybe. I'll check.

>  - "util-cxx.cpp: convert to GTest": please squash it with the other commit
> before merging.

Sure.
Comment 10 Simon McVittie 2014-03-24 13:01:30 UTC
tests/Makefile.am: no special variables

tests/dbus/Makefile.am: XDG_DATA_HOME, XDG_DATA_DIRS

tests/logger/Makefile.am: XDG_DATA_HOME

tests/logger/dbus/Makefile.am: XDG_DATA_HOME, XDG_DATA_DIRS

... so yes, these places should ideally all disarm XDG_CONFIG_HOME, XDG_CACHE_HOME and XDG_RUNTIME_DIR too.
Comment 11 Simon McVittie 2014-03-24 13:03:06 UTC
(In reply to comment #8)
> if you need it for installed tests we probably
> need it for make check as well, no?

I don't think we strictly *need* to disarm XDG_CACHE_HOME, XDG_CONFIG_HOME and XDG_RUNTIME_DIR for the installed-tests right now, but it seems good to isolate them more thoroughly in case libraries we depend on make more use of those variables.
Comment 12 Simon McVittie 2014-03-24 13:03:52 UTC
Created attachment 96288 [details] [review]
All C/C++ tests: run with temporary XDG_*_HOME,  XDG_RUNTIME_DIR

These aren't currently used, so we don't actually create them.

---

In addition to what was previously in the branch.
Comment 13 Simon McVittie 2014-03-24 17:57:55 UTC
Created attachment 96306 [details] [review]
All C/C++ tests: run with temporary XDG_*_HOME,  XDG_RUNTIME_DIR

These aren't currently used, so we don't actually create them...
except for tests/logger/dbus/tmp-cache/telepathy/logger/sqlite-data,
which is in fact created by running the tests.

---

Previous patch broke distcheck.
Comment 14 Xavier Claessens 2014-03-24 18:42:37 UTC
+1.

I still prefer doing it from the C code so we can safely just run ./foo without needing to set env, but we are not there yet.
Comment 15 Simon McVittie 2014-03-25 12:21:47 UTC
Comment on attachment 96306 [details] [review]
All C/C++ tests: run with temporary XDG_*_HOME,  XDG_RUNTIME_DIR

Merged for 0.99.9
Comment 16 Simon McVittie 2014-03-26 09:00:15 UTC
> MC installed-tests improvements (for master, but should also work for next)

Actually, never mind master, let's just do this on next. (Branch is relative to next-import which is Bug #70991.)

http://cgit.freedesktop.org/~smcv/telepathy-mission-control/log?h=next-tests&cache=please-dont
Comment 17 Simon McVittie 2014-04-02 18:07:58 UTC
Collecting the branches for this so far, including the one for Idle that I've just done:

http://cgit.freedesktop.org/~smcv/telepathy-mission-control/log?h=next-tests
http://cgit.freedesktop.org/~smcv/telepathy-gabble/log?h=next-tests
http://cgit.freedesktop.org/~smcv/telepathy-idle/log?h=next-tests

Salut, Rakia and Haze to go. I expect them to look a lot like Idle.
Comment 18 Simon McVittie 2014-05-07 10:37:09 UTC
(In reply to comment #17)
> Salut, Rakia and Haze to go. I expect them to look a lot like Idle.

Gabble, MC, Idle and Rakia merged in or before 0.99.11. Just Salut and Haze to go.
Comment 19 Simon McVittie 2014-05-12 10:57:19 UTC
Salut and Haze also merged. Fixed in git for 0.99.11.

Use of freedesktop.org services, including Bugzilla, is subject to our Code of Conduct. How we collect and use information is described in our Privacy Policy.