Gabble's test suite now takes 5 minutes to run. This puts at least myself and Guillaume off running the whole test suite; I tend only to run the bit that tests what I'm editing. Occasionally, this means we merge code with failing tests. The bulk of the time is from running all the tests that use bytestreams against 8 different bytestream implementations/scenarios. I suspect we could make them faster, if we looked into why they're so slow.
Ideas: - Eliminate reactor.iterate() call, either by switching to a new API, or using some cunning producer/consumer system. I have a branch working on the former (test-api-3); I tried the latter once, but had problems with serializing events. - Profile with gprof/oprofile. - Run tests in parallel. Each test definitely needs a connection to itself, and maybe a CM to itself (which would mean a bus to itself). I thought maybe that Python startup time (including imports etc.) was a bottleneck, but I wrote a test runner that in theory made all startup costs a one-off cost and it only shaved off ~30s. What's our goal, assuming the current number of tests? One minute? Two?
I did a bit of profiling recently. I shaved a few seconds off by disabling introspection for the ConnectionManager objects constructed by the test suite, and a few more by reducing the debug output by 10%. I think turning off automatic introspection across the board would make a significant difference—cProfile confirmed it, but I've lost the logs—but unfortunately most of the tests depend on it. dbus-python doesn't make it very easy to shove in known types without subclassing dbus.Interface() all over the shop. We can do that but it's not a five-minute thing.
If you don't mind depending on the latest version of dbus-python, putting signature='blah' on all method calls should work now (I fixed Bug #36206 in 0.84).
Hmm. Rather embarrassingly it turns out that the vast, vast majority of the time spent parsing XML is not spent parsing introspection XML. It is spent parsing XMPP XML. Patches I used to profile one of the Tubes tests follow, along with a cProfile output file. They do indeed show spinning the reactor to be extremely costly: but I'm not sure how much of that is just busy-looping in time that would be used anyway. Profiling Gabble itself would almost certainly yield interesting facts.
Created attachment 47471 [details] [review] DO NOT MERGE: cProfile for accept-private-stream-tube.py
Created attachment 47472 [details] [review] DO NOT MERGE: disable introspection for accept-private-stream-tube.py
Created attachment 47473 [details] cProfile output for accept-private-stream-tube.py
-- GitLab Migration Automatic Message -- This bug has been migrated to freedesktop.org's GitLab instance and has been closed from further activity. You can subscribe and participate further through the new bug through this link to our GitLab instance: https://gitlab.freedesktop.org/telepathy/telepathy-gabble/issues/36.
Use of freedesktop.org services, including Bugzilla, is subject to our Code of Conduct. How we collect and use information is described in our Privacy Policy.