To the person who set 'high' priority: no, you don't get to set the priority of feature requests to a project you don't maintain. Please do not change bugs' priorities. If you want this to happen sooner, you can propose a high-quality git branch.

dbus-python's build system is currently Autotools, not distutils/setuptools/distribute/whatever else has a setup.py. This is really the feature request "please use setup.py, because it has the advantage over Autotools that pip/virtualenv understand it", I think? If so, it can be addressed from either side: either by making dbus-python use whichever build system is preferred at the moment, or by teaching pip/virtualenv about Autotools.

Another project I used to be involved with (telepathy-python) previously used distutils, and switched away because we became too frustrated by the absence of standard Autotools features like distcheck, so I'm not particularly keen on switching to a non-Autotools build system unless the advantages are really, really compelling and the disadvantages can be avoided; but I'll give people a chance to convince me rather than closing this as NOTABUG.

Thanks for the reply - I just wanted to know why this wasn't fixed yet. We won't use a python "package" that can't be installed with standard tools. For other people I recommend txdbus - it's pure python.

Created attachment 80061 [details] [review]
setup.py and MANIFEST.in

this patch resolve this issue

I've applied the patch that Guy put together and put it on a fork with instructions for installing directly from git at https://github.com/posborne/dbus-python.

It is very common for other python packages to want to declare package dependencies formally.  Is there any chance better support for the preferred python packaging practices can be supported in the main tree?  If so, can we push a new version of the package to pypi (Looks like Simon McVittie is the current owner)?

Obviously, there is still a system dependency on libraries and development headers, so a better solution to the problem might be either a pure python dbus library or one based on either ctypes or cffi.

(In reply to Paul Osborne from comment #4)
> It is very common for other python packages to want to declare package
> dependencies formally.  Is there any chance better support for the preferred
> python packaging practices can be supported in the main tree?

If this means "it must be built with distutils, not Autotools": no, I am not going to do this. As explained in Comment #1, I strongly prefer Autotools for modules that interact with system libraries.

However, Attachment #80061 [details] (wrapping Autotools in a setup.py shim) is an interesting approach to addressing that objection, and I'll try to find time to review/integrate that.

> Obviously, there is still a system dependency on libraries and development
> headers, so a better solution to the problem might be either a pure python
> dbus library or one based on either ctypes or cffi.

If there is a pure-Python or ctypes/FFI-based D-Bus library that is both Pythonic and good at D-Bus, I would be delighted to be able to say "dbus-python is deprecated, you should use xyz". My usual recommendation these days is GDBus via pygi, or QtDBus via PyQt. For non-GLib non-Qt code, do Python people generally approve of txdbus?

dbus-python is not a particularly great library, because it is constrained by backwards compatibility with older versions of itself; for instance, it's full of circular imports, it calls Introspect() which is not really meant to be for production code, sometimes it works out what type the caller wanted by guessing[1], and it can be crashed with a TypeError by API changes in services. Unfortunately, none of this can be fixed without losing its most major feature, namely "is compatible with older dbus-python".

The main reason that I have not handed over dbus-python to a new maintainer is that the people who would be interested in taking over tend to be insufficiently careful about backwards compat. If you want a D-Bus library that is *not* compatible with dbus-python, it's all MIT/X11-licensed, feel free to fork it under a different name.

[1] "In the face of ambiguity, refuse the temptation to guess"... or not.

Comment on attachment 80061 [details] [review]
setup.py and MANIFEST.in

Review of attachment 80061 [details] [review]:
-----------------------------------------------------------------

Sorry for the delay here, it's been a long time since I've had time I can spend on dbus-python. (I still don't really, but I'm reviewing and merging stuff anyway.)

If at all possible, I would still recommend using a system copy of dbus-python. dbus-python requires libdbus, a system C library, which you presumably got from "somewhere else" (for example Centos); I would recommend getting your dbus-python from that same "somewhere else". I'm aware that virtualenv and similar techniques try to isolate sets of Python libraries from each other, but the isolation is never going to be perfect in the case of Python libraries that wrap system-wide C libraries: you're using the system's libdbus (and dbus-daemon for that matter) whether you like it or not.

> [34mThu May 30 16:43:21 2013 +0300[m [33md00bfdf[32m (HEAD, origin/pypi01)[m [34msetup.py

Something has gone wrong with the patch header here. Did you paste from a terminal into a text editor, rather than creating an actual git commit?

::: MANIFEST.in
@@ +1,3 @@
> +include *
> +recursive-include * *
> +prune .git
\ No newline at end of file

If there's something (a file named /MANIFEST?) that I can auto-generate and include in the official (Autotools-produced) tarball to make it a more useful "sdist", then I'd prefer to do that, instead of using distutils to build a separate tarball.

::: setup.py
@@ +1,2 @@
> +# This file was created SPECIFICALLY to create a bdist that can be generated on an x64 Ubuntu and an x64 Centos
> +# machines.

Any chance of a copyright statement, and a specific statement that it is licensed like the rest of dbus-python? (the Expat license)

@@ +18,5 @@
> +root_dir = os.path.abspath(os.path.dirname(__file__))
> +
> +
> +def run_autogen():
> +    global root_dir

This is not necessary if you don't assign to root_dir, which you don't here.

@@ +28,5 @@
> +def load_version():
> +    import re
> +    from contextlib import closing
> +    version = [None, None, None]
> +    with closing(open(os.path.join(root_dir, "configure.ac"), "r")) as f:

open() already returns a context manager, so there should be no need to wrap it in another one.

@@ +57,5 @@
> +        os.environ['PYTHON'] = sys.executable
> +        run_autogen()
> +        spawn(["make", "-C", root_dir], search_path=True)
> +        for name in [ext.name for ext in self.extensions]:
> +            from_path = os.path.join(root_dir, name, ".libs", "{}.so".format(name))

This should be name + sysconfig.get_config_var('EXT_SUFFIX') on Python 3, or name + sysconfig.get_config_var('SO') on Python 2. That's the only real point of incompatibility with non-Linux that I can see here.

(Obviously on Windows, getting a working Autotools build is likely to be more challenging, but projects like mingw and msys2 address that by providing a minimal Unixy compilation environment even on Windows.)

@@ +63,5 @@
> +            log.info("copying extension {} from {} to {}".format(name, from_path, to_path))
> +            shutil.copyfile(from_path, to_path)
> +
> +
> +# HACK HACK HACK: ignore on non-Linux machines since this package won't work anyways.

This is, as the comment says, a hack. dbus-python absolutely requires the C modules and will not work without them; please use a different implementation (like pygi, PyQt or txdbus) if this is not acceptable for your use-case.

dbus-python is also portable to non-Linux OSs: it's available in at least FreeBSD, NetBSD and OpenBSD (I'm not sure whether it's in their respective base OSs or in "ports") and on Mac OS X via Homebrew, and Bug #51725 involves patches from Christoph Höger to compile it under Microsoft Visual C++, which presumably implies Windows. So it certainly isn't true to say that it won't work on non-Linux.

I've reimplemented something similar to Attachment #80061 [details], but using Autotools for the entire build and installation process. It seems to work with virtualenv and pip in my testing; please try it.

You can get a tarball to test with by running "./autogen.sh && make dist" in a git clone, then use "pip install dbus-python-whatever.tar.gz" to install it. Next time I release dbus-python, I'll upload a tarball similar to that one to PyPI.

(In reply to Simon McVittie from comment #6)
> If there's something (a file named /MANIFEST?) that I can auto-generate and
> include in the official (Autotools-produced) tarball to make it a more
> useful "sdist", then I'd prefer to do that, instead of using distutils to
> build a separate tarball.

I've added a MANIFEST and a MANIFEST.in to the official Autotools-generated tarball; hopefully they help.

I believe this is fixed in git master, but please comment or reopen if there's a problem.

Thanks for fixing, but I can't find a "setup.py" in current master:

https://cgit.freedesktop.org/dbus/dbus-python/tree/?id=d7677ab1bcabe60e84215eb17b51b644737bfcb0

(In reply to sedrubal from comment #9)
> Thanks for fixing, but I can't find a "setup.py" in current master

Sorry, I thought I'd pushed it but apparently not. It should really be there now; please try d14e14ad or later.

Great thanks. Can you say, when you'll release this on pypi?

(In reply to sedrubal from comment #11)
> Great thanks. Can you say, when you'll release this on pypi?

When someone who knows the Python/setuptools/etc. ecosystem better than I do - perhaps you - confirms that it works.

You can get a tarball that's basically equivalent to what I'll upload like this:

git clone .../dbus-python
cd dbus-python
./autogen.sh
make dist

If all goes well, that will contain a suitable MANIFEST, MANIFEST.in and setup.py to make tools like pip happy.

Thanks a lot for this, Simon.

I just tried it:

 $ git clone git://anongit.freedesktop.org/dbus/dbus-python
 $ cd dbus-python
 $ ./autogen.sh
 $ make dist
 make  dist-gzip am__post_remove_distdir='@:'
 make[1]: Entering directory '/home/bochecha/Workspace/dbus-python'
 make[1]: *** No rule to make target 'm4/am-check-pymod.m4', needed by 'distdir'.  Stop.
 make[1]: Leaving directory '/home/bochecha/Workspace/dbus-python'
 Makefile:1847: recipe for target 'dist' failed
 make: *** [dist] Error 2

What am I missing? :)

(In reply to Mathieu Bridon from comment #13)
> What am I missing? :)

That was a bug, which I've fixed now. Here is a snapshot tarball for testing if you are still having trouble:

https://people.freedesktop.org/~smcv/dbus-python-1.2.0+26+g19879fa.tar.gz

Here's what I just did in a Python 3 virtualenv:

 $ pip install ./dbus-python-1.2.0+26+g19879fa.tar.gz 
 Processing ./dbus-python-1.2.0+26+g19879fa.tar.gz
 Installing collected packages: dbus-python
   Running setup.py install for dbus-python ... done
 Successfully installed dbus-python-1.2.0.20160217
 $ python
 Python 3.4.3 (default, Jun 29 2015, 12:16:01) 
 [GCC 5.1.1 20150618 (Red Hat 5.1.1-4)] on linux
 Type "help", "copyright", "credits" or "license" for more information.
 >>> import dbus
 >>> proxy = dbus.SystemBus().get_object('org.freedesktop.systemd1', '/org/freedesktop/systemd1')
 >>> manager = dbus.Interface(proxy, dbus_interface='org.freedesktop.systemd1.Manager')
 >>> manager.LoadUnit('dbus.service')
 dbus.ObjectPath('/org/freedesktop/systemd1/unit/dbus_2eservice')

So I'd say this works just fine. :)

Thanks a lot for merging this in!

When I run ./autogen.sh I get this error message and no Makefile will be generated:

[...]
checking whether you want to build HTML docs... if possible
checking whether you want to build API docs... if possible
./configure: line 12589: syntax error near unexpected token `docutils'
./configure: line 12589: `  AX_PYTHON_MODULE(docutils)'

Now run 'make' to compile dbus-python.
$> echo $?
0
$>

- Fedora 23
- Installed packages:
  - python{,3}-docutils.noarch                0.12-0.3.20140510svn7747.fc23
  - epydoc.noarch                         3.0.1-16.fc23

Ah ok but setup.py seems to work anyway. Great job ;)

(In reply to sedrubal from comment #16)
> ./configure: line 12589: syntax error near unexpected token `docutils'
> ./configure: line 12589: `  AX_PYTHON_MODULE(docutils)'

You probably don't have autoconf-archive installed. <https://www.gnu.org/software/autoconf-archive/>