Bug 98006 - Add an org.fdo.Introspectable2 interface which returns structured introspection data
Summary: Add an org.fdo.Introspectable2 interface which returns structured introspecti...
Status: RESOLVED MOVED
Alias: None
Product: dbus
Classification: Unclassified
Component: core (show other bugs)
Version: git master
Hardware: Other All
: medium enhancement
Assignee: D-Bus Maintainers
QA Contact: D-Bus Maintainers
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2016-10-01 15:15 UTC by Philip Withnall
Modified: 2018-10-12 21:29 UTC (History)
3 users (show)

See Also:
i915 platform:
i915 features:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Philip Withnall 2016-10-01 15:15:53 UTC
Receiving introspection data from D-Bus as XML means that anybody consuming that data needs to have an XML parser, which is a bit of a pain if that software otherwise doesn’t need to depend on an XML parser.

Simon and I discussed potentially providing introspection data using the D-Bus type system (i.e. as basically a GVariant) as an alternative — org.freedesktop.DBus.Introspectable2.

Services would have to implement this interface before clients can use it. It would be allowed for a service to implement o.f.D.Introspectable, or o.f.D.Introspectable2, or both, or neither. If all the popular D-Bus libraries implemented both o.f.D.Introspectable and o.f.D.Introspectable2 then high coverage of services could be achieved fairly rapidly.

Here’s my suggestion for the interface design. Unless anybody has any objections, I’ll go ahead and implement it.

interface org.freedesktop.DBus.Introspectable2
  {
    org.freedesktop.DBus.Introspectable2.Introspect (out a(sa(sa(sa(sgya{sv})a{sv})a(sa(sga{sv})a{sv})a(sgya{sv})a{sv})a{sv}) introspection_data)
  }

Because that type string is hideous, here it is in a more block-like format:

array of object structs  /* calling it ‘object’ rather than ‘node’ seems clearer */
  {
    string path  /* node name from the XML format */
    array of interface structs
      {
        string name
        array of method structs
          {
            string name
            array of method argument structs
              {
                string name
                signature type
                byte direction  /* enum of: in, out */
                dict<string, variant> annotations
              }
            dict<string, variant> annotations
          }
        array of signal structs
          {
            string name
            array of signal argument structs
              {
                string name
                signature type
                dict<string, variant> annotations
                /* note: no direction, because it’s always out for signals */
              }
            dict<string, variant> annotations
          }
        array of property structs
          {
            string name
            signature type
            byte access_flags  /* flags of: read, write, read|write */
            dict<string, variant> annotations
          }
        dict<string, variant> annotations
      }
    dict<string, variant> annotations
  }

The data format is a fairly direct conversion of the existing introspection XML format, with the differences that:
 • it allows annotations anywhere (bug #86162);
 • nodes cannot be nested, and must provide their path; they must all be listed in the top-level array, specifying their absolute path instead of nesting;
 • argument names are not optional (because it is really annoying to see ‘arg0’, ‘arg1’, etc. in D-Feet);
 • argument directions are required for method arguments, and are not specified for signal arguments.

The same semantics for empty nodes as given in the specification applies: if a node is empty (its interfaces array is empty), it is assumed that the service has elected not to return introspection data for that node because doing so is too expensive. The client should introspect that node explicitly to get the introspection data.

I should note that this interface is very much intended to return programmatically-generated introspection data. Implementing Introspect() by hand would be a massive headache. As such, I haven’t made any effort to make it easy to write this stuff manually.

I suggest documentation is encoded using Markdown strings (as is the current fashion) as annotations (key name: org.freedesktop.DBus.Introspectable2.Documentation, value: a{ss} mapping a locale name (see `man setlocale`, e.g. ‘en_GB’) to a documentation string in the given language). This differs from previous approaches (see bug #88997), none of which have been formalised in the specification. org.gtk.GDBus.DocString uses DocBook strings; the ‘doc’ namespace uses its own XML elements; and inline XML comments use old-style gtk-doc syntax (i.e. a combination of DocBook and custom syntax). None of these are particularly portable — but I acknowledge that using Markdown means that anyone who consumes the documentation from //this// interface will need a Markdown parser. If they care about documentation, they probably have one of those already. gtk-doc has switched to Markdown syntax in the last few years, and other documentation formats use Markdown too. The idea of support for localisation of documentation is taken from https://wiki.allseenalliance.org/irb/extended_introspection_xml.

Structured type information (as discussed in bug #93912) should be provided as annotations.

Finally, I was wondering about forwards compatibility — this interface is prone to becoming outdated if D-Bus adds some more concepts like properties (even if they are added in a forwards-compatible manner elsewhere in D-Bus). Depending on how likely that is, we could add a ‘version’ argument to the Introspect() method, which controls whether it returns newer versions of the introspection data format. The introspected data itself would have to be returned as a variant, version 1 of which would have the type given above. Personally, I think this is overkill and we should just move to version 3 of the interface if needed, but I thought I should mention the idea just in case.

Feedback very welcome.
Comment 1 Lennart Poettering 2016-12-27 11:28:21 UTC
I like the idea of a structured API for introspection. A couple of points though:

I am very sure that introspection of static API data (i.e. method, signal, property prototypes) and of runtime object data (i.e. node tree) should be separated, and be queriable independently. Code generators usually look at the first data, and by separating this we could even permit pre-compiling this data at software buildtime instead of generating this dynamically during runtime. The node information otoh is inherently dynamic. Splitting this into two calls doesn't even introduce atomicity issues as the static API data shouldn't change anyway.

Also, I wouldn't introduce a new interface for this, but simply add two new method calls to the existing Introspectable interface.

using markdown sounds great, but i'd split that into a third api call or so. i doubt this should be included in every attempt to introspect a service...
Comment 2 Philip Withnall 2016-12-27 23:02:53 UTC
(In reply to Lennart Poettering from comment #1)
> I like the idea of a structured API for introspection. A couple of points
> though:
> 
> I am very sure that introspection of static API data (i.e. method, signal,
> property prototypes) and of runtime object data (i.e. node tree) should be
> separated, and be queriable independently. Code generators usually look at
> the first data, and by separating this we could even permit pre-compiling
> this data at software buildtime instead of generating this dynamically
> during runtime. The node information otoh is inherently dynamic. Splitting
> this into two calls doesn't even introduce atomicity issues as the static
> API data shouldn't change anyway.

Makes sense.

> Also, I wouldn't introduce a new interface for this, but simply add two new
> method calls to the existing Introspectable interface.

Why not? If we keep using the old (v1) interface name, then we can never deprecate the old method.

> using markdown sounds great, but i'd split that into a third api call or so.
> i doubt this should be included in every attempt to introspect a service...

I suspect you mean that *documentation* should not be included in every introspection? Seems reasonable. How about a ‘flags’ parameter to the Introspect() method which enables/disables returning annotations?
Comment 3 GitLab Migration User 2018-10-12 21:29:01 UTC
-- GitLab Migration Automatic Message --

This bug has been migrated to freedesktop.org's GitLab instance and has been closed from further activity.

You can subscribe and participate further through the new bug through this link to our GitLab instance: https://gitlab.freedesktop.org/dbus/dbus/issues/158.


Use of freedesktop.org services, including Bugzilla, is subject to our Code of Conduct. How we collect and use information is described in our Privacy Policy.