Bug 29743

Summary: dynamic protocol bindings
Product: XCB Reporter: Havoc Pennington <hp>
Component: ProtocolAssignee: xcb mailing list dummy <xcb>
Status: RESOLVED MOVED QA Contact: xcb mailing list dummy <xcb>
Severity: normal    
Priority: medium Keywords: patch
Version: unspecified   
Hardware: Other   
OS: All   
Whiteboard:
i915 platform: i915 features:
Attachments: hacky xcb_send_request_dynamic just to get initial idea of perf/size impact
benchmark used with previous patch

Description Havoc Pennington 2010-08-22 19:18:40 UTC
It would be useful to have fast runtime access to something like the XML protocol descriptions (but converted to a binary format), similar to gobject-introspection. This would allow language bindings to dynamically generate wrappers on the fly. It would also allow things like xtrace or more one-off debug hacks to use the protocol descriptions.

Even the C binding would have the option to go dynamic. The protocol bindings are not tiny:
$ size /usr/lib/libxcb-glx.so
   text	   data	    bss	    dec	    hex	filename
  64447	   2616	      8	  67071	  105ff	/usr/lib/libxcb-glx.so

In theory, each request could be nothing but a call to a dynamic invoker like this:

xcb_send_request_dynamic(connection, flags, request_description, ...);

where "..." would be the args currently passed to the wrapper, and "request_description" would be the binary equivalent of the xml description.

Some notes on this:

* it would be nice to add the request name to the current xcb_protocol_request_t for debugging purposes. supporting lookup of request description given major and minor opcode, for example, would be super useful.

const xcb_request_description_t* xcb_describe_request(int major, int minor)

printf("got request %s\n", xcb_describe_request(major, minor)->name);

* ideally request_description would avoid any references to other symbols. I think e.g. the address of the extension id in xcb_protocol_request_t creates relocations. Qt's moc compiler generates descriptions as one huge string, for example, so there's only one symbol and it points to the read-only data section.

* in principle the xcb protocol wrapper libraries could just be the requests description blob in the library itself, plus headers with macros expanding to xcb_send_request_dynamic. for compat reasons you'd have to make the request-sending APIs functions instead, but they could just be functions that invoke xcb_send_request_dynamic() and nothing else. If the libraries contained only one read-only const symbol (the request description) then they would have notably less overhead than they do now. That's not back compatible, but even reducing each function to be just an invoke of xcb_send_request_dynamic() could result in some nice shrinkage.

* a dynamic setup would perhaps be slightly slower, but the reduced size probably matters more in the real world, and it ought to be pretty fast

* the dynamic setup makes non-C language bindings fixed-size, instead of a multiple of the number of requests the binding supports. (same principle as gobject-introspection)

* http://ometer.com/parallel.html could be used to deprecate the static C bindings

* whatever happens with the C bindings, exporting the "introspection data" at runtime instead of in effectively static-only (due to efficiency issues) XML files would be very useful for debug tools and language bindings.

Does this sound reasonable or does it conflict with some XCB goals?
Comment 1 Havoc Pennington 2010-08-28 22:23:59 UTC
Created attachment 38254 [details] [review]
hacky xcb_send_request_dynamic just to get initial idea of perf/size impact

Patch reduces xcb_create_window() to a one-line function in a simpleminded way.
This isn't introspection in a meaningful sense, just a cheap way to see if dynamic marshaling is small or slow.

The basic results are:

* the function comes out around 20 bytes smaller, but the separate description of how to marshal is about 20 bytes long, so no size win or lose.

* therefore the only size win would be to drop the wrapper functions from the library entirely, replacing them with inline functions in the header that just added type safety/convenience to xcb_send_request_dynamic()

* the marshaling is about 1.7 times slower. this is just creating the iovecs, does not include writing to the socket, so 1.7 times is the worst possible slowdown (if you have infinitely fast IO) rather than a real world slowdown.

Looks like no real point in dynamic request marshaling unless it's possible to drop the wrappers from the ABI and go to just inline wrappers in the headers. If doing that the size benefit of dropping all wrappers from the shared lib could be worth the perf cost. But, not ABI compatible obviously.

This isn't saying anything about other benefits of providing programmatic access to the protocol descriptions - the point of this patch was just to understand tradeoffs of dynamic request marshaling.
Comment 2 Havoc Pennington 2010-08-28 22:24:21 UTC
Created attachment 38255 [details] [review]
benchmark used with previous patch
Comment 3 GitLab Migration User 2019-02-16 19:40:11 UTC
-- GitLab Migration Automatic Message --

This bug has been migrated to freedesktop.org's GitLab instance and has been closed from further activity.

You can subscribe and participate further through the new bug through this link to our GitLab instance: https://gitlab.freedesktop.org/xorg/proto/xcbproto/issues/8.

Use of freedesktop.org services, including Bugzilla, is subject to our Code of Conduct. How we collect and use information is described in our Privacy Policy.