Bug 54105

Created attachment 66160 [details] [review]
rpc: Add buffer code

     * Buffer code is used for encoding RPC messages
     * Support read-only buffers from callers, and writeable
       buffers allocated by p11-kit

Created attachment 66161 [details] [review]
rpc: Implement PKCS#11 messages/client/server code

     * This enables passing around byte arrays which represent PKCS#11 RPC
       calls.
     * Caller is responsible for connecting/disconnecting and so on.
     * Client side caller gets a PKCS#11 CK_FUNCTION_LIST_PTR from
       p11_rpc_client_register() to call into, which generates callbacks
       with byte arrays to be transported.
     * Server side calls p11_rpc_server_handle() with a CK_FUNCTION_LIST_PTR
       on which relevant methods get called.
     * Doesn't yet implement the actual daemon or clients etc...

Something a bit ugly about these patches is the way that CK_FUNCTION_LIST modules are allocated to the client.

This is because of the way PKCS#11 works. We want to be able to return a CK_FUNCTION_LIST_PTR to calling code and have them act on it just as if it was a function list retrieved from a dlopened() PKCS#11 module.

In object oriented parlance: a PKCS#11 module can be thought of as the 'class' and slots are its 'object' instances. However unfortunately, and unlike with most OO the class pointer itself doesn't get passed to the various 'static' virtual methods.

In the RPC code we really have multiple modules that are the same 'class' or implementation, because they call out to different modules on the RPC server side. On the client side we need to track which one was called into, so we can relay that information to the RPC server. To do that, we have to reimplement the various PKCS#11 functions multiple times and have a different CK_FUNCTION_LIST in order to have different behavior for each module on the client side.

You see this in the RPC_DEFINE_CLIENT() macro. We then use RPC_CLIENT_ELEMENTS to define multiple clients each with a different number suffix for the various methods. Currently we define 64 clients ... which should be enough for anyone, famous last words :O 

But that's easily changed with a patch.

I guess we could provide a function to unregister a client module. Any calls through the CK_FUNCTION_LIST would then be undefined. This would be similar to a module might be dlclose'd.

If such an unregistered client module was registered again, then there's the issue of code erroneously calling into the newly registered module. But I guess that's the same issue as a dlclosed module being dlopened in the same memory space.

FWIW, in p11_rpc_client_register() we allocate no memory. So we won't have memory leaks or anything like that by not unregistering, as long as all modules are finalized as would be expected.

another possibility would be to use libffi and construct a new set of function pointers for each new module

(In reply to comment #4)
> another possibility would be to use libffi and construct a new set of function
> pointers for each new module

That's true. In general we need to make p11-kit buildable anywhere, so I had steered away from libffi. But the RPC stuff isn't going to be buildable on Windows, so maybe it's worth adding a libffi dependency. Will think about that.

Attachment 66161 [details] pushed as c785ab6 - rpc: Implement PKCS#11 messages/client/server code