Service activation works quite well for starting services on-demand. However, over time memory usage will continue to go up. The problem is that if a service decides to just call exit() at some arbitrary point in time, there is a race condition where a message for it may be in-flight.
There are several possible in-flight situations.
1) A message is in the queue from a client to the bus, unread by the bus.
2) A message is in the bus pending a write to the service
3) A message has been written to the service, but not read by the service
What would be nice is if DBus had a way for a process to explicitly say "hold any messages for org.foo.MyService, I'm planning to exit".
For situation 1), dbus doesn't yet know about the message, so it says "OK". Then later when it does process the message, if the service has not yet relinquished the service name, the bus holds the message until that event happens. If the service has released the name in some way, service activation proceeds.
For situation 2) and 3), dbus returns a reply that says "messages pending, don't exit".
Needs a bit more analysis but it seems plausible.
This has come up a few times before. I think you just need to 1) release the name then 2) process the pending queue.
The exception is if your app needs an exclusive lock on some resource, like the sound card. In that case, one solution is to have a separate lock: which can be a second bus name, or an fcntl lock, or whatever else. And the starting-up replacement service would wait for this lock before it starts processing the queue.
Downsides of doing this with some special feature are:
* older dbus wouldn't have it anyway
* it complicates bus names and message routing even more, and nobody understands it already
Two bus names (one for exclusivity, one for message routing) I'm pretty sure works perfectly and is pretty straightforward. An app needs both names to begin processing the queue, only the "routing" name to receive messages, and only the "lock" name to continue processing messages (the routing name has to be owned to _begin_ processing but the lock name to continue).
In your cases:
1) and 2) should both go to the new service instance if the old has dropped the routing name. The old one processes what remains of the queue, then drops lock name. When the new instance gets the lock name, it can begin processing the new messages.
3) should be addressed if the old instance iterates the dbus main loop source until there's nothing to read and no messages left.
3) could maybe be enhanced by a call that just says "close() my connection from the bus side" which would allow a more deterministic way to read everything still in the socket buffer. The bus could only close() once the currently-in-process message is written out in full. However, in practice I think this is mostly hygiene and Correctness, while just iterating the main loop source until it stops and calling close() on the client side probably works OK in practice.
*** This bug has been marked as a duplicate of bug 11454 ***