Bug 52669 - asynchronous + batched item changes
Summary: asynchronous + batched item changes
Alias: None
Product: SyncEvolution
Classification: Unclassified
Component: SyncEvolution (show other bugs)
Version: unspecified
Hardware: All All
: high normal
Assignee: Patrick Ohly
QA Contact:
Depends on:
Blocks: 52680 64838
  Show dependency treegraph
Reported: 2011-11-08 15:53 UTC by Patrick Ohly
Modified: 2014-07-23 09:13 UTC (History)
1 user (show)

See Also:
i915 platform:
i915 features:


Description Patrick Ohly 2012-07-29 18:36:00 UTC

---- Reported by patrick.ohly@intel.com 2011-11-08 15:53:42 +0000 ----

When making changes to a backend (deleting/updating/adding items), the result (success/failure + item ID + change tracking information) is expected right away. That prevents interleaving execution of the change and processing the next item and batching of multiple changes.

This is bad for performance when the backend needs to talk to some remote resource (CalDAV/CardDAV server, ActiveSync).

It might be possible to address this by using the delayed item ID resolution mechanism in libsynthesis. All the intermediate layers (TrackingSyncSource, MappingSyncSource, APIs) also need to be changed. Finally the backends must be changed to take advantage of this.

The first candidate for this is the ActiveSync backend where batching changes would result in a considerable reduction of total sync runtime because of less network roundtrips.

--- Bug imported by patrick.ohly@gmx.de 2012-07-29 20:36 UTC  ---

This bug was previously known as _bug_ 23967 at https://bugs.meego.com/show_bug.cgi?id=23967
Comment 1 Patrick Ohly 2013-07-16 14:35:55 UTC
Implemented for EDS contacts and adding/updating. It was not implemented for EDS and deleting (although the core engine would support it now) because the EDS API does not provided detailed enough error reports to recognize the "not found" errors per item.
Comment 2 Patrick Ohly 2014-04-14 19:47:19 UTC
(In reply to comment #1)
> Implemented for EDS contacts and adding/updating.

However, it turned out to be faulty when used for something other than simple PBAP caching and therefore is only used for that in SyncEvolution 1.4.

commit c435e937cd406e904c437eec51a32a6ec6163102
Author: Patrick Ohly <patrick.ohly@intel.com>
Date:   Fri Nov 29 14:02:05 2013 +0100

    engine: disable batching by default
    Change the EDS backend to not batch by default and only enable it
    for PIM Manager PBAP syncs.
    This avoids unnecessary overhead when running a normal SyncML sync
    with a non-SyncEvolution peer, because then the batched operation
    would be flushed immediately by the Synthesis engine, and more
    importantly, it fixes a segfault in
    Client::Sync::eds_event_eds_contact::testLargeObject EDS<->DAV tests.
    Enabling batching during general syncs obviously needs further work...
Comment 3 Patrick Ohly 2014-07-23 09:13:41 UTC
Fix in libsynthesis master branch:

commit 7b636720a46699e82c7caadf4550a0cd23571465
Author: Patrick Ohly <patrick.ohly@intel.com>
Date:   Wed Jul 2 13:12:58 2014 +0200

    fix asynchronous item update
    gcc 4.9 found a flaw in the code path for updating an item when the DB
    detects a duplicate: some variables where not preserved across
    function restarts.

And from the SyncEvolution master branch:

commit d06e40d365b5f9a213af3938b1dac78ae18b1b85
Author: Patrick Ohly <patrick.ohly@intel.com>
Date:   Mon Jul 21 13:40:29 2014 +0200

    engine: enable batching by default (FDO #52669)
    This reverts commit c435e937cd406e904c437eec51a32a6ec6163102.
    Commit 7b636720a in libsynthesis fixes an unitialized memory read in
    the asynchronous item update code path.
    Testing confirms that we can now used batched writes reliably with EDS
    (the only backend currently supporting asynchronous writes +
    batching), so this change enables it again also for local and
    SyncEvolution<->SyncEvolution sync (with asynchronous execution of
    contact add/update overlapped with SyncML message exchanges) and other
    SyncML syncs (with changes combined into batches and executed at the
    end of each message).

Use of freedesktop.org services, including Bugzilla, is subject to our Code of Conduct. How we collect and use information is described in our Privacy Policy.