Bug 67909 - PBAP Sync Incremental failure with 5000 contacts
Summary: PBAP Sync Incremental failure with 5000 contacts
Status: RESOLVED FIXED
Alias: None
Product: SyncEvolution
Classification: Unclassified
Component: SyncEvolution (show other bugs)
Version: 1.3.99.3
Hardware: Other Linux (All)
: highest major
Assignee: Patrick Ohly
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2013-08-08 17:02 UTC by Eugenio Parodi
Modified: 2013-08-09 16:42 UTC (History)
1 user (show)

See Also:
i915 platform:
i915 features:


Attachments
The syncevo-dbus-server output Dump (23.17 KB, text/plain)
2013-08-08 17:02 UTC, Eugenio Parodi
Details
Dbus Full Session Dump (433.02 KB, text/plain)
2013-08-08 17:03 UTC, Eugenio Parodi
Details
sync.py output dump (453 bytes, text/plain)
2013-08-08 17:03 UTC, Eugenio Parodi
Details
ignore Status=active instead of terminating when seeing it (1.86 KB, patch)
2013-08-09 14:36 UTC, Patrick Ohly
Details | Splinter Review

Description Eugenio Parodi 2013-08-08 17:02:56 UTC
Created attachment 83852 [details]
The syncevo-dbus-server output Dump

Version Used 1.3.99.4
Bluez 5.5


* Syncevolution version Based on:
https://bugs.freedesktop.org/show_bug.cgi?id=64838

syncevolution-1.3.99.4.tar.gz
   + patches:
0002-D-Bus-server-utility-class-for-localed-D-Bus-interfa.txt
0003-D-Bus-server-LocaledListener-must-not-fail-without-s.patch
0004-PIM-adapt-to-locale-changes-at-runtime-FDO-66618.patch



* Test:
I Tried to sync an Android Phone Via PBAP with 5001 contacts.

The Bluetooth Stack Works fine.
I was able to retrieve these 5000 contacts via Obex (PullAll).



* Issue Found:
It seems that syncevo-dbus-server, trying to Sync the contacts in parallel to the PBAP download, after getting the first N contacts from the transfer, is not able to catch the next contacts downloaded.
Obex still continue to download the contacts (total amount of time 1' 30")
At the end of the download, syncevo-dbus-server print "Synchronization successful" and print the results of the first bunch of contacts retrieved.


* Still to confirm or check:
It seems that the tmp file used for the transfer is removed during the Obexd transfer.


* Attached:
- The Dbus Dump of the Entire session
- The syncevo-dbus-server output Dump
- The sync.py dump
Comment 1 Eugenio Parodi 2013-08-08 17:03:30 UTC
Created attachment 83853 [details]
Dbus Full Session Dump
Comment 2 Eugenio Parodi 2013-08-08 17:03:58 UTC
Created attachment 83854 [details]
sync.py output dump
Comment 3 Patrick Ohly 2013-08-09 07:13:02 UTC
The relevant logic which checks for new data is here:

bool PullAll::getContact(int contactNumber, pcrecpp::StringPiece &vcard)
{
    SE_LOG_DEBUG(NULL, "get PBAP contact #%d", contactNumber);
    if (contactNumber < 0 ||
        contactNumber >= m_numContacts) {
        SE_LOG_DEBUG(NULL, "invalid contact number");
        return false;
    }

    Content::iterator it;
    while ((it = m_content.find(contactNumber)) == m_content.end() &&
           m_session &&
           (!m_session->transferComplete() ||
            m_tmpFile.moreData())) {
        // Wait? We rely on regular propgress signals to wake us up.
        // obex 0.47 sends them every 64KB, at least in combination
        // with a Samsung Galaxy SIII. This may depend on both obexd
        // and the phone, so better check ourselves and perhaps do it
        // less often - unmap/map can be expensive and invalidates
        // some of the unread data (at least how it is implemented
        // now).
        while (!m_session->transferComplete() && m_tmpFile.moreData() < 128 * 1024) {
            g_main_context_iteration(NULL, true);
        }
        m_session->checkForError();
        if (m_tmpFile.moreData()) {
            // Remap. This shifts all addresses already stored in
            // m_content, so beware and update those.
            pcrecpp::StringPiece oldMem = m_tmpFile.stringPiece();
            m_tmpFile.unmap();
            m_tmpFile.map();
            pcrecpp::StringPiece newMem = m_tmpFile.stringPiece();
            ssize_t delta = newMem.data() - oldMem.data();
            BOOST_FOREACH (Content::value_type &entry, m_content) {
                pcrecpp::StringPiece &vcard = entry.second;
                vcard.set(vcard.data() + delta, vcard.size());
            }

            // File exists and obexd has written into it, so now we
            // can unlink it to avoid leaking it if we crash.
            m_tmpFile.remove();

            // Continue parsing where we stopped before.
            pcrecpp::StringPiece next(newMem.data() + m_tmpFileOffset,
                                      newMem.size() - m_tmpFileOffset);
            const char *end = addVCards(m_content.size(), next);
            int newTmpFileOffset = end - newMem.data();
            SE_LOG_DEBUG(NULL, "PBAP content parsed: %d out of %d (total), %d out of %d (last update)",
                         newTmpFileOffset,
                         newMem.size(),
                         (int)(end - next.data()),
                         next.size());
            m_tmpFileOffset = newTmpFileOffset;
        }
    }

    if (it == m_content.end()) {
        SE_LOG_DEBUG(NULL, "did not get the expected contact #%d, perhaps some contacts were deleted?", contactNumber);
        return false;
    }
    vcard = it->second;
    return true;
}

I see org.freedesktop.DBus.Properties.PropertiesChanged signals in the D-Bus log, which should wake up SyncEvolution and cause it to check the file size. In other words, I it is not obvious why it doesn't read more contacts from the file.

Can I get a syncevolution-log.html file of the target side with loglevel=4?
Comment 4 Patrick Ohly 2013-08-09 13:49:27 UTC
I was able to reproduce the problem. Expect a patch soon.
Comment 5 Patrick Ohly 2013-08-09 14:36:12 UTC
Created attachment 83893 [details] [review]
ignore Status=active instead of terminating when seeing it

This patch fixes the problem for me.
Comment 6 Eugenio Parodi 2013-08-09 16:42:26 UTC
It seems fixed also here.
I'll try to perform in any case some more tests.
Thanks.


Use of freedesktop.org services, including Bugzilla, is subject to our Code of Conduct. How we collect and use information is described in our Privacy Policy.