Bug 98019

Summary:	UTF-8 encoded non ascii characters from text/x-vcalendar (v1.0) broken when synced from Nokia N9
Product:	SyncEvolution	Reporter:	EKO <deloptes>
Component:	SyncML	Assignee:	Patrick Ohly <patrick.ohly>
Status:	RESOLVED FIXED	QA Contact:
Severity:	normal
Priority:	medium	CC:	deloptes, syncevolution-issues
Version:	1.5.1
Hardware:	Other
OS:	All
Whiteboard:
i915 platform:		i915 features:
Attachments:	debug log as requested the patch for Nokia N9 sending ==0D=0A= at end of line

Description EKO 2016-10-02 19:09:33 UTC

Created attachment 126958 [details]
debug log as requested

The problem can be observed when syncing Cyrillic or German text from Nokia N9 to the desktop (TDE in my case). The problem with German text is visible when text is long (gets folded) and contains some non ascii (äöüÄÖÜß)

Steps to reproduce:

SUMMARY: "Тест дълго описание за задача татата попопо мамама бабаба какака лалала хихихи"
DESCRIPTION: "Тест дълго описание за задача татата попопо мамама бабаба какака лалала хихихи"

1. Create an item in the desktop calendar with long title (summary) and a long text (description) with the text sample from above
2. Sync to the phone (Nokia N9)
3. Modify the item on the phone (so that it syncs back again)
4. Sync to the desktop
  syncevolution -r loglevel=4 nokia_N9 calendar+todo

It is obvious that TMimeDirProfileHandler::parseText(...) gets the quoted-printable already broken. Note that according RFC 2045 each line should end with at least '=' (softbreak), where in the given example below '\r\n' is also encoded into =0D=0A resulting in each line ending with '==0D=0A=', which in turn breaks the quoted-printable parser in mimedirprofile.cpp and renders the use of syncevolution with N9 and non ascii to null.

In the log we see:
[2016-09-27 08:51:45.496] 'Item_Parse' - parsing SyncML item, SyncOp=add, format=plain-text, RemoteID=526 [--][++] [->end] [->enclosing]
[2016-09-27 08:51:45.496] Created new item of datatype 'vCalendar10', localID='' remoteID='526'
[2016-09-27 08:51:45.496] Parsing: 
[2016-09-27 08:51:45.496] 
BEGIN:VCALENDAR
PRODID:-//K Desktop Environment//NONSGML libkcal 4.3//EN
VERSION:1.0
BEGIN:VEVENT
DTSTART:20160928T080000Z
DTEND:20160928T081500Z
DCREATED:20160927T065014Z
UID:libkcal-1150394398.619
SEQUENCE:1
LAST-MODIFIED:20160927T065054Z
X-ORGANIZER:MAILTO:emanoil.kotsev@fincom.at
DESCRIPTION;ENCODING=QUOTED-PRINTABLE;CHARSET=utf-8:=D0=A2=D0=B5=D1=81=D1=82 =D0=B4=D1=8A=D0=BB=D0=B3=D0=BE =D0=BE=D0=BF=D0==0D=0A=
=B8=D1=81=D0=B0=D0=BD=D0=B8=D0=B5 =D0=B7=D0=B0 =D0=B7=D0=B0=D0=B4=D0=B0==0D=0A=
=D1=87=D0=B0 =D1=82=D0=B0=D1=82=D0=B0=D1=82=D0=B0 =D0=BF=D0=BE=D0=BF=D0==0D=0A=
=BE=D0=BF=D0=BE =D0=BC=D0=B0=D0=BC=D0=B0=D0=BC=D0=B0 =D0=B1=D0=B0=D0=B1==0D=0A=
=D0=B0=D0=B1=D0=B0 =D0=BA=D0=B0=D0=BA=D0=B0=D0=BA=D0=B0 =D0=BB=D0=B0=D0==0D=0A=
=BB=D0=B0=D0=BB=D0=B0 =D1=85=D0=B8=D1=85=D0=B8=D1=85=D0=B8
SUMMARY;ENCODING=QUOTED-PRINTABLE;CHARSET=utf-8:=D0=A2=D0=B5=D1=81=D1=82 =D0=B4=D1=8A=D0=BB=D0=B3=D0=BE =D0=B7=D0=B0=D0==0D=0A=
=B3=D0=BB=D0=B0=D0=B2=D0=B8=D0=B5 =D0=B7=D0=B0 =D0=B7=D0=B0=D0=B4=D0=B0==0D=0A=
=D1=87=D0=B0 =D1=82=D0=B0=D1=82=D0=B0=D1=82=D0=B0 =D0=BF=D0=BE=D0=BF=D0==0D=0A=
=BE=D0=BF=D0=BE =D0=BC=D0=B0=D0=BC=D0=B0=D0=BC=D0=B0 =D0=B1=D0=B0=D0=B1==0D=0A=
=D0=B0=D0=B1=D0=B0 =D0=BA=D0=B0=D0=BA=D0=B0=D0=BA=D0=B0 =D0=BB=D0=B0=D0==0D=0A=
=BB=D0=B0=D0=BB=D0=B0 =D1=85=D0=B8=D1=85=D0=B8=D1=85=D0=B8 123
CLASS:PUBLIC
PRIORITY:0
TRANSP:0
X-EPOCAGENDAENTRYTYPE:APPOINTMENT
END:VEVENT
END:VCALENDAR
[2016-09-27 08:51:45.496] parseMimeDir: property not parsed (unknown or not storable): PRODID:-//K Desktop Environmen
[2016-09-27 08:51:45.496] parseMimeDir: property DTSTART parsing delayed, rank=1
[2016-09-27 08:51:45.496] parseMimeDir: property DTEND parsing delayed, rank=1
[2016-09-27 08:51:45.496] parseMimeDir: property not parsed (unknown or not storable): X-ORGANIZER:MAILTO:emanoil.kot
[2016-09-27 08:51:45.496] No value stored for field index 8 because remote indicates not supported in devInf
[2016-09-27 08:51:45.496] parseMimeDir: property not parsed (unknown or not storable): X-EPOCAGENDAENTRYTYPE:APPOINTM
[2016-09-27 08:51:45.496] parseMimeDir: now parsing delayed property rank=1: :20160928T080000Z 
DTEND:20160
[2016-09-27 08:51:45.496] parseMimeDir: now parsing delayed property rank=1: :20160928T081500Z 
DCREATED:20
[2016-09-27 08:51:45.496] Successfully parsed: 
[2016-09-27 08:51:45.496] Item LocalID='', RemoteID='526', operation=add, size: [maxlocal,maxremote,actual]
[2016-09-27 08:51:45.496] 
-  0 :    integer SYNCLVL         [ n/a,   0,     0] : <unassigned>
-  1 :    integer ISEVENT         [ n/a,   0,     0] : 1
-  2 :  timestamp DMODIFIED       [  -1,   0,     0] : 2016-09-27T06:50:54Z (TZ: UTC)
-  3 :  timestamp DCREATED        [  -1,   0,     0] : 2016-09-27T06:50:14Z (TZ: UTC)
-  4 :  timestamp DGENERATED      [ n/a,   0,     0] : <unassigned>
-  5 :     string UID             [  -1,   0,    22] : "libkcal-1150394398.619"
-  6 :     string CATEGORIES      [  -1,   0,     0] : <array with 0 elements>
-  7 :    integer CLASS           [  -1,   0,     0] : 0
-  8 :    integer TRANSP          [ n/a,   0,     0] : <unassigned>
-  9 :  multiline SUMMARY         [  -1,   0,   159] : "Тест дълго за�=
�лавие за з�...а какака ла�=
�ала хихихи 123"
- 10 :  multiline DESCRIPTION     [  -1,   0,   155] : "Тест дълго оп�=
�сание за з�...аба какака ла�=
�ала хихихи"
- 11 :  multiline LOCATION        [  -1,   0,     0] : <empty>
- 12 :        url URL             [ n/a,   0,     0] : <unassigned>
- 13 :     string RR_FREQ         [  -1,   0,     0] : <empty>
- 14 :    integer RR_INTERVAL     [   0,   0,     0] : <empty>
- 15 :    integer RR_FMASK        [   0,   0,     0] : <empty>
- 16 :    integer RR_LMASK        [   0,   0,     0] : <empty>
- 17 :  timestamp RR_END          [   0,   0,     0] : <empty>
- 18 :  timestamp DTSTART         [  -1,   0,     0] : 2016-09-28T08:00:00Z (TZ: UTC)
- 19 :  timestamp DTEND           [  -1,   0,     0] : 2016-09-28T08:15:00Z (TZ: UTC)
- 20 :  timestamp DURATION        [ n/a,   0,     0] : <unassigned>
- 21 :  timestamp COMPLETED       [  -1,   0,     0] : <empty>
- 22 :  timestamp DUE             [  -1,   0,     0] : <empty>
- 23 :     string GEO_LAT         [ n/a,   0,     0] : <unassigned>
- 24 :     string GEO_LONG        [ n/a,   0,     0] : <unassigned>
- 25 :    integer PRIORITY        [  -1,   0,     0] : 0
- 26 :    integer STATUS          [  -1,   0,     0] : <empty>
- 27 :    integer PERCENT_COMPLETE [ n/a,   0,     0] : <unassigned>
- 28 :  timestamp ALARM_TIME      [  -1,   0,     0] : <empty>
- 29 :     string ALARM_SNOOZE    [  -1,   0,     0] : <empty>
- 30 :     string ALARM_REPEAT    [  -1,   0,     0] : <empty>
- 31 :     string ALARM_MSG       [  -1,   0,     0] : <empty>
- 32 :     string ALARM_ACTION    [   0,   0,     0] : <empty>
- 33 :    integer ALARM_REL       [   0,   0,     0] : <empty>
- 34 :     string ALARM_UID       [ n/a,   0,     0] : <unassigned>
- 35 :     string PARENT_UID      [ n/a,   0,     0] : <unassigned>
- 36 :  timestamp EXDATES         [  -1,   0,     0] : <array with 0 elements>
- 37 :  timestamp EXDATES_DETACHED [  -1,   0,     0] : <array with 0 elements>
- 38 :  timestamp ORIGSTART       [ n/a,   0,     0] : <unassigned>
- 39 :    integer SEQNO           [  -1,   0,     0] : 1
- 40 :     string ATTENDEES       [  -1,   0,     0] : <array with 0 elements>
- 41 :     string ATTENDEE_CNS    [  -1,   0,     0] : <array with 0 elements>
- 42 :    integer ATTENDEE_PARTSTATS [  -1,   0,     0] : <array with 0 elements>
- 43 :    integer ATTENDEE_ROLE   [  -1,   0,     0] : <array with 0 elements>
- 44 :    integer ATTENDEE_RSVP   [  -1,   0,     0] : <array with 0 elements>
- 45 :     string ATTENDEE_LANG   [  -1,   0,     0] : <array with 0 elements>
- 46 :    integer ATTENDEE_CUTYPE [  -1,   0,     0] : <array with 0 elements>
- 47 :     string ORGANIZER       [  -1,   0,     0] : <empty>
- 48 :     string ORGANIZER_CN    [ n/a,   0,     0] : <unassigned>
+
[2016-09-27 08:51:45.496] 'ScriptExecute' - Start executing Sc

Comment 1 Patrick Ohly 2016-10-06 13:26:30 UTC

Fix was discussed on mailing list ("[SyncEvolution] Broken UTF-8 item passed to backend") and is getting tested as part of next nightly build.

Comment 2 EKO 2016-10-07 21:41:38 UTC

Created attachment 127131 [details] [review]
the patch for Nokia N9 sending ==0D=0A= at end of line

The resulting patch from our discussion and work.
Thanks

Comment 3 Patrick Ohly 2018-01-09 16:38:47 UTC

Was fixed already in 1.5.2.

Use of freedesktop.org services, including Bugzilla, is subject to our Code of Conduct. How we collect and use information is described in our Privacy Policy.