Bug 60418 - FILEOPEN and EDITING particular .odt with excessive lots of Comments causes heavy CPU and memory load, can crash on saving
Summary: FILEOPEN and EDITING particular .odt with excessive lots of Comments causes h...
Status: NEW
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Writer (show other bugs)
Version: 3.6.5.2 release
Hardware: All All
: medium normal
Assignee: Not Assigned
QA Contact: Florian Reisinger
URL: http://www.mail-archive.com/discuss@d...
Whiteboard: perf
Keywords:
: 60419 (view as bug list)
Depends on:
Blocks:
 
Reported: 2013-02-07 12:17 UTC by Florian Reisinger
Modified: 2013-11-25 13:19 UTC (History)
5 users (show)

See Also:
i915 platform:
i915 features:


Attachments
Testfile (480.35 KB, application/vnd.oasis.opendocument.text)
2013-02-07 12:17 UTC, Florian Reisinger
Details
Without Comment tags (525.73 KB, application/vnd.oasis.opendocument.text)
2013-02-07 22:16 UTC, GiorgioMigliaccio
Details
Without Frame tags (430.20 KB, application/vnd.oasis.opendocument.text)
2013-02-07 22:16 UTC, GiorgioMigliaccio
Details
Solution - Without Form tags (440.91 KB, application/vnd.oasis.opendocument.text)
2013-02-07 22:17 UTC, GiorgioMigliaccio
Details

Description Florian Reisinger 2013-02-07 12:17:06 UTC
Created attachment 74336 [details]
Testfile

1) Load attached document
On my computer (Notebook AMD X2 RM-75 2.2 GHz Dual Core it took 1min 11sec to load the document and a crash when closing LibreOffice...
On my Windows 7 x64 LibreOffice needs 462.692 KB
(Version 4.1.0.0.alpha0+ (Build ID: bdfd8de57bf5767ce5c179a5e8705c7587f7b32)
TinderBox: Win-x86@6, Branch:master, Time: 2013-02-06_22:06:22)
and Kb using Version 3.6.5.2 (Build ID: 5b93205): 468.420 KB

2) Try to close it (APPHANG, APPCRASH) [or it hang itself up for 30 sec +]

But 1) is the "main" bug of this report
Comment 1 Rainer Bielefeld Retired 2013-02-07 14:32:30 UTC
Not NEW due to   
 <https://wiki.documentfoundation.org/QA/BugTriage#Set_Status_.26_Prioritize>
I will do some more tests

@Florian Reisinger
V. 3.6.5.2 has been mentioned by whom?
What OS have been tested?
What specific characteristics of reporter's documents are under suspect? 120 pages is not a very large documents.
How have the documents been created?
We should have original reporter from ML here.
Comment 2 Rainer Bielefeld Retired 2013-02-07 14:33:51 UTC
Document contains 2500 ODF Errors, most of them 'unexpected attribute "ls-langcode"'
Comment 3 Florian Reisinger 2013-02-07 15:01:30 UTC
@Giorgio Migliaccio: Please comment, why it is not ODF conform....
If there is no 100% ODF compitable ODF testfile I am going to close it as NOTOURBUG

@Rainer: I did the testing both on Win 7 (BTW It is in the description)

In my case: Win7 x64

the original reporter is in CC
It is just about the loading speed
I don't know how this is created --> If creation is not done by us --> NOTOURBUG, is it?

So, this should be all information I have...

PS: Could you please link to the ODF validator used...
Comment 4 Rainer Bielefeld Retired 2013-02-07 15:10:19 UTC
Currently my suspect is that it might have to do with the lots of lots of Comments in the document. AOOo 3.4.1 seems a little faster for FILEOPEN, but has similar problems, I think that never worked better.
Especially to delete Comments is very difficult
Comment 5 GiorgioMigliaccio 2013-02-07 15:12:54 UTC
*** Bug 60419 has been marked as a duplicate of this bug. ***
Comment 6 GiorgioMigliaccio 2013-02-07 15:17:38 UTC
(In reply to comment #3)
> @Giorgio Migliaccio: Please comment, why it is not ODF conform....
> If there is no 100% ODF compitable ODF testfile I am going to close it as
> NOTOURBUG
> 
> @Rainer: I did the testing both on Win 7 (BTW It is in the description)
> 
> In my case: Win7 x64
> 
> the original reporter is in CC
> It is just about the loading speed
> I don't know how this is created --> If creation is not done by us -->
> NOTOURBUG, is it?
> 
> So, this should be all information I have...
> 
> PS: Could you please link to the ODF validator used...

We've created the documents with LibreOffice 3.5.4 but, for integrating it into our own solution, we needed to extend the file with some of our own attributes which will contain a guid/key. 
So those attributes on your elements are the only thing that's added by us and is not according specs, but shouldn't be the reason why these documents are behaving as they are in LibreOffice/OpenOffice.org.
The tests were run on Windows 7 32 and 64-bit and 4 GB of RAM with LibreOffice versions 3.4.5, 3.5.4, 3.5.5, 3.6.x and 4.0 beta. All gave the same results, were 3.4.5 gave slightly better results with our full-blown resolved document, containing all the sub-documents.
Comment 7 Rainer Bielefeld Retired 2013-02-07 15:47:15 UTC
Validator: <http://odf-validator2.rhcloud.com/odf-validator2/>

Documents with thousands of comments are not a usual application for an Office Suite, so I reduce priority.


@GiorgioMigliaccio:
Please do not cite and do not use your mail client for Comments.
Is there any possibility for you to check whether the comments cause the problems?
Comment 8 Rainer Bielefeld Retired 2013-02-07 15:55:10 UTC
Hm. The document contains more than 200000 Frames for 42000 words - this is a VRY unusual document.

@GiorgioMigliaccio:
I wonder why I get offered to delete comments what seem to be related to a PW protected Area, what might be a REAL bug.
Can you please check that and submit a different Bug (and add me to CC) if my suspect concerning Comment deletion is correct?
Comment 9 GiorgioMigliaccio 2013-02-07 16:05:15 UTC
@Rainer : this document indeed contains a lot of frames. We mark ranges of text which are conditional or need to be looped with a start 'tag' and end 'tag', which are put physically in the document using frames, and also variables are put there using frames which get a specific attribute and guid so we can link it to our own objectmodel which specifies some additional info for these ranges. We've tried and investigated all LibreOffice (or OpenOffice.org) had to offer us at the time we started and this came out being the most flexible solution.
Trying to delete a comment for user x just crashes LibreOffice 4.0.0.2.
I'll try to remove them from the xml, manually, and see if this improves the document speed and memory usage in LO.
Comment 10 Rainer Bielefeld Retired 2013-02-07 17:06:52 UTC
(In reply to comment #9)
That would be great, we will have to separate the task into bite-sized different bugs. I believe most promising would be to create several documents based on this sample here:
One only with lots of frames
One only with lots of comments
One only with lots of ...

For each of those troublemakers a separate Bug (with "Block this one) should be submitted. So the possible roots can be examined in separate steps, what would increase chance for a quick fix.
Comment 11 GiorgioMigliaccio 2013-02-07 22:14:17 UTC
Ok, found it.
The particular document(s) were containing hundres of the following lines:
<form:form form:name="Form" form:apply-filter="true" form:command-type="table" form:control-implementation="ooo:com.sun.star.form.component.Form" office:target-frame="" xlink:href="" xlink:type="simple">
               <form:properties>
                  <form:property form:property-name="PropertyChangeNotificationEnabled" office:value-type="boolean" office:boolean-value="true"/>
               </form:properties>
</form:form>


Removing these from the content.xml sped up the document again to open in 5 secs and use only some 50 MB of memory!

I also tried removing the frames and the comments from the document, and this didn't change much, even though there were hundreds of them in the document, so, this seems implemented very well! :-)

But the main culprit seems to be the form:form tag for some reason, I don't even know how or when it got in to the documents.
The strange thing is that the same entry is occurring hundreds of times in the same document.
Comment 12 GiorgioMigliaccio 2013-02-07 22:16:05 UTC
Created attachment 74383 [details]
Without Comment tags
Comment 13 GiorgioMigliaccio 2013-02-07 22:16:34 UTC
Created attachment 74384 [details]
Without Frame tags
Comment 14 GiorgioMigliaccio 2013-02-07 22:17:07 UTC
Created attachment 74385 [details]
Solution - Without Form tags
Comment 15 Rainer Bielefeld Retired 2013-02-08 18:25:46 UTC
@GiorgioMigliaccio (In reply to comment #11)
Thank you for the additional sample documents, I will try next week to narrow down influence to the visible worse performance compared to LibO 3.4.5.

The most promising way to get a more visible improvement for your root problem with the "form form form" tags might be that you do some own tests how they get into the documents. And if there is a suspect that it's a LibO bug you should submit an additional new bug with a detailed description how a developer can reproduce the bug with very simple test equipment. Please feel free to add me to CC.
Comment 16 David 2013-05-04 01:32:32 UTC
I believe this bug is a duplicate of bug 61558.  I've found that toggling the "Hidden Paragraphs" option under the view menu will cause it to start responding properly while selecting "Update All" under the tools menu will cause an extreme lag.
Comment 17 GiorgioMigliaccio 2013-05-06 07:02:13 UTC
Thanks David.
Will try that.

We still didn't manage to find out why or when these form:form tags got into the document, which caused the incredibly slowdown and memory consumption.

But you mentioned 'Hidden paragraphs', and I know some of our document authors were playing with that feature, it might as well be the root cause of the problem.
So, perhaps, once you toggle that feature on/off, you get the form:form tags.

I will test this and give feedback.
Comment 18 David 2013-05-06 19:15:58 UTC
I don't think it actually has anything to do with hidden paragraphs since I have several large documents that act the same way and AFAIK they don't have any hidden paragraphs.
Comment 19 ign_christian 2013-07-09 14:34:56 UTC
I can open attachment 74336 [details] in about half minute using Pentium 1.3Ghz & 2Gb ram (LO 4.0.4.2 - Win7 32bit). 
Confirm crash when closing that file (perhaps freezing because when open LO again not opening document recovery).


Use of freedesktop.org services, including Bugzilla, is subject to our Code of Conduct. How we collect and use information is described in our Privacy Policy.