Bug 84802 - Huge Files not responding
Summary: Huge Files not responding
Status: RESOLVED INVALID
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: LibreOffice (show other bugs)
Version:
(earliest affected)
4.3.1.1 rc
Hardware: All All
: high major
Assignee: Not Assigned
URL:
Whiteboard:
Keywords:
Depends on:
Blocks: Chart
  Show dependency treegraph
 
Reported: 2014-10-08 13:02 UTC by SriJanani
Modified: 2016-12-20 15:52 UTC (History)
4 users (show)

See Also:
Crash report or crash signature:


Attachments
blocage pour finalise l enregistrement libreoffice (21.88 KB, image/jpeg)
2014-10-13 08:24 UTC, AAC
Details
Graph plot screen shot (325.66 KB, image/jpeg)
2014-10-16 06:27 UTC, SriJanani
Details

Note You need to log in before you can comment on or make changes to this bug.
Description SriJanani 2014-10-08 13:02:54 UTC
Huge files not responding in Libre office all versions. We tested it keeps crashing. The files contains 60000 rows with normal data and texts, however the same file works perfectly in MS office its high time productivity for us. Need your help to fix it. I am attaching the file. Try to draw a graph for the column B,C and D. Even more than 2 MB files having same issue.
 
PC: Core 2Duo with 2 GB RAM OS- Windows XP
 
Regards, SriJanani
Comment 1 m_a_riosv 2014-10-08 20:24:16 UTC Comment hidden (obsolete)
Comment 2 SriJanani 2014-10-09 03:08:38 UTC
The actual original file size when extracted using excel is 15 MB after manually converting to Libre calc is 4 MB. Due to file sixe limitation unable to attach it. Please advice.

Thank You,

Regards,
SriJanani
Comment 3 Julien Nabet 2014-10-09 20:38:05 UTC
SriJanani: you may use dropbox or follow this link:
https://wiki.documentfoundation.org/QA/Bugzilla/Attachments/Temporary_Storage_for_Big_Files
Comment 4 SriJanani 2014-10-11 07:27:32 UTC
I have uploaded the file please check and revert.
Comment 5 m_a_riosv 2014-10-11 12:54:07 UTC
Where the file is?
Comment 6 SriJanani 2014-10-13 07:06:54 UTC
I have uploaded the file in the below link

https://wiki.documentfoundation.org/File:File_converted_Sep_30th-_With_reference_to_bug_84819.ods 

Regards,
SriJanani
Comment 7 AAC 2014-10-13 08:24:12 UTC
Created attachment 107764 [details]
blocage pour finalise l enregistrement libreoffice

je ne peux terminer l installation de libre office 
voir le bug suivant
Comment 8 Julien Nabet 2014-10-14 20:09:40 UTC
AAC: please don't hijack this bugtracker, it's completely unrelated.
You can submit a new bug by using this link:
https://www.libreoffice.org/get-help/bug/
Comment 9 Julien Nabet 2014-10-14 20:16:42 UTC
SriJanani: I could open the file with 4.3.2 LO Debian package.
Could you give a minimal step by step process to try to crash or hang?
Comment 10 SriJanani 2014-10-16 06:27:08 UTC
Created attachment 107915 [details]
Graph plot screen shot

Attached the screenshot of the file not responding while we try to plot the graph for the column- B,C,D.

Try to plot the graph it hangs and not reponds. Opening is ok takes 5 mins.
Thank You,

Regards,
SriJanani
Comment 11 SriJanani 2014-10-17 12:05:58 UTC
I request you to try with the windows platform OS like windows 7 and windows XP.

Awaiting your response.

Regards,
SriJanani
Comment 12 Tim Lloyd 2014-10-20 04:58:09 UTC
confirmed on Fedora 21 LO4.4.0.0

Open file (it takes 2-3 minutes) and select columns B,C & D. Insert->Object->Chart and calc locks up. I left it for 10 minutes. No response. I also copied the contents into a new document. Same problem.

Same problem in XP with LO4.2.0.4

Changing status to NEW
Comment 13 Julien Nabet 2014-10-20 19:49:08 UTC
On pc Debian x86-64 with master sources updated today, I could reproduce this.

Now I'll reduce the importance since having thousands of data for a chart is not a current thing for just an "Office" usage. But perhaps I'm wrong.
Comment 14 SriJanani 2014-10-23 04:59:52 UTC
Now I am confused how do we come to a conclusion why its happening... It will be a big challenge if we cannot use files with large data.

Regards,
SriJanani
Comment 15 Julien Nabet 2014-10-23 05:41:24 UTC
Michael/Markus: Thought it could be interesting to have your opinion about this tracker.
In my opinion trying to make a chart with 60 000 rows is rather a test limit exercise than a current case but perhaps I'm wrong.
Comment 16 Michael Meeks 2014-10-23 10:13:15 UTC
Crashing is bad; though there may be an out-of-memory condition I guess. Ultimately, big charts are rendered with the drawing-layer which is pretty fundamentally large & slow I'm afraid =) it's quite a chunk of engineering work to improve that. Possibly OpenGL rendering would improve the rendering performance at least.
Comment 17 SriJanani 2014-10-25 03:53:40 UTC
I would also like to report that even the base file is created in Libre office format and worked upon. If we have multiple thousand rows and columns then we do some vlookup or filtering commands the file crashes.

This is a major concern for us since we deal with lot of hige files.

Regards,
Srijanani
Comment 18 SriJanani 2014-10-28 12:57:33 UTC
Awaiting your response for the same huge file issues. Please advice.

Regards,
SriJanani
Comment 19 Michael Meeks 2014-10-28 14:19:33 UTC
Hi Srijanani,

> This is a major concern for us since we deal with lot of hige files.
...
> Awaiting your response for the same huge file issues. Please advice.

I think, perhaps, your expectations are not well aligned here with what a volunteer project can deliver; there are ~6000 open bugs, of which this is just one - with prolly a lower priority than most: we know charting large data sets is not at all optimal. There is some chart performance work going on to target LibreOffice 4.4 which may (or may not) address your specific problem and may arrive in a usable state for an enterprise in around six months.

Feel free to get involved in the project to fix your issue though; I'd love to see patches to help out here; I'd recommend starting by getting a profile of the code in cachegrind, and seeing where the bottlenecks are. Patches are most welcome.

HTH,

Michael.
Comment 20 SriJanani 2014-10-30 12:34:03 UTC
Hi Michael,

Thanks for getting back to us. We appreciate the efforts that the volunteer team is delivering, but we would be happy to contribute what ever we can do from our end. 

We are a huge crowd where we can be the best testing ground for this project. Since we are not developers I would like to take your help so that we can contribute as much as we can as you mentioned below.

1) We need steps how to send you patches or test logs
2) We are not aware of profile code in cachegrind- Let us know how to do? any softwares will help?


Awaiting your response-> we will happy to help you for the development.

Thank You,

Regards,
SriJanani
Comment 21 Michael Meeks 2014-10-30 13:10:37 UTC
Hi SriJanani,

> we would be happy to contribute what ever we can do from our end. 

Great - our biggest need is for developers to work on the project to fix bugs, performance issues etc.

> We are a huge crowd where we can be the best testing
> ground for this project.

We don't really lack testing; we have tens of millions of users who test & file bugs. Then again - if you're using the latest master builds its useful to file regressions quickly vs. that. Its also really useful to get involved with QA:

https://wiki.documentfoundation.org/QA/BugTriage

> Since we are not developers I would like to take your help so that we
> can contribute as much as we can as you mentioned below.

As I said, you need a developer to fix this; you can try to make it easier for one to care about this.

> 1) We need steps how to send you patches or test logs

Patches are unified diffs of code changes; they need to be created by a developer =)

> 2) We are not aware of profile code in cachegrind- Let us know how to do?
> any softwares will help?

If you run Linux, install a build with debuginfo - well, just google for cachegrind and how it is used; certainly you can train yourself in how to do this. Even with a cachegrind trace; I'm not hyper-optimistic you'll find anyone jumping in to fix this bug for you.

But you may be lucky.
Comment 22 QA Administrators 2015-05-06 14:14:12 UTC Comment hidden (obsolete)
Comment 23 QA Administrators 2015-06-08 14:26:43 UTC Comment hidden (obsolete)
Comment 24 Yuri Khan 2016-10-14 11:34:45 UTC
I want to resurrect this issue.

I have a Calc spreadsheet with some 75000 rows, three columns, unsaved data.

I inadvertently click the Chart button on the toolbar. LibreOffice takes my data hostage and hangs. I can either wait for it to show me the dialog box where I can choose the chart type, or I can kill LibreOffice, losing my data.


Expected behavior:

* After clicking the Chart button or selecting the Insert | Object | Chart menu command, the Chart Wizard appears and is responsive immediately. The preview generation is started asynchronously or skipped altogether.

* As the user modifies settings that affect the preview, preview generation is restarted asynchronously.

* If the user clicks Finish in the Chart Wizard, preview generation is stopped and the actual chart generation starts. A Stop button is displayed; clicking it cancels chart generation.


Plotting being slow is not the principal issue here; we all can understand there are limits to computer performance. However, the user must remain in control at all times; waiting forever or killing LibreOffice along with the data is a shitty choice.
Comment 25 Buovjaga 2016-12-20 15:52:37 UTC
Yuri: this issue has too many comments already, so it would be best, if you copied your proposal in comment 24 to a new report.

The performance issue is a broader one, so I will close this for now. It will require a big renovation.