Bug 76552 - Massive memory leak in <sheet>.moveRange (only) when called after <sheets>.insertNewByName
Summary: Massive memory leak in <sheet>.moveRange (only) when called after <sheets>.in...
Status: RESOLVED WORKSFORME
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Calc (show other bugs)
Version:
(earliest affected)
4.1.0.4 release
Hardware: x86-64 (AMD64) Windows (All)
: medium major
Assignee: Not Assigned
URL:
Whiteboard: (target:4.4.0)
Keywords:
Depends on:
Blocks:
 
Reported: 2014-03-24 12:05 UTC by Roman Polach
Modified: 2015-01-11 19:40 UTC (History)
4 users (show)

See Also:
Crash report or crash signature:


Attachments
A test document with the macro eating lot of memory in 4.1.0 (10.98 KB, application/vnd.oasis.opendocument.spreadsheet)
2014-03-24 12:05 UTC, Roman Polach
Details
Code of the macro found in previous attachment (691 bytes, text/plain)
2014-08-04 15:32 UTC, Roman Polach
Details
some numbers for 3 versions of LO (2.37 KB, text/plain)
2014-08-05 14:15 UTC, Terrence Enger
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Roman Polach 2014-03-24 12:05:34 UTC
Created attachment 96285 [details]
A test document with the macro eating lot of memory in 4.1.0

Problem description:
Attached document contains a basic macro sub (with button for launching it) and an empty sheet.
The macro creates 77 new sheets and then moves 66 ranges (not in newly created sheets but in the
original sheet) and finally displays a message.
In LibreOffice 4.0.6 (and probably all earlier versions) macro eats 521M RAM + 553M swap.
In LibreOffice 4.1.0 (and probably all later versions) macro eats 1300M RAM + 1277M swap.

Note 1:
This is not only symbolic problem. More complicated macro I use (containing similar pattern)
causes crashes of LibreOffice when document is closed after macro run (in 4.1.x) or crashes
directly while running this macro (4.2.x) while everything is ok e.g. in LibreOffice 3.5.5. Crashes are ugly and partly random (due to memory usage of other applications).

Note 2:
Strange thing is that when the code creating sheets is removed from the macro, things go ok.
The same is when moveRanges are removed form the code. It only causes problems when moveRanges
follow creating sheets (while moveRanges operates on different sheets than those created by macro.)

Note 3:
The numbers of RAM and swap above are values measured in VirtualBox, Windows XP, 1699M virtual RAM.
Before the macro is run but after the document is open, LibreOffice takes about 120-130M RAM and 100M swap.

Note 4:
The problem is also reproducible outside VirtualBox: In 4.0.6 it takes 451M RAM and in 4.1.0 1263M RAM. Crashes described above are seen in running outside VirtualBox.
Comment 1 Roman Polach 2014-05-26 12:34:54 UTC
This problem is still present in 4.3.0beta1.
Memory lost: 1300 MB RAM + 1277 MB swap!
Comment 2 Roman Polach 2014-06-09 15:26:16 UTC
* Still the big problem in 4.3.0-beta2
* When (attached example) document closed after macro run,
  memory is released slowly...
Comment 3 Roman Polach 2014-06-11 09:04:39 UTC
Isn't it a regression?
Because the real problem starts with 4.1.0 and previous versions were ok?
Comment 4 Roman Polach 2014-06-23 10:27:18 UTC
Still the same problem in 4.2.5 final!
Does anybody care?

I also think in 4.2.5 closing the document
after macro run takes even longer than before:
about 15 seconds.
Comment 5 Roman Polach 2014-07-30 09:59:34 UTC
Again, the same problem in 4.3.0 final.
Document eats 1G RAM and 1.5G swap.
It takes about 15 seconds to close the document
(slow freeing the memory).

Last usable LibreOffice version was 3.5.5.
Still nobody cares?
Comment 6 Roman Polach 2014-08-04 10:52:56 UTC
Not only in 4.3.0 final,
the bug is present also in 4.1.6 final and 4.2.6 final:
4.1.6: final workbook state: 1.3G RAM used, 1.28G swap used,
4.2.6: final workbook state: 0.6G RAM used, 1.6G swap used,
15 seconds to close the final document (slow release of memory?)
Comment 7 Kohei Yoshida 2014-08-04 12:19:22 UTC
Don't add lists to CC!
Comment 8 Roman Polach 2014-08-04 15:32:25 UTC
Created attachment 104003 [details]
Code of the macro found in previous attachment
Comment 9 Terrence Enger 2014-08-05 14:15:22 UTC
Created attachment 104072 [details]
some numbers for 3 versions of LO

I have been playing around with various versions of LibreOffice.  In
each version I opened the attachment, ran the macro, closed the file,
opened the file, ran the macro, closed the file.  I ran `top` for
soffice.bin at interesting points.

The virtual memory usage reported is merely a gross number.  I do not
know how much of that number is free space on the heap, so I have no
opinion about whether or not the numbers are evidence of a "leak".

Comparing bibisect-43all latest to oldest:
  - memory usage is much higher in latest
  - closing the file shows a decrease in usage in latest
  - second open/run/close made a net increase of 568MB in latest
  - CPU usage is about 4 times greater in latest.

From a debug build of master fetched 2014-07-12:
  - second and third "open file" did not increase virtual memory usage
    at all; indeed the second "open file" saw a slight decrease.
  - picking out virtual memory usage after successive "close file":
    2679, 3484, 3827.
  - the time to add successive sheets grows non-linearly
  - each moveRange in first execution of the macro increased virtual
    memory usage by about 49.8.  Wow, even in a debug build.

The numbers are in the attachment.  Columns VIRT and CPU are taken
from `top` for soffice.bin at the end of the described operation.  Be
warned that I scribbled these figures down from the screen and then
typed them in, so there is more than a small possibility of errur.

All my observations are on debian-wheezy 64-bit.
Comment 10 Roman Polach 2014-08-29 10:18:59 UTC
This problem also affect newest 4.3.1.2 (final).
Total RAM+swap used after macro run is 2.45G.
Comment 11 Roman Polach 2014-09-26 17:31:24 UTC
The problem is still present in 4.3.2.2 (final).
Total RAM+swap used after macro run is 2.05G.
Comment 12 Roman Polach 2014-10-30 14:31:50 UTC
Interesting things happen with 4.2.7 (.2-final)
although the problem is not gone.

In VirtualBox box with 1688MB RAM:
before macro run soffice process takes about 150M RAM + 120M swap.
During the macro run it grows to 1.4G RAM + 1.4G swap
but then it starts to drop and it stops at 520M RAM + 1.62G swap.

But the closing documents still lasts ages (almost a minute).
Comment 13 Roman Polach 2014-10-30 14:50:49 UTC
Almost the same as for 4.2.7 (.2-final)
applies to 4.3.3 (.2-final)
...only the final state is cca 810M RAM + 1.5G swap.
Comment 14 Julien Nabet 2015-01-04 09:10:57 UTC
On pc Debian x86-64 with 4.3.3 LO Debian package (i5, 6GB), memory consumption increased until 43%.
On 4.3 sources with debug symbols, until more than 60%
On 4.4 sources with debug symbols, until a little less than 6% (and macro was faster).

Roman: for the test, could you give a try to pre release 4.4.0 (see http://www.libreoffice.org/download/pre-releases/)?
Comment 15 Roman Polach 2015-01-05 09:08:31 UTC
Seems to be really good in 4.4 rc1.
From initial 166M RAM + 140M swap
after macro ends, there is only
207M RAM + 181M swap used. Also
the speed seems to be much higher.
Comment 16 Julien Nabet 2015-01-05 09:47:43 UTC
Thank you for your feedback, let's put this one to WFM (from 4.4.0).
Comment 17 Roman Polach 2015-01-05 16:21:20 UTC
Probably FIXED is more appropriate than WORKSFORME?
Comment 18 Julien Nabet 2015-01-05 16:27:15 UTC
FIXED is appropriate when there's a specific fix known. It's not the case here.
Comment 19 Terrence Enger 2015-01-11 19:40:39 UTC
My experiments confirm WORKSFORME.  Here are sults from Linux daily
build of 2015-01-09:

                              VIRT(MB)    CPU(sec)
    open file                 1185        0:02.26
    run macro                 1271        1:47.78
    close file                1270        1:48.85
    open file                 1270        1:50.14
    run macro                 1272        3:36.29
    close file                1271        3:37.34