Bug 64690 - EDITING: Process hang on find/replace in Basic code involving "\&"
Summary: EDITING: Process hang on find/replace in Basic code involving "\&"
Status: RESOLVED FIXED
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Calc (show other bugs)
Version:
(earliest affected)
Inherited From OOo
Hardware: All All
: medium critical
Assignee: Andreas Heinisch
URL:
Whiteboard: BSA target:7.0.0 target:6.4.5 target:...
Keywords: haveBacktrace
Depends on:
Blocks: Find-Search BASIC-IDE
  Show dependency treegraph
 
Reported: 2013-05-16 22:42 UTC by Jim Avera
Modified: 2021-01-28 15:08 UTC (History)
9 users (show)

See Also:
Crash report or crash signature:


Attachments
Demo (8.49 KB, application/vnd.oasis.opendocument.spreadsheet)
2013-05-16 22:46 UTC, Jim Avera
Details
gdb backtrace, produced from Control-C at the hang (16.60 KB, text/x-log)
2013-05-16 22:54 UTC, Jim Avera
Details
Flamegraph (36.33 KB, application/x-bzip)
2020-01-26 12:52 UTC, Julien Nabet
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Jim Avera 2013-05-16 22:42:57 UTC
Certain "Find & Replace" operations in Basic-language macro code cause the process to hang (must be killed externally).

It seems to happen when Regular Expressions are enabled and the replacement text is "something\&".   Note that & is supposed to be a magic replacement token when using Regular Expressions, but not, presumably, with a backslash.

Regardless, the process should never hang permanently.

STEPS TO REPRODUCE:
1. Open the attached demo spreadsheet (ReplaceBug.ods)
   It is NOT necessary to enable macro execution.

2. Tools->Macros->Organize Macros-LibreOffice Basic
   Navigate to the "BugDemo" module in the document
   (ReplaceBug.ods was the name used to attach it to this bug)
   Click "Edit" to view the code.

3. Follow the instructions shown in the comments, which are as follows:

'abc
'def

'
'  1. Select the two comment lines above ('abc and 'def)
'  2. Edit->Find and Replace (or Control-H)
'  3. Click "More Options" and 
'       check "Selection Only"
'       check "Regular Expressions"
'  4. In the "Find" box, enter "."
'  5. In the "Replace" box, enter "ABC\&"
'  6. Click ReplaceAll
'  (process hangs)

NOTE: The hang does not occur when editing cell content (rather than Basic source code). or if the replacement text is "\&" by itself (rather than "ABC\&").
   
Operating System: Ubuntu
Version: 4.0.3.3 release
Comment 1 Jim Avera 2013-05-16 22:46:53 UTC
Created attachment 79445 [details]
Demo
Comment 2 Jim Avera 2013-05-16 22:53:39 UTC
Attaching a gdb backtrace produced by running 
  localc --backtrace ReplaceBug.ods
and running the procedure to cause the hang, then typing Control-C in the terminal.
Comment 3 Jim Avera 2013-05-16 22:54:31 UTC
Created attachment 79446 [details]
gdb backtrace, produced from Control-C at the hang
Comment 4 Julien Nabet 2013-05-17 19:30:31 UTC
On pc Debian x86-64 with master sources updated today, I haven't reproduced the problem.

Joren: Do you have 4.0.3? If yes, would you have some time to give it a try?
Comment 5 Jorendc 2013-05-17 22:20:34 UTC
Thanks for reporting.

I can reproduce this behavior using Mac OSX 10.8.3 with Versie 4.0.3.3 .3.3 (Bouw-id: 0eaa50a932c8f2199a615e1eb30f7ac74279539); Dutch UI;

I have to force quit LibreOffice after I hit the Replace All button.

Kind regards,
Joren
Comment 6 Julien Nabet 2013-05-20 17:07:18 UTC
Thank you Joren for having confirmed this one! :-)

Noel: one for you? (perhaps just a matter of cherry-picking 1 or some specific changes from master)
Comment 7 Noel Power 2013-05-21 08:54:49 UTC
(In reply to comment #6)
> Thank you Joren for having confirmed this one! :-)
> 
> Noel: one for you? (perhaps just a matter of cherry-picking 1 or some
> specific changes from master)
I can still reproduce this on master ( although it seems a little random, sometimes can take a number of attempts to reproduce it )

I don't know why this is happening in the Basic editor only ( maybe it is just luck ( or bad luck ) ) I don't see this as anything really specific to basic ( or in this case the IDE ) The backtrace shows this is happening somewhere in the guts of i18n/transliteration stuff ( initiated from the text engine ) I am pretty unfamiliar with this area, don't think I can help with this at the moment. Possibly Eike might have an idea
Comment 8 Julien Nabet 2015-03-19 16:21:38 UTC
Any update with recent LO version?
Indeed, bt must be useless.
Comment 9 Jim Avera 2015-03-19 17:19:26 UTC
The bug is still there in the daily build
LibreOfficeDev 4.4.2.0.0 ce8aba4da2a287c2b51d1d34a290284b2f754dc2
Comment 10 QA Administrators 2015-10-14 19:50:31 UTC Comment hidden (obsolete)
Comment 11 Jim Avera 2015-10-16 02:54:23 UTC
Bug is still there -- reproduced using 5.1.0.0.alpha1+
Happens every time on my system.

Julian (serval2412@yahoo.fr): You marked this bug "NEEDINFO".  Would you please update further, or post a note indicating what info you need?

Thanks.
Comment 12 Julien Nabet 2015-10-16 05:08:12 UTC
Since you reproduced this on master sources updated recently, I don't need more info for the moment.
Comment 13 Joel Madero 2015-10-18 18:23:47 UTC
Moving to NEW as it seems more than confirmed.
Comment 14 QA Administrators 2016-11-08 11:24:09 UTC Comment hidden (obsolete)
Comment 15 Jim Avera 2016-11-08 19:27:31 UTC
Confirmed bug is still there in

Version: 5.1.5.2
Build ID: 7a864d8825610a8c07cfc3bc01dd4fce6a9447e5
CPU Threads: 4; OS Version: Windows 6.2; UI Render: default; 
Locale: en-US (en_US); Calc: group
Comment 16 QA Administrators 2017-11-09 07:44:17 UTC Comment hidden (obsolete)
Comment 17 Jim Avera 2017-11-10 01:07:51 UTC
Bug still there in master.   Locks up immediately, 100% CPU.

Version: 6.0.0.0.alpha0+
Build ID: cfbb8b5090537e79ba70e250ddee86d53facbe15
CPU threads: 8; OS: Linux 4.10; UI render: default; VCL: gtk2; 
TinderBox: Linux-rpm_deb-x86_64@70-TDF, Branch:master, Time: 2017-10-18_22:54:20
Locale: en-US (en_US.UTF-8); Calc: group
Comment 18 Jim Avera 2017-11-10 01:12:00 UTC
Tried LO 3.3 and the bug is not there, works as expected (each character is replaced by "ABC&").

Added "regression" keyword
Comment 19 Alex H. 2017-12-09 15:01:02 UTC
Still occurs in this build, which I built myself:

Version: 6.1.0.0.alpha0+ (x64)
Build ID: 7c77ff5dd2d0573a56f8b59dc9113c23e0ea29c9
CPU threads: 4; OS: Windows 10.0; UI render: default; 
Locale: nl-NL (nl_NL); Calc: CL

I looked at this with a debugger and it seems that the problem is that the "search area", so to speak, doesn't get updated properly to reflect that each character is replaced by five characters, i.e. there is a net change in size.

In more detail, I think that the problem is in TextView::Replace() in libo-core\vcl\source\edit\textview.cxx. I found that it works fine on the first line of the selection, but hangs up on the second line. After it has replaced the first character, it invokes

pTextEngine->Search( aSel, rSearchOptions );

with aSel containing maStartPaM == {7, 5} and maEndPaM == {7, 4}.

Search(...) does a Justify() that swaps start and end, so that the search range now begins at 4. This, however, is part of the 5-character string just inserted. Because the string to find is regexp '.', one of the characters just inserted now gets replaced again by 5 characters. The overall effect is that the string keeps on growing.

I'd like to fix this one, but it's a bit beyond my current capability.
Comment 20 Michael Meeks 2017-12-12 10:32:49 UTC
Xisco - any chance of a bisection if this is indeed a high prio. regression (it is rather an old one).
Comment 21 Xisco Faulí 2017-12-12 10:52:23 UTC
I can also reproduce it on Linux

LibreOffice 3.3.0 
OOO330m19 (Build:6)
tag libreoffice-3.3.0.4

and on Win

LibreOffice 3.3.0 
OOO330m19 (Build:6)
tag libreoffice-3.3.0.4
Comment 22 QA Administrators 2018-12-13 03:50:47 UTC Comment hidden (obsolete)
Comment 23 Jim Avera 2018-12-13 17:25:56 UTC
Bug still there in master.   Do you need a new backtrace?

It is easy to reproduce, so may be bisection is feasible.

Version: 6.3.0.0.alpha0+
Build ID: b15b1a2a90fa4c239ff8a6a33e73ff50ea422abf
CPU threads: 12; OS: Linux 4.15; UI render: default; VCL: gtk3; 
TinderBox: Linux-rpm_deb-x86_64@86-TDF, Branch:master, Time: 2018-11-28_06:21:05
Locale: en-US (en_US.UTF-8); UI-Language: en-US
Calc: threaded
Comment 24 Jim Avera 2018-12-13 17:32:39 UTC
> It is easy to reproduce, so may be bisection is feasible.

Sorry, I realize now the bug was inherited from OO
Comment 25 Julien Nabet 2020-01-26 12:52:22 UTC
Created attachment 157435 [details]
Flamegraph

On pc Debian x86-64 with master sources updated today, I could reproduce the hang.
Here's a Flamegraph.
Comment 26 Julien Nabet 2020-01-26 14:17:35 UTC
It seems there's an infinite loop in this part:
#5  0x00007ffff7f49933 in rtl_uString_newReplaceStrAt(rtl_uString**, rtl_uString*, sal_Int32, sal_Int32, rtl_uString*) (ppThis=0x7fffffff0010, pStr=0x7fffd45b8010, nIndex=4, nCount=0, pNewSubStr=0x55555a6bae00)
    at /home/julien/lo/libreoffice/sal/rtl/strtmpl.cxx:1603
#6  0x00007ffff0544df2 in rtl::OUString::replaceAt(int, int, rtl::OUString const&) const (this=0x55555a4f29e0, index=4, count=0, newStr="ABC\\&") at /home/julien/lo/libreoffice/include/rtl/ustring.hxx:2298
#7  0x00007ffff0765825 in TextNode::InsertText(int, rtl::OUString const&) (this=0x55555a4f29e0, nPos=4, rText="ABC\\&") at /home/julien/lo/libreoffice/vcl/source/edit/textdoc.cxx:267
#8  0x00007ffff0766b45 in TextDoc::InsertText(TextPaM const&, rtl::OUString const&) (this=0x55555a2bc890, rPaM=..., rStr="ABC\\&") at /home/julien/lo/libreoffice/vcl/source/edit/textdoc.cxx:479
#9  0x00007ffff077c4ec in TextEngine::ImpInsertText(TextSelection const&, rtl::OUString const&) (this=0x55555a42af50, rCurSel=..., rStr="ABC\\&") at /home/julien/lo/libreoffice/vcl/source/edit/texteng.cxx:772
#10 0x00007ffff07a8f67 in TextView::Replace(i18nutil::SearchOptions const&, bool, bool) (this=0x55555a50ca20, rSearchOptions=..., bAll=true, bForward=true)
    at /home/julien/lo/libreoffice/vcl/source/edit/textview.cxx:2206
#11 0x00007fffd49d76ca in basctl::ModulWindow::StartSearchAndReplace(SvxSearchItem const&, bool) (this=0x55555a447bd0, rSearchItem=..., bFromStart=false)
    at /home/julien/lo/libreoffice/basctl/source/basicide/baside2.cxx:1254
#12 0x00007fffd49f03d9 in basctl::Shell::ExecuteSearch(SfxRequest&) (this=0x55555a38e3e0, rReq=...) at /home/julien/lo/libreoffice/basctl/source/basicide/basides1.cxx:142
#13 0x00007fffd4a032e9 in SfxStubbasctl_ShellExecuteSearch(SfxShell*, SfxRequest&) (pShell=0x55555a38e3e0, rReq=...) at /home/julien/lo/libreoffice/workdir/SdiTarget/basctl/sdi/basslots.hxx:156
Comment 27 Andreas Heinisch 2020-04-28 17:11:14 UTC
The infinite loop occurs at: https://opengrok.libreoffice.org/xref/core/vcl/source/edit/textview.cxx?r=be53f326#2205
Comment 28 Andreas Heinisch 2020-04-30 08:31:37 UTC
I can even reproduce it without any ampersand (&) in the replace clause. Just search for the dot (.) and replace it with ABC. At the end of the search string, the selection will not be expanded, and therefore an infinite loop occurs at https://opengrok.libreoffice.org/xref/core/vcl/source/edit/textview.cxx?r=be53f326#2205.
Comment 29 Andreas Heinisch 2020-05-04 17:53:56 UTC
https://gerrit.libreoffice.org/c/core/+/93432
Comment 30 Commit Notification 2020-05-04 18:50:22 UTC
Andreas Heinisch committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/commit/e7f3731b8d3e930f85e7df0c0e55bbb1aaea191b

tdf#64690 - Extend selection on find/replace

It will be available in 7.0.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 31 Commit Notification 2020-05-06 11:53:52 UTC
Andreas Heinisch committed a patch related to this issue.
It has been pushed to "libreoffice-6-4":

https://git.libreoffice.org/core/commit/d936d5658aba2ab921fe144f5240f42304e1c5ff

tdf#64690 - Extend selection on find/replace

It will be available in 6.4.5.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 32 Jim Avera 2020-05-06 21:13:35 UTC
Thanks for fixing the hang.

However matching start-of-line using a lone ^ does not work, or not correctly.  It seems impossible to prepend anything to a group of lines, for example, to make them into a block comment by prepending "'  " in Basic code.

Maybe now that someone who knows how Find & Replace works is here, this can be looked at.

The problem is that ^ by itself does not match unless the line contains something after the start-of-line position (i.e. it is not completely empty).

1. Open the ReplaceBug.ods demo and navigate to the Basic code as described in the original comment.

2. Select lines 3-5, i.e., the empty sub declaration ("Sub Main", the empty line, and "End Sub")

3. Control-H.
   Check "Regular expressions" 
   Set Find: to the single character ^  (should match starts of lines)
   Set Replace: to the character '  (the Basic comment-starter char)
   Click Replace All

RESULTS: The apostrophie is prepended to only the non-empty lines.
EXPECTED RESULTS: Should be prepended to every line.
Comment 33 Xisco Faulí 2020-05-06 21:18:36 UTC
Hi Jim,
Could you please report a new bug for it ?
Comment 34 Andreas Heinisch 2020-05-09 10:09:00 UTC
I filled a new bug report for you Jim.
Comment 35 b. 2020-10-03 05:37:06 UTC
new bug is in: 
https://bugs.documentfoundation.org/show_bug.cgi?id=132870
and work on the underlying problem in:
https://bugs.documentfoundation.org/show_bug.cgi?id=135538
Comment 36 Commit Notification 2021-01-28 15:08:11 UTC
Xisco Fauli committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/commit/33656a6c7500d0f799b0e4ed97bda0a9e58a7010

tdf#64690: sw: Add UItest

It will be available in 7.2.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.