Bug 7869

I can reproduce this with libX11-1.0.0 (FC5) and 1.0.3 (FC6test)
easy for both scim (with /FrontEnd/X11/Dynamic=false) and gcin. 
xorg-x11-6.8.2 (FC4) seems to be okay.

Note this is very easy to reproduce in gedit as first described by
krisna on scim-devel:

$ scim -d
$ XMODIFIERS=@im=SCIM GTK_IM_MODULE=xim gedit

then press ctrl-N and try to type something.

Just to emphasize that is issue is not specific to scim,
it can also easily be reproduced with kinput2 and gedit:

$ kinput2 &
$ XMODIFIERS=@im=kinput2 GTK_IM_MODULE=xim gedit

then activate kinput2 with Shift-Space and press Ctrl-n
at which point gedit goes into a hard loop.

SlickEdit also seems to suffer from this bug.  Especially when locale is set to
Thai it reproduces almost instantaneously.  Reproduced on FC5, Ubuntu 6.06, and
OpenSUSE 10.1/SLED 10.1.

Created attachment 7606 [details] [review]
A patch

This patch fix0rs the problem by immediately sending a sync reply if an event
arrives on an unfocused IC.

Sorry about the phenomenal bug spam, guys.  Adding xorg-team@ to the QA contact so bugs don't get lost in future.

(In reply to comment #4)
> Created an attachment (id=7606) [details]
> A patch
> 
> This patch fix0rs the problem by immediately sending a sync reply if an event
> arrives on an unfocused IC.

Very big thanks for trying to fix it. Although it appears to alleviate the problem, at least a little, it sadly does not fix it: eg xterm can still easily be made to lose key events by ALT-TAB switching between apps. And the worst for me is trying to use a chat program in flash for Firefox. I cannot ever type anything at all, and the patch makes no different. Have to quit skim and Firefox and then restart. However, since I wanted to chat in Japanese, it's not really useful without skim...

I have the same problem with my brand new Xubuntu 7.10 (gutsy gibbon).
* im-switch 1.14
* xfce 4.4.1
* scim 1.4.7-1ubuntu2
* scim-gtk2-immodule 1.4.7-1ubuntu2
etc. 

I very often run into the problem when i open a dialog box, but also on other occasions. Once, when i tried to solve the problem by choosing Exit from the right-click menu on the scim taskbox icon, 
* the two xfce panels disappeared
* my 4 desktops were first still there, but later they melted into one
* all open terminals (bash) disappeared. 

I got my panels back by deleting all xfce session files in the cache and then restarting. 

But the blocked keyboard problem remains. Several times, i could make it work again by closing scim and keeping the application open. 

I would be really very happy if someone found a solution!
Christoph

A similar problem has been reported for openSUSE 10.3, see

http://bugzilla.novell.com/show_bug.cgi?id=335555

I cannot reproduce the problem using the procedures described by
Jens Petersen in comment #1 and comment #2.

(In reply to comment #9)
> I cannot reproduce the problem using the procedures described by
> Jens Petersen in comment #1 and comment #2.

Mike Fabian:
Can you reproduce it by opening new windows from running applications (e.g., a new mail compose window from thunderbird), and then hitting a key before the new window is completely loaded? This worked always for me before I installed the patch Arne Goetje recommended on https://bugs.launchpad.net/ubuntu/+source/scim/+bug/66104
Since then, my keyboard has not yet frozen again, but it seems the patch is not a real solution according to the people who seem to know the ropes. 

(In reply to comment #9)
> I cannot reproduce the problem using the procedures described by
> Jens Petersen in comment #1 and comment #2.
> 

It was a gedit bug, see http://bugzilla.gnome.org/show_bug.cgi?id=351476
And it was fixed in gedit.

But I suspect that this bug is also caused by same reason. 
For any reason, if the XFilterEvent() function is called twice with same event,  the XIM server and clients will not work properly. I think the test is need to check whether xim client call the XFilterEvent() function twice or not.

Now, I can clarify this problem.
This is caused by wrong xim function calling sequence.

XIM system works as follows:

1. The client calls XNextEvent() and receive a new key event.
2. The client calls XFilterEvent() with the newly received key event and 
   it's focused ic client window.
3. In the XFilterEvent() function, Xlib filters the key event and send it to 
   XIM server, so the client does not process this key event any more.
   Xlib mark the flag of the ic associated with the argument window of 
   XFilterEvent() to wait for XIM_SYNC_REPLY and put back the key event 
   to the event queue with XPutBackEvent().
4. As the XFilterEvent() function filters the key event, the clients has no 
   event to process and should call XNextEvent() again. Now the clients gets 
   the key event.
5. The client calls XFilterEvent() with the key event and focused ic client 
   window. XFilterEvent() function does not filters the event this time, but 
   it finds ic associated with the argument window, send XIM_SYNC_REPLY to 
   the ic in xim server.
6. The key event does not filtered, therefore the client process the key 
   event. The client may call X{mb,wc}LookupString to get the string.

According to the above scenario, XFilterEvent() function should be called twice with same ic client window, or the xim server can not receive XIM_SYNC_REPLY.

If the client change the focus of XIC between XNextEvent() and XFilterEvent() 
(between 4 and 5) before sending XIM_SYNC_REPLY, the xim server will be 
blocked to wait XIM_SYNC_REPLY.

So this problem happens with synchronous XIM protocol. If the xim server is 
running static flow mode, this problem happens easily, because in static flow 
mode, every protocol is synchronous. Some people use Dynamic flow mode to work 
arround this problem. But they still may have this problem, because there are 
several synchronous protocol in dynamic flow mode. 

So the client should not change the focus between XNextEvent() and 
XFilterEvent(), if they pass the ic client window to XFilterEvent().

So I made a small program to reproduce this problem easily.
The program will change the focus ic when the 'tab' key is pressed between 
XNextEvent() and XFilterEvent().

I will attach the source.

Created attachment 13363 [details]
sample program to reproduce this bug

compile:
gcc test.c -o test -lX11

To reproduce the problem, follow this instruction:
Set the locale properly to use XIM.
Set the XMODIFIERS properly. (ex export XMODIFIERS=@im=SCIM or export XMODIFIERS=@im=nabi)
If you use scim, change the /FrontEnd/X11/Dynamic to false.

Run the program.
Change the input mode to input some multibyte string.
Press some keys to check whether the xim works or not. This program will print some messages to show the input.
Close the program.

Run the program again.
Press 'tab'.
This time, the program will not show any key press messages.

Created attachment 13398 [details] [review]
a patch for xim sync problem

I made a new patch for this problem.

I think that the sync flag should not have different value per Xic, but it should have different value per Xim. If the Xic focus changing can happen in the middle of sync protocol, it's more reasonable for Xim to have the sync flag. IMdkit also have sync flag per clients not per ic.

So I changed the FABLICATED() and NEED_SYNC_REPLY() macros to set the value of Xim instead of Xic and fixed typo FABLICATED -> FABRICATED.

Created attachment 13405 [details] [review]
a patch for xim sync problem

Sorry, I omitted a version number.
I updated above patch for libX11-1.0.3 
I tested this patch on Debian unstable, libX11-1.0.3.tar.gz (debian package libx11-6)

Choe Hwanjin, can users like me help here by testing your patch? What should we do? 

(In reply to comment #16)
> Choe Hwanjin, can users like me help here by testing your patch? What should we
> do? 
> 

Yes, it may be helpful.

(In reply to comment #17)
> (In reply to comment #16)
> > Choe Hwanjin, can users like me help here by testing your patch? What should we
> > do?  
> Yes, it may be helpful.

But what do I have to do? How do I test the patch?

Information: I have also installed a workaround from http://ppa.launchpad.net/arnegoetje/ubuntu (see https://bugs.launchpad.net/ubuntu/+source/scim/+bug/66104). I hope the two patches don't eat my linux :-)

(In reply to comment #18)
> (In reply to comment #17)
> > (In reply to comment #16)
> > > Choe Hwanjin, can users like me help here by testing your patch? What should we
> > > do?  
> > Yes, it may be helpful.
> 
> But what do I have to do? How do I test the patch?
> 
> Information: I have also installed a workaround from
> http://ppa.launchpad.net/arnegoetje/ubuntu (see
> https://bugs.launchpad.net/ubuntu/+source/scim/+bug/66104). I hope the two
> patches don't eat my linux :-)
> 

I think you should not apply the both patches. I recommend you to apply the second patch ( a patch for xim sync problem) only.

You can test it with the sample program that I uploaded before, or follow other reproducing instructions.

I also made patch that solves it on XIM server side.
You can see it on the SCIM mailing list:
http://sourceforge.net/mailarchive/forum.php?thread_name=604f48110712260753o15834321ta3c62d7e8c65f91e%40mail.gmail.com&forum_name=scim-devel

So Choe, so should we apply your last patch and revert 1a1a42a3ca1dfaf42f1094936b71c140fc030fcb from Søren?  (see http://cgit.freedesktop.org/xorg/lib/libX11/diff/?id=1a1a42a3ca1dfaf42f1094936b71c140fc030fcb)

Søren, ideas?

(In reply to comment #20)
> So Choe, so should we apply your last patch and revert
> 1a1a42a3ca1dfaf42f1094936b71c140fc030fcb from Søren?  (see
> http://cgit.freedesktop.org/xorg/lib/libX11/diff/?id=1a1a42a3ca1dfaf42f1094936b71c140fc030fcb)
> 

These 2 patchs are conflict with each other.
Because my patch removes the "NEED_SYNC_REPLY" flag from IC and, Søren's patch uses that flag.
I recommend applying my patch.

I do remember that when I wrote my patch I considered it a horrible hack, mostly because XIM one big horrible hack by itself. I haven't reviewed the new patch in detail, but it doesn't look wrong to me.

So, feel free to revert my patch and apply the new one.

updating my email address since I changed my affiliations.

Created attachment 21093 [details] [review]
A patch vs. Xlib 1.1.4

This patch is a combination of revising Søren Sandmann Pedersen patch as found in Xlib 1.1.4, and applies Choe Hwanjin's patch. I've tested this patch with OpenSUSE 11.1 beta and release candidates, since both OO.org and KDE4 still show problems with Søren's patch.

(In reply to comment #24)
> Created an attachment (id=21093) [details]
> A patch vs. Xlib 1.1.4
> 
> This patch is a combination of revising Søren Sandmann Pedersen patch as found
> in Xlib 1.1.4, and applies Choe Hwanjin's patch. I've tested this patch with
> OpenSUSE 11.1 beta and release candidates, since both OO.org and KDE4 still
> show problems with Søren's patch.
> 

Do you have a build of your libX11 package? I tried to create a build of libX11 + your patch in the openSUSE build service but my applications don't get any input unless I kill scim.

I did have to modify the patch to leave in the "FABLICATED" symbols since applications were crashing when trying to find the IS_FABLICATED symbol, otherwise. Do the applications linking against libX11 need a recompile against libx11 with this patch?

openSUSE bug:
https://bugzilla.novell.com/show_bug.cgi?id=335555
my libX11:
https://build.opensuse.org/package/show?package=xorg-x11-libX11&project=home%3Aphilipsb%3Abranches%3Ahome%3Asndirsch%3AFactory

Thanks, Brandon

openSUSE pulled in a patch that seems to fix this for me:

 
 https://bugzilla.novell.com/show_bug.cgi?id=434865#c49

Cheers, Brandon

Can you send the patch (looks like the one from SuSE is newer) to xorg-devel 
for review?

commit 738f7b867341c1da87ee667d48815822715c3e75
Author: Choe Hwanjin <choe.hwanjin@gmail.com>
Date:   Thu Oct 13 07:58:02 2011 +0900

    XIM: Make Xim handle NEED_SYNC_REPLY flag
    
    NEED_SYNC_REPLY flag should be in Xim not in Xic.
    Because the focused Xic can be changed before sending sync reply.
    After focused Xic changed, the new Xic doesn't have NEED_SYNC_REPLY
    flag enabled, so libX11 doesn't send XIM_SYNC_REPLY packet.
    
    This patch adds sync reply flag to Xim and removes sync reply
    from Xic.
    
    Fixes: https://bugs.freedesktop.org/show_bug.cgi?id=7869
    
    Signed-off-by: Choe Hwanjin <choe.hwanjin@gmail.com>
    Reviewed-by: Jeremy Huddleston <jeremyhu@apple.com>