Bug 44397 - daemon crashes (segmentation fault) when switching the sink of Gnome sounds (module-remap-sink)
Summary: daemon crashes (segmentation fault) when switching the sink of Gnome sounds (...
Alias: None
Product: PulseAudio
Classification: Unclassified
Component: daemon (show other bugs)
Version: unspecified
Hardware: x86-64 (AMD64) Linux (All)
: medium major
Assignee: beroal
QA Contact: pulseaudio-bugs
Depends on:
Blocks: 45812
  Show dependency treegraph
Reported: 2012-01-02 12:28 UTC by beroal
Modified: 2012-03-30 03:21 UTC (History)
3 users (show)

See Also:
i915 platform:
i915 features:

the console log (10.22 KB, text/plain)
2012-01-02 12:28 UTC, beroal
daemon.conf (2.30 KB, text/plain)
2012-01-21 23:28 UTC, beroal
default.pa (6.11 KB, text/plain)
2012-01-21 23:31 UTC, beroal
Preliminary patch (1.65 KB, patch)
2012-03-23 05:10 UTC, David Henningsson
Details | Splinter Review
daemon.conf (2.27 KB, text/plain)
2012-03-24 04:24 UTC, beroal
default.pa (6.11 KB, text/plain)
2012-03-24 04:25 UTC, beroal
the console log, with the preliminary patch (45.07 KB, text/plain)
2012-03-24 04:25 UTC, beroal
client.conf (1.26 KB, text/plain)
2012-03-26 07:00 UTC, beroal
pacmd ls (38.99 KB, text/plain)
2012-03-26 09:26 UTC, beroal
Patch to catch loops of length > 1 (1.73 KB, patch)
2012-03-26 20:19 UTC, Arun Raghavan
Details | Splinter Review
patch.beroal.0 (1.10 KB, patch)
2012-03-27 01:44 UTC, beroal
Details | Splinter Review

Description beroal 2012-01-02 12:28:07 UTC
Created attachment 55057 [details]
the console log

Steps to reproduce:
0. Run "pulseaudio -vvvvv".
1. Open context menu of "gnome-sound-applet" in the Gnome panel, "Sound Preferences".
2. Open the tab "Output".
3. Change a device for sound output several times.
4. "pulseaudio" crashes with the log given in an attachment.

The version of "pulseaudio" is the latest git version.
Comment 1 beroal 2012-01-02 12:40:16 UTC
The bug is reproducible.
Comment 2 beroal 2012-01-02 12:54:04 UTC
A stack trace:
#0  0x00007ffff7b92c15 in pa_sink_set_rtpoll () from /usr/lib/libpulsecore-1.98.so
#1  0x00007fffe8cf6e2e in ?? () from /usr/lib/pulse-1.98/modules/module-remap-sink.so
#2  0x00007ffff7b98539 in pa_sink_attach_within_thread () from /usr/lib/libpulsecore-1.98.so
The last line is repeated many times.
Comment 3 Tanu Kaskinen 2012-01-03 10:14:38 UTC
Could you install the debug symbols and send the stack trace again after that? On Debian based distributions the debug symbols are in the package "pulseaudio-dbg", and on Fedora they are in "pulseaudio-debug". I don't know about other distributions, but they should be similar.
Comment 4 beroal 2012-01-04 11:26:15 UTC
A stack trace:
#0  0x00007ffff7b727de in pa_msgobject_check_type (type_id=0x612040 "pa_object") at pulsecore/msgobject.c:29
#1  0x00007ffff7b8eabe in pa_sink_check_type (type_id=<optimized out>) at pulsecore/sink.c:59
#2  0x00007ffff7b9131d in pa_object_cast (o=0x76ce90) at ./pulsecore/object.h:64
#3  pa_sink_assert_ref (o=0x76ce90) at ./pulsecore/sink.h:289
#4  pa_sink_set_rtpoll (s=<optimized out>, p=0x0) at pulsecore/sink.c:777
#5  0x00007fffe9be260b in sink_input_attach_cb (i=0x775dc0) at modules/module-remap-sink.c:240
#6  0x00007ffff7b98d99 in pa_sink_attach_within_thread (s=0x777430) at pulsecore/sink.c:2861
The lines #5, #6 are repeated many times.
Comment 5 beroal 2012-01-19 21:02:05 UTC
Often a segmentation fault with the same stacktrace occurs when starting playing in "mplayer". Verified in the latest git version.
Comment 6 Tanu Kaskinen 2012-01-20 09:00:49 UTC
I might try reproducing this tomorrow. Could you share your pulseaudio configuration? Apparently you've done some modifications, like loading the remap sink module.
Comment 7 beroal 2012-01-21 23:28:32 UTC
Created attachment 55942 [details]
Comment 8 beroal 2012-01-21 23:31:02 UTC
Created attachment 55943 [details]
Comment 9 Tanu Kaskinen 2012-02-09 00:09:54 UTC
I still haven't tried reproducing this... Anyway, I'd like to investigate this before 2.0, so I'll add this to the list of blocker bugs.
Comment 10 David Henningsson 2012-03-23 05:10:20 UTC
Created attachment 58920 [details] [review]
Preliminary patch


It's difficult to know exactly what has happened here, but if those lines appear many times, it sounds like we have a cycle in the filter sink graph, somehow the filter is set to output to itself.

I've created a patch that should prevent that from happening. Could you try the latest git version, apply the attached patch and see if it prevents your problem? Thanks!
Comment 11 beroal 2012-03-24 04:24:01 UTC
Created attachment 58966 [details]
Comment 12 beroal 2012-03-24 04:25:03 UTC
Created attachment 58967 [details]
Comment 13 beroal 2012-03-24 04:25:38 UTC
Created attachment 58968 [details]
the console log, with the preliminary patch

PulseAudio does not crash with your patch. Instead, a running client ("mplayer") freezes after I did the item 4 in the bug description. Afterwards, neither the client nor PulseAudio may be terminated by the signal TERM. I attached a new console log, it contains "that would create a cycle." I also updated the configuration files.
Comment 14 beroal 2012-03-24 04:27:54 UTC
Clients use a PulseAudio server via a unix socket.
Comment 15 David Henningsson 2012-03-26 03:49:19 UTC
Thanks. Well, that at least confirms that some kind of cycle was about to be created and I assume it was avoided.

For what happens instead (freeze), I wonder if that is not a separate bug. You have a lot of "Pool full" errors, which is usually a bad sign. Do the "pool full" errors start to appear before you start switching outputs?

Then, that the gnome volume control wants to make cycles in the graph, that's also a separate bug I guess...
Comment 16 beroal 2012-03-26 05:05:27 UTC
(In reply to comment #15)
> For what happens instead (freeze), I wonder if that is not a separate bug. You
I am not sure. Without your patch, I get not a freeze, but "Segmentation fault" when I send the TERM signal to PulseAudio. Is this "Segmentation fault" considered a bug, should I report it?

> have a lot of "Pool full" errors, which is usually a bad sign. Do the "pool
> full" errors start to appear before you start switching outputs?
Yeah, I get the message "Pool full" even without your patch when playing. Is it considered a bug, should I report it?
Comment 17 David Henningsson 2012-03-26 05:16:44 UTC
Not sure I follow. 

If you get a segfault only when my patch is NOT present, we can assume my patch fixes that segfault.

If you get a segfault when my patch IS present, could you post a stack trace for that segfault in this bug, please?

For the pool full errors, I haven't looked into what that means in more detail. I just get a feeling it means something is wrong...
Comment 18 beroal 2012-03-26 07:00:11 UTC
Created attachment 59067 [details]
Comment 19 beroal 2012-03-26 07:22:17 UTC
If I send the TERM signal:
- With your patch, PulseAudio ignores it. It is not frozen, it throws log messages.
- Without your patch, PulseAudio yields "Segmentation fault".

Also, with your patch, PulseAudio does not respond to clients. (My previous report is incorrect.)
$ mplayer -ao pulse 14\ -\ It\ Was\ a\ Very\ Good\ Year\ -\ Frank\ Sinatra.mp3
MPlayer SVN-r34426-4.6.2 (C) 2000-2011 MPlayer Team
AO: [pulse] Init failed: Timeout
Failed to initialize audio driver 'pulse'
Could not open/initialize audio device -> no sound.
Audio: no sound
Video: no video

Exiting... (End of file)
Comment 20 beroal 2012-03-26 08:06:54 UTC
With the patch:
If I clear the state of PulseAudio (in ~/.pulse), it responds to clients. When I remap a client (mplayer, audacious) with pavucontrol to another sink, PulseAudio stops responding.
Comment 21 David Henningsson 2012-03-26 08:09:35 UTC
Is PulseAudio taking up 100% CPU in that state, and if so, can you check with gdb where that is (full stack trace)?

Comment 22 beroal 2012-03-26 08:58:26 UTC
(In reply to comment #21)
> Is PulseAudio taking up 100% CPU in that state, and if so, can you check with
> gdb where that is (full stack trace)?
Yes, 100% CPU.
#0  find_filter_sink_input (s=0x7026f0, target=<optimized out>) at pulsecore/sink-input.c:1387
#1  pa_sink_input_may_move_to (i=0x741a90, dest=0x7026f0) at pulsecore/sink-input.c:1409
#2  0x00007ffff7b8d795 in pa_sink_input_move_to (i=0x741a90, dest=0x7026f0, save=true) at pulsecore/sink-input.c:1727
#3  0x00007ffff1cf64a1 in command_move_stream (pd=<optimized out>, command=67, tag=61, t=<optimized out>, userdata=0x7101c0) at pulsecore/protocol-native.c:4493
#4  0x00007ffff76d4e45 in pa_pdispatch_run (pd=0x7268c0, packet=<optimized out>, creds=0x70dc20, userdata=0x7101c0) at pulsecore/pdispatch.c:336
#5  0x00007ffff1cfe921 in pstream_packet_callback (p=0x70dad0, packet=0x7311b0, creds=0x70dc20, userdata=0x7101c0) at pulsecore/protocol-native.c:4747
#6  0x00007ffff76d8b1d in do_read (p=0x70dad0) at pulsecore/pstream.c:812
#7  do_something (p=0x70dad0) at pulsecore/pstream.c:180
#8  0x00007ffff76d9600 in io_callback (io=<optimized out>, userdata=<optimized out>) at pulsecore/pstream.c:209
#9  0x00007ffff76c736c in callback (m=<optimized out>, e=<optimized out>, fd=<optimized out>, f=<optimized out>, userdata=0x642e20) at pulsecore/iochannel.c:160
#10 0x00007ffff7920c87 in dispatch_pollfds (m=0x63d070) at pulse/mainloop.c:677
#11 pa_mainloop_dispatch (m=0x63d070) at pulse/mainloop.c:927
#12 0x00007ffff7920dde in pa_mainloop_iterate (m=0x63d070, block=<optimized out>, retval=0x7fffffffe578) at pulse/mainloop.c:958
#13 0x00007ffff7920e2c in pa_mainloop_run (m=0x63d070, retval=0x7fffffffe578) at pulse/mainloop.c:973
#14 0x000000000040c0cd in main (argc=<optimized out>, argv=<optimized out>) at daemon/main.c:1135
Comment 23 Arun Raghavan 2012-03-26 09:07:38 UTC
Could you dump the output of 'pacmd ls' from /before/ you start a move/playback that breaks everything?
Comment 24 beroal 2012-03-26 09:26:50 UTC
Created attachment 59079 [details]
pacmd ls
Comment 25 beroal 2012-03-26 11:34:54 UTC
@David: Maybe, you should replace "s = s->input_to_master->origin_sink;" with "s = s->input_to_master->sink;" in your patch?
Comment 26 beroal 2012-03-26 14:56:37 UTC
The console log:
D: [pulseaudio] module-stream-restore.c: Client gnome-control-center changes entry sink-input-by-media-role:filter.
I.e. "gnome-control-center" connects all filter sink inputs to the sink selected in "Sound Preferences". Do I understand it correctly? If so then I will report it to the Gnome bugtracker.
Comment 27 Arun Raghavan 2012-03-26 20:19:05 UTC
Created attachment 59098 [details] [review]
Patch to catch loops of length > 1

Roman, here's another, independent, patch that might fix this. Could you give this a go? (and thank you for your patience in pinning this down!)

My suspicion is that this is a loop, as David surmises, but the loop length is >1 which David's patch won't catch. This patch isn't bullet proof -- it would fail in a loop with more than one type of filter, but I don't see that this would happen in your configuration. If this works, we can convert this into proper loop detection, independent of the module name logic.
Comment 28 David Henningsson 2012-03-26 23:48:42 UTC
@Roman, comment #25, well spotted! Yes, I think this should change. Could you try your change and see if it resolves your problem?

@Arun, the patch is indeed meant to catch cycles even if the length > 1, hence the while loop. However maybe the while loop should itself be more resilient against existing loops.
Comment 29 Arun Raghavan 2012-03-26 23:51:41 UTC
Comment on attachment 59098 [details] [review]
Patch to catch loops of length > 1

David, indeed, I was being blind.
Comment 30 beroal 2012-03-27 01:44:24 UTC
Created attachment 59107 [details] [review]

This patch does not lead to freezes.
Comment 31 David Henningsson 2012-03-30 03:21:00 UTC
Ok, thanks for the feedback! I have now committed a patch for both sinks and sources, which should be in the 1.99.2 release. 
Please reopen this bug if this should not be fixed in that release. Thank you!

Use of freedesktop.org services, including Bugzilla, is subject to our Code of Conduct. How we collect and use information is described in our Privacy Policy.