Summary: | module-rtp-send floods the network with UDP packets, crippling it severely | ||
---|---|---|---|
Product: | PulseAudio | Reporter: | gt6 |
Component: | modules | Assignee: | pulseaudio-bugs |
Status: | RESOLVED MOVED | QA Contact: | pulseaudio-bugs |
Severity: | major | ||
Priority: | high | CC: | alloftheinternetinone, darwinskernel, dev.rindeal+bugs.freedesktop.org, freedesktop, freedesktop, hans, i-tek, jjardon, jwilliams, lennart, martin, mdeiseroth88, nekohayo, pfeiffer, piotr, pomme_compote-freedesktop, sam |
Version: | unspecified | ||
Hardware: | All | ||
OS: | Linux (All) | ||
Whiteboard: | |||
i915 platform: | i915 features: | ||
Attachments: | pcap - load-module module-rtp-send source=rtp.monitor |
Description
gt6
2012-01-14 04:24:47 UTC
OP also linked to this: https://bugs.launchpad.net/ubuntu/+source/pulseaudio/+bug/411688 Same issue. Using rtp-send causes flooding of my whole local network. Computer using pulseaudio is connected via LAN on a router. I am using Ubuntu 12.04 / 64bit. Other computers are connected via WLAN and they loose connection. Sometimes router crashes. Flooding has a bandwidth of 180 KByte/s. I have seen in google that this problem is existing since at least 2008 !!!!! Why the guys report bugs but noone is caring?? Same thing here, rtp module is an insteresting feature but is just unusable. An update anyone on this problem ? No updates. I'm not aware of anyone working on this. Subscribing in hope of fix. Exactly the same problem here. Running Ubuntu 12.10 64bit with pulseaudio 2.1 I have this problem too. My network becomes completely unusable. I also observe a constant bandwidth of a bit less than 200KB/s. This fits the expected PCM data rate (176KB/s). But this shouldn't congest the network. And indeed the same bandwidth caused by module-tunnel-sink works completely fine. Are our routers at fault or is something fishy with module-rtp-send? Just confirming that I'm seeing that kind of behavior on Fedora 19 too (so not ubuntu-specific), enabling the Multicast/RTP sender will cause constant UDP upload activity towards 224.0.0.56, in my case at ~180 KB/s. I think this explains my other computer (laptop) having trouble associating to the wifi, even though this desktop computer is connected through a wired connection. My router is basically new hardware (Netgear WNR3500L/U/v2) running the Tomato firmware, and I really doubt that anything is at fault on that side. Created attachment 92370 [details] pcap - load-module module-rtp-send source=rtp.monitor Adding a pcap of the traffic I get when loading module-rtp-send: load-module module-rtp-send source=rtp.monitor and doing nothing more than 'pulseaudio -k'. Jump down past packet 343 where the first SDP capture is, for meaningful wireshark display. It looks like there are two issues here: 1) I presume module-rtp-send shouldn't be sending a constant stream of zero-filled packets just by being loaded with no sources assigned to the sink. That seems really inefficient, especially if sound output is infrequent and/or you have a network with many computers and want to be able to support n-to-n source/sink setups (n number of constant all-zero multicast streams...). 2) I also saw the local WiFi network become crippled by the packet storm. I see in the capture a packet rate of 137 per second. I'm running a -n access point running OpenWRT 12.09 and have a solid 180Mbps+ connection, so I looked into a bit further and found: a) multicast on wireless is effectively useless, or perhaps dangerous. Multicast packets always use the lowest possible data rate on WiFi, which causes them to consume all available bandwidth and crowd out all other traffic: https://dev.openwrt.org/ticket/10271 It appears that even running multicast on a wired network where wifi access points are connected can cause a takedown of the wifi networks if the wifi access points aren't filtering multicast. b) there are some proposals for fixing this in a later revision of 802.11 but nothing we can use for a long time. c) there are a bunch of people who have tried to use PA with RTP and some Raspberry Pis to distribute music and they've crashed and burned. The Wiki should probably note that this shouldn't be attempted if there's a WiFi network involved. Still, there is strong demand for this use case, if not RTP necessarily.* d) Multicast on WiFi is so bad that it's more efficient to wrap the UDP in TCP and proxy it at the gateway: http://wiki.openwrt.org/doc/howto/udp_multicast http://www.udpxy.com/index-en.html In my testing, igmp snooping on OpenWRT (and presumably all of its derivatives) doesn't work yet. I wonder if supporting udpxy's protocol in PA would make sense for the WiFi use case. Perhaps smarter would be: * there appears to be an evolving open source standard around the SlimProto http://wiki.slimdevices.com/index.php/SlimProtoTCPProtocol http://wiki.slimdevices.com/index.php/SlimProtoDeveloperGuide https://code.google.com/p/squeezelite/source/browse/slimproto.c http://wiki.slimdevices.com/index.php/Design_of_new-streaming http://wiki.slimdevices.com/index.php/Synchronization and the LMS implementation is said to have excellent synchronization which looks to be handled fairly simply in the perl code. A PA sender and receiver that implemented a subset of squeeze might be neat to have - might even be usable with some of the hardware players. Note: LMS (perl) is GPLv2 but slimsqueeze (c) is GPLv3 so direct code-reuse into PA is probably not possible. This bug is still present in Fedora 23 x86_64 and pulseaudio-7.1-1 My machine was sending ~ 180KB/s to 224.0.0.56 until I unchecked "Enable Multicast/RTP sender" in paprefs. For peace of mind, I ended up adding "unload-module module-rtp-send source=rtp.monitor" into /etc/pulse/default.pa I noticed the same bug in my system ( Ubuntu 14.04.5) while I was trying to set up pulseaudio server in Windows 10 over the local network. In my case network flooding occurs whenever I launch pavucontrol, upload traffics reaches up to 200kb/s and it reduced the local network unusable. When I close it the network attack diminishes. I added `unload-module module-rtp-send` into `/etc/pulse/default.pa` to workaround the bug. I've also noticed this in the most up to data pulse versions from archlinux. I also cannot use a ttl of 1, as the router does not decrease the value when routing into wlan, as it has the same subnet. Neither does destination ip 24.0.0.1 help. From my side I have no chance to change the router config, because the internet is provided by someone else. What works for me though is to stream the music directly to the destination via destination_ip=192.168.178.x I could also add a menu in paprefs that'd also adress the ip directly, however this is more of a struggle because of the old gtk2 (if anyone here is following the mailing list). It would be highly appreciated to fix this bug in the module itself and reduce the payload somehow. I read through the comments, and to me it looks like module-rtp-send doesn't have any flooding bug. The data rate of PCM audio is what it is, and the network hardware isn't always able to handle it. Multicast RTP is not enabled by default. If the user enables it, I don't know what we could possibly do to not "flood" the network. Since this is a common problem for people, there is certainly room for improvement in the UI and documentation so that users don't get so surprised when things don't work. Adding to my previous comment: it could be argued that the bug is that we transmit PCM audio instead of compressed audio. Adding support for compressed audio should help with the network stress. Does anyone volunteer to implement this? I know nothing about RTP, but from a user perspective, I have two questions: 1) Transferring a WAV file over the network goes faster than real time (in terms of the audio contained within). Why should the network struggle with uncompressed RTP audio, then? 2) Why are packets generated when nothing is played? If there is no audio activity in PA, there is no need to send any packets. Apologies if this doesn't make sense, I'm talking purely as an end-user. Sending a wav file means a TCP connection between two computers. TCP connections are the most common type of traffic, and the hardware is optimized accordingly. UDP multicast is different, and apparently network hardware is not necessarily as efficient with processing that protocol. Comment #9 seems to suggest that the wifi specification itself forces multicast to be inefficient, so it's not necessarily just the hardware's fault. Starting from PA version 5.0, module-rtp-send should not generate traffic when nothing is playing (it can still be configured to play silence, but paprefs doesn't configure the module that way). If you observe different behaviour, please file a new bug and attach the output of "pactl list". You are right that this feature needs to be enabled first to make the bug rise. However I do not think that documentating this bug/behavior solves the problem. I do want to use music streaming, but I cannot do this as my router routes the traffic into the WLAN. Lowering the rate is possibly not a good idea as it results in worse quality. The format is by default "s16be" about which I have no idea what a different format would help. I do not know if the data can somehow be compress further, because this would definitely be an idea, but I dont know the procols internal. So the only left option is to solve the routing. For my personal setup there is no way to fix this on the router side, so the sender pulse module needs to take care of it. Using an ip as destination works perfectly and would be an option to use (which should be added to paprefs). However I tried that the last days and due to dhcp my receiver always gets a new IP and using hostnames would be a better idea. But the module does not seem to support hostnames. So I am wondering if this could be fixed quite simple or needs a whole more work to do. Another option would be to possibly create a port on the localhost and route that to the destination ip/name. And the kernel itself can do the name resolution. However I have no idea about that, but this could at least be a workaround to solve this issue. Other opinions? (In reply to freedesktop from comment #17) > Lowering the rate is possibly not a good idea as it results in worse > quality. The format is by default "s16be" about which I have no idea what a > different format would help. I do not know if the data can somehow be > compress further, because this would definitely be an idea, but I dont know > the procols internal. The data can certainly be compressed. I don't know if you've ever compared the sizes of mp3 and wav files. Typical compressed audio, when using a lossy codec, requires perhaps one tenth of the bandwidth that uncompressed audio requires, without too much effect on the audio quality. > So the only left option is to solve the routing. For my personal setup there > is no way to fix this on the router side, so the sender pulse module needs > to take care of it. Using an ip as destination works perfectly and would be > an option to use (which should be added to paprefs). However I tried that > the last days and due to dhcp my receiver always gets a new IP and using > hostnames would be a better idea. > > But the module does not seem to support hostnames. So I am wondering if this > could be fixed quite simple or needs a whole more work to do. Changing module-rtp-send to support hostnames doesn't sound terribly complicated. Sure mp3 would be way better. I just dont know how rtp works. I think its reasonable to use mp3 here. However I dont know how that works for streams and with pulse. Fixing the compression method would be the better fix, however a specific ip/hostname would not hurt neither. To my knowledge RTP can carry pretty much any format. You said on the mailing list that you want to use Kodi to receive the stream, and I don't know what Kodi supports, but I would guess that it supports many formats. Don't get too excited about compressed audio, though. I'm not volunteering to implement it, and it's probably pretty complicated to integrate to PulseAudio. In case someone is willing to take this task, the current "plan" (if you can call it a plan - it's all very fuzzy at the moment) is to use gstreamer to deal with the codecs, but in any case, please bring this topic up on the mailing list before spending too much time implementing it, so that there's an agreement about the design. *** Bug 98649 has been marked as a duplicate of this bug. *** All problems with RTP module flooding network, Wifi are caused by lack of IGMP support in rtp_send module. When pulseaudio with rtp_recv module is started proper IGMPv2 signalling works like it should: root@orangepione:~# tshark -i eth0 -Y "igmp" Capturing on 'eth0' 117 6.950723148 192.168.254.141 -> 239.255.1.56 IGMPv2 46 Membership Report group 239.255.1.56 177 10.430705605 192.168.254.141 -> 239.255.1.56 IGMPv2 46 Membership Report group 239.255.1.56 215 13.660683494 192.168.254.141 -> 239.255.1.56 IGMPv2 46 Membership Report group 239.255.1.56 Pulseaudio creates Multicast group (I don't use default group). As IGMP signalling is OK - IGMP capable switches enable that multicast group only on ports that need it. If pulseaudio is configured with rtp_send module - there is no IGMP signalling. Looking at code of module-rtp-recv.c we can find: mr4.imr_multiaddr = ((const struct sockaddr_in*) sa)->sin_addr; r = setsockopt(fd, IPPROTO_IP, IP_ADD_MEMBERSHIP, &mr4, sizeof(mr4)); IP_ADD_MEMBERSHIP tells kernel to join multicast group. There is no such command is module-rtp-send.c It needs fixing ASAP -- GitLab Migration Automatic Message -- This bug has been migrated to freedesktop.org's GitLab instance and has been closed from further activity. You can subscribe and participate further through the new bug through this link to our GitLab instance: https://gitlab.freedesktop.org/pulseaudio/pulseaudio/issues/505. |
Use of freedesktop.org services, including Bugzilla, is subject to our Code of Conduct. How we collect and use information is described in our Privacy Policy.