Bug 44777 - module-rtp-send floods the network with UDP packets, crippling it severely
Summary: module-rtp-send floods the network with UDP packets, crippling it severely
Status: NEW
Alias: None
Product: PulseAudio
Classification: Unclassified
Component: modules (show other bugs)
Version: unspecified
Hardware: All Linux (All)
: high major
Assignee: pulseaudio-bugs
QA Contact: pulseaudio-bugs
URL:
Whiteboard:
Keywords:
: 98649 (view as bug list)
Depends on:
Blocks:
 
Reported: 2012-01-14 04:24 UTC by gt6
Modified: 2017-10-02 20:18 UTC (History)
15 users (show)

See Also:
i915 platform:
i915 features:


Attachments
pcap - load-module module-rtp-send source=rtp.monitor (15.05 KB, application/octet-stream)
2014-01-19 03:15 UTC, Bill McGonigle
Details

Note You need to log in before you can comment on or make changes to this bug.
Description gt6 2012-01-14 04:24:47 UTC
When I enable module-rtp-send either by ticking [x] Enable Multicast/RTP sender in paprefs or adding something like 

load-module module-null-sink sink_name=rtp
load-module module-rtp-send

to /etc/pulse/default.pa, pulseaudio starts flooding the packet with more than 100 UDP packets per second. My router can't handle that and fails to serve any other devices. The network is practially DOS'd by pulseaudio.

Output from tcpdump -i eth0, as soon as the [x] Enable Multicast/RTP sender checkbox in paprefs is ticket, starts about 2 seconds later:

13:15:37.872284 IP 192.168.1.7.42051 > 224.0.0.56.sapv1: UDP, length 236
13:15:40.106404 IP 192.168.1.7.53698 > 224.0.0.56.46808: UDP, length 1292
13:15:40.113622 IP 192.168.1.7.53698 > 224.0.0.56.46808: UDP, length 1292
13:15:40.120879 IP 192.168.1.7.53698 > 224.0.0.56.46808: UDP, length 1292
13:15:40.128150 IP 192.168.1.7.53698 > 224.0.0.56.46808: UDP, length 1292
13:15:40.135305 IP 192.168.1.7.53698 > 224.0.0.56.46808: UDP, length 1292
13:15:40.142550 IP 192.168.1.7.53698 > 224.0.0.56.46808: UDP, length 1292
13:15:40.149782 IP 192.168.1.7.53698 > 224.0.0.56.46808: UDP, length 1292
13:15:40.157019 IP 192.168.1.7.53698 > 224.0.0.56.46808: UDP, length 1292
13:15:40.164254 IP 192.168.1.7.53698 > 224.0.0.56.46808: UDP, length 1292
13:15:40.171478 IP 192.168.1.7.53698 > 224.0.0.56.46808: UDP, length 1292
13:15:40.178687 IP 192.168.1.7.53698 > 224.0.0.56.46808: UDP, length 1292
13:15:40.185959 IP 192.168.1.7.53698 > 224.0.0.56.46808: UDP, length 1292
13:15:40.193206 IP 192.168.1.7.53698 > 224.0.0.56.46808: UDP, length 1292
13:15:40.200402 IP 192.168.1.7.53698 > 224.0.0.56.46808: UDP, length 1292
13:15:40.207614 IP 192.168.1.7.53698 > 224.0.0.56.46808: UDP, length 1292
13:15:40.214899 IP 192.168.1.7.53698 > 224.0.0.56.46808: UDP, length 1292
13:15:40.222118 IP 192.168.1.7.53698 > 224.0.0.56.46808: UDP, length 1292
13:15:40.229332 IP 192.168.1.7.53698 > 224.0.0.56.46808: UDP, length 1292
13:15:40.236563 IP 192.168.1.7.53698 > 224.0.0.56.46808: UDP, length 1292
13:15:40.243806 IP 192.168.1.7.53698 > 224.0.0.56.46808: UDP, length 1292
13:15:40.251050 IP 192.168.1.7.53698 > 224.0.0.56.46808: UDP, length 1292
13:15:40.258285 IP 192.168.1.7.53698 > 224.0.0.56.46808: UDP, length 1292
13:15:40.265471 IP 192.168.1.7.53698 > 224.0.0.56.46808: UDP, length 1292
13:15:40.272692 IP 192.168.1.7.53698 > 224.0.0.56.46808: UDP, length 1292
13:15:40.279998 IP 192.168.1.7.53698 > 224.0.0.56.46808: UDP, length 1292
13:15:40.287151 IP 192.168.1.7.53698 > 224.0.0.56.46808: UDP, length 1292
13:15:40.294461 IP 192.168.1.7.53698 > 224.0.0.56.46808: UDP, length 1292
13:15:40.301695 IP 192.168.1.7.53698 > 224.0.0.56.46808: UDP, length 1292
13:15:40.308892 IP 192.168.1.7.53698 > 224.0.0.56.46808: UDP, length 1292
13:15:40.316144 IP 192.168.1.7.53698 > 224.0.0.56.46808: UDP, length 1292
13:15:40.323376 IP 192.168.1.7.53698 > 224.0.0.56.46808: UDP, length 1292
13:15:40.330585 IP 192.168.1.7.53698 > 224.0.0.56.46808: UDP, length 1292
13:15:40.337884 IP 192.168.1.7.53698 > 224.0.0.56.46808: UDP, length 1292
13:15:40.345098 IP 192.168.1.7.53698 > 224.0.0.56.46808: UDP, length 1292
13:15:40.352285 IP 192.168.1.7.53698 > 224.0.0.56.46808: UDP, length 1292
13:15:40.359539 IP 192.168.1.7.53698 > 224.0.0.56.46808: UDP, length 1292
13:15:40.366760 IP 192.168.1.7.53698 > 224.0.0.56.46808: UDP, length 1292
13:15:40.374038 IP 192.168.1.7.53698 > 224.0.0.56.46808: UDP, length 1292
13:15:40.381829 IP 192.168.1.7.53698 > 224.0.0.56.46808: UDP, length 1292
13:15:40.388471 IP 192.168.1.7.53698 > 224.0.0.56.46808: UDP, length 1292
13:15:40.395700 IP 192.168.1.7.53698 > 224.0.0.56.46808: UDP, length 1292
13:15:40.402964 IP 192.168.1.7.53698 > 224.0.0.56.46808: UDP, length 1292
(...)


Some info:

pulseaudio -vvvvv says nothing interesting at all when enabling RTP.

[root@tachychineta shapeshifter]# uname -a
Linux tachychineta 3.1.5-1-ARCH #1 SMP PREEMPT Sun Dec 11 06:26:14 UTC 2011 i686 Intel(R) Core(TM)2 CPU T5600 @ 1.83GHz GenuineIntel GNU/Linux

[root@tachychineta shapeshifter]# pacman -Qi $(pacman -Qq | grep pulse) | grep -e "Name" -e "Version"
Name           : libcanberra-pulse
Version        : 0.28-2
Name           : libpulse
Version        : 1.1-2
Name           : pulseaudio
Version        : 1.1-2
Name           : pulseaudio-alsa
Version        : 1-2
Name           : pulseaudio-equalizer-bzr
Version        : 4-4

[root@tachychineta shapeshifter]# grep -v "^#" /etc/pulse/default.pa | grep -v "^$"
.nofail
.fail
load-module module-device-restore
load-module module-stream-restore
load-module module-card-restore
load-module module-augment-properties
.ifexists module-udev-detect.so
load-module module-udev-detect
.else
load-module module-detect
.endif
.ifexists module-jackdbus-detect.so
.nofail
.fail
.endif
.ifexists module-bluetooth-discover.so
load-module module-bluetooth-discover
.endif
.ifexists module-esound-protocol-unix.so
load-module module-esound-protocol-unix
.endif
load-module module-native-protocol-unix
.ifexists module-gconf.so
.nofail
load-module module-gconf
.fail
.endif
load-module module-default-device-restore
load-module module-rescue-streams
load-module module-always-sink
load-module module-intended-roles
load-module module-suspend-on-idle
.ifexists module-console-kit.so
.nofail
load-module module-console-kit
.fail
.endif
load-module module-position-event-sounds
load-module module-cork-music-on-phone
load-module module-filter-heuristics
load-module module-filter-apply
.ifexists module-dbus-protocol.so
load-module module-dbus-protocol
.endif
Comment 1 Arun Raghavan 2012-01-14 04:28:32 UTC
OP also linked to this: https://bugs.launchpad.net/ubuntu/+source/pulseaudio/+bug/411688
Comment 2 lops777 2012-06-13 14:38:03 UTC
Same issue. Using rtp-send causes flooding of my whole local network. Computer using pulseaudio is connected via LAN on a router. I am using Ubuntu 12.04 / 64bit. Other computers are connected via WLAN and they loose connection. Sometimes router crashes. Flooding has a bandwidth of 180 KByte/s. 

I have seen in google that this problem is existing since at least 2008 !!!!! Why the guys report bugs but noone is caring??
Comment 3 mathieu 2012-09-23 08:09:12 UTC
Same thing here, rtp module is an insteresting feature but is just unusable. An update anyone on this problem ?
Comment 4 Tanu Kaskinen 2012-09-24 13:38:12 UTC
No updates. I'm not aware of anyone working on this.
Comment 5 alloftheinternetinone 2013-02-17 22:48:24 UTC
Subscribing in hope of fix.
Comment 6 sjimmie 2013-03-21 21:19:36 UTC
Exactly the same problem here.
Running Ubuntu 12.10 64bit with pulseaudio 2.1
Comment 7 Thomas Martitz 2013-04-15 15:19:23 UTC
I have this problem too. My network becomes completely unusable.

I also observe a constant bandwidth of a bit less than 200KB/s. This fits the expected PCM data rate (176KB/s). But this shouldn't congest the network. And indeed the same bandwidth caused by module-tunnel-sink works completely fine.

Are our routers at fault or is something fishy with module-rtp-send?
Comment 8 Jean-François Fortin Tam 2013-07-03 04:33:41 UTC
Just confirming that I'm seeing that kind of behavior on Fedora 19 too (so not ubuntu-specific), enabling the Multicast/RTP sender will cause constant UDP upload activity towards 224.0.0.56, in my case at ~180 KB/s. I think this explains my other computer (laptop) having trouble associating to the wifi, even though this desktop computer is connected through a wired connection.

My router is basically new hardware (Netgear WNR3500L/U/v2) running the Tomato firmware, and I really doubt that anything is at fault on that side.
Comment 9 Bill McGonigle 2014-01-19 03:15:54 UTC
Created attachment 92370 [details]
pcap - load-module module-rtp-send source=rtp.monitor

Adding a pcap of the traffic I get when loading module-rtp-send:

  load-module module-rtp-send source=rtp.monitor

and doing nothing more than 'pulseaudio -k'.  Jump down past packet 343 where the first SDP capture is, for meaningful wireshark display.  It looks like there are two issues here:

1) I presume module-rtp-send shouldn't be sending a constant stream of zero-filled packets just by being loaded with no sources assigned to the sink.  That seems really inefficient, especially if sound output is infrequent and/or you have a network with many computers and want to be able to support n-to-n source/sink setups (n number of constant all-zero multicast streams...).

2) I also saw the local WiFi network become crippled by the packet storm.  I see in the capture a packet rate of 137 per second.  I'm running a -n access point running OpenWRT 12.09 and have a solid 180Mbps+ connection, so I looked into a bit further and found:

a) multicast on wireless is effectively useless, or perhaps dangerous.  Multicast packets always use the lowest possible data rate on WiFi, which causes them to consume all available bandwidth and crowd out all other traffic:

  https://dev.openwrt.org/ticket/10271

It appears that even running multicast on a wired network where wifi access points are connected can cause a takedown of the wifi networks if the wifi access points aren't filtering multicast.

b) there are some proposals for fixing this in a later revision of 802.11 but nothing we can use for a long time.

c) there are a bunch of people who have tried to use PA with RTP and some Raspberry Pis to distribute music and they've crashed and burned.  The Wiki should probably note that this shouldn't be attempted if there's a WiFi network involved.  Still, there is strong demand for this use case, if not RTP necessarily.*

d) Multicast on WiFi is so bad that it's more efficient to wrap the UDP in TCP and proxy it at the gateway:

  http://wiki.openwrt.org/doc/howto/udp_multicast
  http://www.udpxy.com/index-en.html

In my testing, igmp snooping on OpenWRT (and presumably all of its derivatives) doesn't work yet.   I wonder if supporting udpxy's protocol in PA would make sense for the WiFi use case.  Perhaps smarter would be:

* there appears to be an evolving open source standard around the SlimProto 

  http://wiki.slimdevices.com/index.php/SlimProtoTCPProtocol
  http://wiki.slimdevices.com/index.php/SlimProtoDeveloperGuide
  https://code.google.com/p/squeezelite/source/browse/slimproto.c
  http://wiki.slimdevices.com/index.php/Design_of_new-streaming
  http://wiki.slimdevices.com/index.php/Synchronization

and the LMS implementation is said to have excellent synchronization which looks to be handled fairly simply in the perl code.  A PA sender and receiver that implemented a subset of squeeze might be neat to have - might even be usable with some of the hardware players.  Note: LMS (perl) is GPLv2 but slimsqueeze (c) is GPLv3 so direct code-reuse into PA is probably not possible.
Comment 10 marecek77 2016-04-06 09:06:12 UTC
This bug is still present in Fedora 23 x86_64 and pulseaudio-7.1-1
My machine was sending ~ 180KB/s to 224.0.0.56 until I unchecked "Enable Multicast/RTP sender" in paprefs.

For peace of mind, I ended up adding "unload-module module-rtp-send source=rtp.monitor" into /etc/pulse/default.pa
Comment 11 Kenn Sully 2016-10-13 13:02:28 UTC
 I noticed the same bug in my system ( Ubuntu 14.04.5) while I was trying to set up pulseaudio server in Windows 10 over the local network. In my case network flooding occurs whenever I launch pavucontrol, upload traffics reaches up to 200kb/s and it reduced the local network unusable. When I close it the network attack diminishes.
I added `unload-module module-rtp-send` into `/etc/pulse/default.pa` to workaround the bug.
Comment 12 freedesktop 2016-11-01 21:15:48 UTC
I've also noticed this in the most up to data pulse versions from archlinux. I also cannot use a ttl of 1, as the router does not decrease the value when routing into wlan, as it has the same subnet. Neither does destination ip 24.0.0.1 help. From my side I have no chance to change the router config, because the internet is provided by someone else.

What works for me though is to stream the music directly to the destination via destination_ip=192.168.178.x

I could also add a menu in paprefs that'd also adress the ip directly, however this is more of a struggle because of the old gtk2 (if anyone here is following the mailing list). It would be highly appreciated to fix this bug in the module itself and reduce the payload somehow.
Comment 13 Tanu Kaskinen 2016-11-02 10:53:24 UTC
I read through the comments, and to me it looks like module-rtp-send doesn't have any flooding bug. The data rate of PCM audio is what it is, and the network hardware isn't always able to handle it. Multicast RTP is not enabled by default. If the user enables it, I don't know what we could possibly do to not "flood" the network.

Since this is a common problem for people, there is certainly room for improvement in the UI and documentation so that users don't get so surprised when things don't work.
Comment 14 Tanu Kaskinen 2016-11-02 11:10:55 UTC
Adding to my previous comment: it could be argued that the bug is that we transmit PCM audio instead of compressed audio. Adding support for compressed audio should help with the network stress. Does anyone volunteer to implement this?
Comment 15 gt6 2016-11-02 11:25:40 UTC
I know nothing about RTP, but from a user perspective, I have two questions:
 1) Transferring a WAV file over the network goes faster than real time (in terms of the audio contained within). Why should the network struggle with uncompressed RTP audio, then?
 2) Why are packets generated when nothing is played? If there is no audio activity in PA, there is no need to send any packets.
Apologies if this doesn't make sense, I'm talking purely as an end-user.
Comment 16 Tanu Kaskinen 2016-11-02 13:07:17 UTC
Sending a wav file means a TCP connection between two computers. TCP connections are the most common type of traffic, and the hardware is optimized accordingly. UDP multicast is different, and apparently network hardware is not necessarily as efficient with processing that protocol. Comment #9 seems to suggest that the wifi specification itself forces multicast to be inefficient, so it's not necessarily just the hardware's fault.

Starting from PA version 5.0, module-rtp-send should not generate traffic when nothing is playing (it can still be configured to play silence, but paprefs doesn't configure the module that way). If you observe different behaviour, please file a new bug and attach the output of "pactl list".
Comment 17 freedesktop 2016-11-03 17:27:35 UTC
You are right that this feature needs to be enabled first to make the bug rise. However I do not think that documentating this bug/behavior solves the problem. I do want to use music streaming, but I cannot do this as my router routes the traffic into the WLAN.

Lowering the rate is possibly not a good idea as it results in worse quality. The format is by default "s16be" about which I have no idea what a different format would help. I do not know if the data can somehow be compress further, because this would definitely be an idea, but I dont know the procols internal.

So the only left option is to solve the routing. For my personal setup there is no way to fix this on the router side, so the sender pulse module needs to take care of it. Using an ip as destination works perfectly and would be an option to use (which should be added to paprefs). However I tried that the last days and due to dhcp my receiver always gets a new IP and using hostnames would be a better idea.

But the module does not seem to support hostnames. So I am wondering if this could be fixed quite simple or needs a whole more work to do.

Another option would be to possibly create a port on the localhost and route that to the destination ip/name. And the kernel itself can do the name resolution. However I have no idea about that, but this could at least be a workaround to solve this issue.

Other opinions?
Comment 18 Tanu Kaskinen 2016-11-03 18:11:45 UTC
(In reply to freedesktop from comment #17)
> Lowering the rate is possibly not a good idea as it results in worse
> quality. The format is by default "s16be" about which I have no idea what a
> different format would help. I do not know if the data can somehow be
> compress further, because this would definitely be an idea, but I dont know
> the procols internal.

The data can certainly be compressed. I don't know if you've ever compared the sizes of mp3 and wav files. Typical compressed audio, when using a lossy codec, requires perhaps one tenth of the bandwidth that uncompressed audio requires, without too much effect on the audio quality.

> So the only left option is to solve the routing. For my personal setup there
> is no way to fix this on the router side, so the sender pulse module needs
> to take care of it. Using an ip as destination works perfectly and would be
> an option to use (which should be added to paprefs). However I tried that
> the last days and due to dhcp my receiver always gets a new IP and using
> hostnames would be a better idea.
> 
> But the module does not seem to support hostnames. So I am wondering if this
> could be fixed quite simple or needs a whole more work to do.

Changing module-rtp-send to support hostnames doesn't sound terribly complicated.
Comment 19 freedesktop 2016-11-03 18:33:45 UTC
Sure mp3 would be way better. I just dont know how rtp works. I think its reasonable to use mp3 here. However I dont know how that works for streams and with pulse.

Fixing the compression method would be the better fix, however a specific ip/hostname would not hurt neither.
Comment 20 Tanu Kaskinen 2016-11-03 19:03:42 UTC
To my knowledge RTP can carry pretty much any format. You said on the mailing list that you want to use Kodi to receive the stream, and I don't know what Kodi supports, but I would guess that it supports many formats.

Don't get too excited about compressed audio, though. I'm not volunteering to implement it, and it's probably pretty complicated to integrate to PulseAudio.

In case someone is willing to take this task, the current "plan" (if you can call it a plan - it's all very fuzzy at the moment) is to use gstreamer to deal with the codecs, but in any case, please bring this topic up on the mailing list before spending too much time implementing it, so that there's an agreement about the design.
Comment 21 Tanu Kaskinen 2016-11-11 11:12:54 UTC
*** Bug 98649 has been marked as a duplicate of this bug. ***
Comment 22 Piotr Górski 2017-05-11 11:46:21 UTC
All problems with RTP module flooding network, Wifi are caused by lack of IGMP support in rtp_send module. 

When pulseaudio with rtp_recv module is started proper IGMPv2 signalling works like it should:

root@orangepione:~# tshark -i eth0 -Y "igmp"
Capturing on 'eth0'
117 6.950723148 192.168.254.141 -> 239.255.1.56 IGMPv2 46 Membership Report group 239.255.1.56
177 10.430705605 192.168.254.141 -> 239.255.1.56 IGMPv2 46 Membership Report group 239.255.1.56
215 13.660683494 192.168.254.141 -> 239.255.1.56 IGMPv2 46 Membership Report group 239.255.1.56

Pulseaudio creates Multicast group (I don't use default group). As IGMP signalling is OK - IGMP capable switches enable that multicast group only on ports that need it. 

If pulseaudio is configured with rtp_send module - there is no IGMP signalling. 

Looking at code of module-rtp-recv.c we can find:

            mr4.imr_multiaddr = ((const struct sockaddr_in*) sa)->sin_addr;
            r = setsockopt(fd, IPPROTO_IP, IP_ADD_MEMBERSHIP, &mr4, sizeof(mr4));

IP_ADD_MEMBERSHIP tells kernel to join multicast group. 

There is no such command is module-rtp-send.c 

It needs fixing ASAP


Use of freedesktop.org services, including Bugzilla, is subject to our Code of Conduct.