Created attachment 49710 [details] Kinect, module-alsa-card.c: Failed to find a working profile. Hi, the Microsoft Kinect sensor device comes with array of four microphones on it and it shows up as a USB Audio Class device after uploading a firmware to it, but the Linux audio stack still needs some touches before the device works out of the box. This report starts from pulseaudio to track what the missing pieces are, I suspect snd-usb-audio is involved too. This is a timeline of what happens: 1. the sensor device is connected; 2. a udev rule detects the audio part of the device (idVendor=045e, idProduct=02ad, Product: Xbox NUI Audio) and uploads the firmware to it; 3. the device is re-enumerated and shows up as a proper USB Audio Class device (idVendor=045e, idProduct=02bb, Product: Kinect USB Audio). At this point pulseaudio fails to detect it (full log in the attachments): pulseaudio[1643]: module-alsa-card.c: Failed to find a working profile. pulseaudio[1643]: module.c: Failed to load module "module-alsa-card" (argument: "device_id="1" name="usb-Microsoft_Kinect_USB_Audio_A44884D23275040A-01-Audio" card_name="alsa_card.usb-Microsoft_Kinect_USB_Audio_A44884D23275040A-01-Audio" tsched=yes ignore_dB=no card_properties="module-udev-detect.discovered=1""): initialization failed. I noticed that alsamixer tells "This sound device does not have any controls.", I don't know if this is the cause of the message above. However if I kill and restart pulseaudio the device gets detected and can be used mostly OK, even if the names of the Microphones are not right yet (see the attached screenshot), I imagined this happened because udev is not involved when restarting the audio server with the device connected already, but I am not sure of this. About alsamixer not showing any controls, by looking at the USB descriptor (attached as well) I thought it was because snd-usb-audio does not support the microphones arrays yet (wTerminalType 0x0205 Microphone Array), a quick "grep -r 205 sound/usb" seems to confirm this but again I am not 100% sure. I am looking into this but I wanted to ask for advice here too, and keep history of the pulseaudio point of view of the issue anyways. Thanks, Antonio Ospite http://ao2.it
Created attachment 49712 [details] after restarting the server pavucontrol shows the four microphones but with wrong names.
Created attachment 49714 [details] the UAC descriptor of the Kinect USB Audio device
Hi Antonio, Thanks for the bug report. Would it be possible to get us a better log from PA. The details in the syslog are pretty sparse, so if you could disable autospawning via: "echo autospawn=no >>~/.pulse/client.conf" and then run PA manually on the command line via "pulseaudio -vvvvv" then it should give you a lot more info which could help. I suspect that the trick here will be to write a specific profile for this device (it's quite specialised after all) which will then be able to give the right names to the devices. Note however, that our channel names are relatively fixed (they have to be so that sounds can be mapped to channels properly) and thus you may not be able to get the perfect names here. http://pulseaudio.org/wiki/WritingProfiles In addition to the the debug output please also supply: 1. amixer -c1 (where 1 is the card number - cat /proc/adound/cards) 2. pacmd ls (when the device is working properly - perhaps after restarting PA) Note that even when PA is restarted, udev is used to detect things. I suspect the problem of it not showing up is related to some kind of race condition.
(In reply to comment #3) > Hi Antonio, > > Thanks for the bug report. > > Would it be possible to get us a better log from PA. The details in the syslog > are pretty sparse, so if you could disable autospawning via: "echo autospawn=no > >>~/.pulse/client.conf" and then run PA manually on the command line via > "pulseaudio -vvvvv" then it should give you a lot more info which could help. > OK, full log attached in 'pulseaudio_verbose_kinect_NOT_detected.txt' you can skip to when I plug in the device by going to line 1018 "ABOUT TO PLUG IN THE Kinect", and the tentative detection is until line 4223 "NOT DETECTED" I noticed that if the device is _already_ plugged in when pulseaudio starts, then it gets recognized automatically, it looks like this happens because the Kinect is detected _before_ the other soundcard in the system, this seems to be confirmed by the "pacmd ls" output from below. I am attaching also the log for this case, see 'pulseaudio_verbose_kinect_detected_first.txt'. I tried to compare the failure with the success to narrow down the cause but I can't figure it out yet. > I suspect that the trick here will be to write a specific profile for this > device (it's quite specialised after all) which will then be able to give the > right names to the devices. Note however, that our channel names are relatively > fixed (they have to be so that sounds can be mapped to channels properly) and > thus you may not be able to get the perfect names here. > > > http://pulseaudio.org/wiki/WritingProfiles > OK, I can look into this later can't I? The misnaming is not related to the failed detection I suppose. > In addition to the the debug output please also supply: > > 1. amixer -c1 (where 1 is the card number - cat /proc/adound/cards) > 2. pacmd ls (when the device is working properly - perhaps after restarting > PA) 1. amixer shows nothing, no controls at all: $ cat /proc/asound/cards 0 [NVidia ]: HDA-Intel - HDA NVidia HDA NVidia at 0xfcf78000 irq 20 1 [Audio ]: USB-Audio - Kinect USB Audio Microsoft Kinect USB Audio at usb-0000:00:02.1-5.1, high speed $ amixer -c1 $ 2. output from "pacmd ls" is attached, when the Kinect is recognized it shows up as the first sourdcard to pulseaudio, see from line 525. > Note that even when PA is restarted, udev is used to detect things. I suspect > the problem of it not showing up is related to some kind of race condition. > Thanks, Antonio
Created attachment 49909 [details] When plugging the Kinect after pulseaudio is running, it is not recognized
Created attachment 49910 [details] Kinect plugged _before_ pulsteaudio gets started, detected
Created attachment 49911 [details] pacmd ls
OK, with the attached profile (kinect-audio.conf) and the proper udev rules (55-kinect_audio.rules, using Drew's upload_fw), I have the device working fine even when plugged _after_ pulseaudio has started. The profile is very simple it just defines a 4 channels, input-only mapping on hw:%f. Can this help you to understand what the cause of the original failure with default.conf is? I found a possible cure, but I am still not sure about what the cause of the problem was. Is there still room for improvement about the channels names? I mean, am I forced to use fixed names ( front-left, front-right, rear-left, rear-right), or can I use something like Mic1, Mic2, Mic3, Mic4? Regards, Antonio Ospite http://ao2.it
Created attachment 50964 [details] kinect-audio.conf
Created attachment 50965 [details] 55-kinect_audio.rules
OK, so we generally don't probe for 4ch input. It's just not that common, so I think a specialist profile like this makes sense. The firmware uploader is obviously a bit custom, but I can add the profile mapping quite happily to our official repository if it makes sense. The firmware stuff is likely something you need to add to the alsa-firmware project. Not sure of any legal problems with that tho'. Should the inputs really be labelled as digital? Are they not actually analogue? At present, I'm really not sure why it would work with hotplug and not coldplug :( If I can ever borrow a friends Kinect system I will try and have a look, but so far I don't know anyone nearby with such kit.
(In reply to comment #11) > OK, so we generally don't probe for 4ch input. It's just not that common, so I > think a specialist profile like this makes sense. > > The firmware uploader is obviously a bit custom, but I can add the profile > mapping quite happily to our official repository if it makes sense. > > Should the inputs really be labelled as digital? Are they not actually > analogue? > About the kinect.conf profile, I can send a patch about it, if this is the way we want to fix it, and I will not mention "Digital" at all, as you suggested. > The firmware stuff is likely something you need to add to the alsa-firmware > project. Not sure of any legal problems with that tho'. > Yep, I'll split the udev file in two, the "profile" rule to be shipped with pulseaudio in 90-pulseaudio.rules, and the "firmware loading" one to be shipped independently. > At present, I'm really not sure why it would work with hotplug and not coldplug > :( If I can ever borrow a friends Kinect system I will try and have a look, but > so far I don't know anyone nearby with such kit. I tested on the latest development version from git://anongit.freedesktop.org/pulseaudio/pulseaudio and it does NOT even work on hotplug, this seems to be because this difference in default.conf: -[Mapping iec958-surround-40] -device-strings = iec958:%f -channel-map = front-left,front-right,rear-left,rear-right -priority = 1 - After some time reading the log I thinks I can at least isolate what could be the important messages: With the _stable_ version it does not work if I plug the device after loading PA: D: alsa-util.c: Trying iec958:1 with SND_PCM_NO_AUTO_FORMAT ... I: (alsa-lib)confmisc.c: Unable to find definition 'cards.USB-Audio.pcm.iec958.0:CARD=1,AES0=4,AES1=130,AES2=0,AES3=2' I: (alsa-lib)conf.c: function snd_func_refer returned error: No such file or directory I: (alsa-lib)conf.c: Evaluate error: No such file or directory I: (alsa-lib)pcm.c: Unknown PCM iec958:1 I: alsa-util.c: Error opening PCM device iec958:1: No such file or directory I: module.c: Loaded "module-alsa-card" (index: #19; argument: "device_id="1" name="usb-Microsoft_Kinect_USB_Audio_A44884D23275040A-01-Audio" card_name="alsa_card.usb-Microsoft_Kinect_USB_Audio_A44884D23275040A-01-Audio" tsched=yes ignore_dB=no card_properties="module-udev-detect.discovered=1""). Instead it works when the device is already plugged in: D: alsa-mixer.c: Looking at profile input:iec958-surround-40 D: alsa-mixer.c: Checking for recording on Surround digitale 4.0 (IEC958) (iec958-surround-40) D: alsa-util.c: Trying iec958:1 with SND_PCM_NO_AUTO_FORMAT ... D: alsa-util.c: Managed to open iec958:1 D: alsa-util.c: snd_pcm_hw_params_set_format(Signed 16 bit Little Endian) failed: Argomento non valido D: alsa-util.c: snd_pcm_hw_params_set_format(Signed 16 bit Big Endian) failed: Argomento non valido D: alsa-util.c: snd_pcm_hw_params_set_format(Float 32 bit Little Endian) failed: Argomento non valido D: alsa-util.c: snd_pcm_hw_params_set_format(Float 32 bit Big Endian) failed: Argomento non valido D: alsa-util.c: Maximum hw buffer size is 4096 ms D: alsa-util.c: Set buffer size first (to 1599 samples), period size second (to 399 samples). I: alsa-util.c: Device iec958:1 doesn't support 44100 Hz, changed to 16000 Hz. I: alsa-util.c: Device iec958:1 doesn't support sample format s16le, changed to s32le. D: alsa-mixer.c: Profile input:iec958-surround-40 supported. I: module-card-restore.c: Restoring profile for card alsa_card.usb-Microsoft_Kinect_USB_Audio_A44884D23275040A-01-Audio. I: card.c: Created 2 "alsa_card.usb-Microsoft_Kinect_USB_Audio_A44884D23275040A-01-Audio" D: reserve-wrap.c: Successfully create reservation lock monitor for device 'Audio1' D: alsa-util.c: Trying iec958:1 with SND_PCM_NO_AUTO_FORMAT ... D: alsa-util.c: Managed to open iec958:1 D: alsa-util.c: snd_pcm_hw_params_set_format(Signed 16 bit Little Endian) failed: Argomento non valido D: alsa-util.c: snd_pcm_hw_params_set_format(Signed 16 bit Big Endian) failed: Argomento non valido D: alsa-util.c: snd_pcm_hw_params_set_format(Float 32 bit Little Endian) failed: Argomento non valido D: alsa-util.c: snd_pcm_hw_params_set_format(Float 32 bit Big Endian) failed: Argomento non valido D: alsa-util.c: Maximum hw buffer size is 4096 ms D: alsa-util.c: Set buffer size first (to 32000 samples), period size second (to 32000 samples). I: alsa-util.c: Device iec958:1 doesn't support 44100 Hz, changed to 16000 Hz. I: alsa-util.c: Device iec958:1 doesn't support sample format s16le, changed to s32le. I: alsa-source.c: Successfully opened device iec958:1. I: alsa-source.c: Selected mapping 'Surround digitale 4.0 (IEC958)' (iec958-surround-40). I: alsa-source.c: Successfully enabled mmap() mode. I: alsa-source.c: Successfully enabled timer-based scheduling mode. I: (alsa-lib)control.c: Invalid CTL iec958:1 I: alsa-mixer.c: Unable to attach to mixer iec958:1: File o directory non esistente I: alsa-mixer.c: Successfully attached to mixer 'hw:1' ... It looks like it tried the four channel mapping (iec958-surround-40) on iec958:1, but then ends up using hw:1 anyways, but I still haven't figured out the WHY, this is just a hint for when (if) you will look at that. The symptoms looked similar to the unrelated post here: http://www.mail-archive.com/pulseaudio-discuss@mail.0pointer.de/msg04847.html Thanks, Antonio Ospite http://ao2.it
(In reply to comment #12) > (In reply to comment #11) > > OK, so we generally don't probe for 4ch input. It's just not that common, so I > > think a specialist profile like this makes sense. > > > > The firmware uploader is obviously a bit custom, but I can add the profile > > mapping quite happily to our official repository if it makes sense. > > > > Should the inputs really be labelled as digital? Are they not actually > > analogue? > > > > About the kinect.conf profile, I can send a patch about it, if this is the way > we want to fix it, and I will not mention "Digital" at all, as you suggested. > > > The firmware stuff is likely something you need to add to the alsa-firmware > > project. Not sure of any legal problems with that tho'. > > > > Yep, I'll split the udev file in two, the "profile" rule to be shipped with > pulseaudio in 90-pulseaudio.rules, and the "firmware loading" one to be > shipped independently. Perfect, thanks. > > At present, I'm really not sure why it would work with hotplug and not coldplug > > :( If I can ever borrow a friends Kinect system I will try and have a look, but > > so far I don't know anyone nearby with such kit. > > I tested on the latest development version from > git://anongit.freedesktop.org/pulseaudio/pulseaudio and it does NOT even work > on hotplug, this seems to be because this difference in default.conf: > > -[Mapping iec958-surround-40] > -device-strings = iec958:%f > -channel-map = front-left,front-right,rear-left,rear-right > -priority = 1 > - I presume you mean without modifying it with your specific profile? I guess this was removed as it was thought to be invalid (and does it even work with iec958: for input anyway?) > After some time reading the log I thinks I can at least isolate what could be > the important messages: > > With the _stable_ version it does not work if I plug the device after loading > PA: > > D: alsa-util.c: Trying iec958:1 with SND_PCM_NO_AUTO_FORMAT ... > I: (alsa-lib)confmisc.c: Unable to find definition > 'cards.USB-Audio.pcm.iec958.0:CARD=1,AES0=4,AES1=130,AES2=0,AES3=2' > I: (alsa-lib)conf.c: function snd_func_refer returned error: No such file or > directory > I: (alsa-lib)conf.c: Evaluate error: No such file or directory > I: (alsa-lib)pcm.c: Unknown PCM iec958:1 > I: alsa-util.c: Error opening PCM device iec958:1: No such file or directory > I: module.c: Loaded "module-alsa-card" (index: #19; argument: "device_id="1" > name="usb-Microsoft_Kinect_USB_Audio_A44884D23275040A-01-Audio" > card_name="alsa_card.usb-Microsoft_Kinect_USB_Audio_A44884D23275040A-01-Audio" > tsched=yes ignore_dB=no card_properties="module-udev-detect.discovered=1""). This smells like a race - e.g. udev is telling us there is a new device before it's actually ready for use - i.e the firmware load is not yet completely and thus the card is not yet ready. > Instead it works when the device is already plugged in: > > D: alsa-mixer.c: Looking at profile input:iec958-surround-40 > D: alsa-mixer.c: Checking for recording on Surround digitale 4.0 (IEC958) > (iec958-surround-40) > D: alsa-util.c: Trying iec958:1 with SND_PCM_NO_AUTO_FORMAT ... > D: alsa-util.c: Managed to open iec958:1 > D: alsa-util.c: snd_pcm_hw_params_set_format(Signed 16 bit Little Endian) > failed: Argomento non valido > D: alsa-util.c: snd_pcm_hw_params_set_format(Signed 16 bit Big Endian) failed: > Argomento non valido > D: alsa-util.c: snd_pcm_hw_params_set_format(Float 32 bit Little Endian) > failed: Argomento non valido > D: alsa-util.c: snd_pcm_hw_params_set_format(Float 32 bit Big Endian) failed: > Argomento non valido > D: alsa-util.c: Maximum hw buffer size is 4096 ms > D: alsa-util.c: Set buffer size first (to 1599 samples), period size second (to > 399 samples). > I: alsa-util.c: Device iec958:1 doesn't support 44100 Hz, changed to 16000 Hz. > I: alsa-util.c: Device iec958:1 doesn't support sample format s16le, changed to > s32le. > D: alsa-mixer.c: Profile input:iec958-surround-40 supported. > I: module-card-restore.c: Restoring profile for card > alsa_card.usb-Microsoft_Kinect_USB_Audio_A44884D23275040A-01-Audio. > I: card.c: Created 2 > "alsa_card.usb-Microsoft_Kinect_USB_Audio_A44884D23275040A-01-Audio" > D: reserve-wrap.c: Successfully create reservation lock monitor for device > 'Audio1' > D: alsa-util.c: Trying iec958:1 with SND_PCM_NO_AUTO_FORMAT ... > D: alsa-util.c: Managed to open iec958:1 > D: alsa-util.c: snd_pcm_hw_params_set_format(Signed 16 bit Little Endian) > failed: Argomento non valido > D: alsa-util.c: snd_pcm_hw_params_set_format(Signed 16 bit Big Endian) failed: > Argomento non valido > D: alsa-util.c: snd_pcm_hw_params_set_format(Float 32 bit Little Endian) > failed: Argomento non valido > D: alsa-util.c: snd_pcm_hw_params_set_format(Float 32 bit Big Endian) failed: > Argomento non valido > D: alsa-util.c: Maximum hw buffer size is 4096 ms > D: alsa-util.c: Set buffer size first (to 32000 samples), period size second > (to 32000 samples). > I: alsa-util.c: Device iec958:1 doesn't support 44100 Hz, changed to 16000 Hz. > I: alsa-util.c: Device iec958:1 doesn't support sample format s16le, changed to > s32le. > I: alsa-source.c: Successfully opened device iec958:1. > I: alsa-source.c: Selected mapping 'Surround digitale 4.0 (IEC958)' > (iec958-surround-40). > I: alsa-source.c: Successfully enabled mmap() mode. > I: alsa-source.c: Successfully enabled timer-based scheduling mode. > I: (alsa-lib)control.c: Invalid CTL iec958:1 > I: alsa-mixer.c: Unable to attach to mixer iec958:1: File o directory non > esistente > I: alsa-mixer.c: Successfully attached to mixer 'hw:1' > ... > > It looks like it tried the four channel mapping (iec958-surround-40) on > iec958:1, > but then ends up using hw:1 anyways, but I still haven't figured out the WHY, > this is just a hint for when (if) you will look at that. It could be a timing thing... perhaps it's just as simple as that. Question is, how to fix it! > The symptoms looked similar to the unrelated post here: > http://www.mail-archive.com/pulseaudio-discuss@mail.0pointer.de/msg04847.html Yeah I remember that issue. Not sure it was ever solved tho' :(
(In reply to comment #13) > (In reply to comment #12) > > (In reply to comment #11) > > > OK, so we generally don't probe for 4ch input. It's just not that common, so I > > > think a specialist profile like this makes sense. > > > > > > The firmware uploader is obviously a bit custom, but I can add the profile > > > mapping quite happily to our official repository if it makes sense. > > > > > > Should the inputs really be labelled as digital? Are they not actually > > > analogue? > > > > > > > About the kinect.conf profile, I can send a patch about it, if this is the way > > we want to fix it, and I will not mention "Digital" at all, as you suggested. > > > > > The firmware stuff is likely something you need to add to the alsa-firmware > > > project. Not sure of any legal problems with that tho'. > > > > > > > Yep, I'll split the udev file in two, the "profile" rule to be shipped with > > pulseaudio in 90-pulseaudio.rules, and the "firmware loading" one to be > > shipped independently. > > Perfect, thanks. > OK, so a patch is on its way. > > > At present, I'm really not sure why it would work with hotplug and not coldplug > > > :( If I can ever borrow a friends Kinect system I will try and have a look, but > > > so far I don't know anyone nearby with such kit. > > > > I tested on the latest development version from > > git://anongit.freedesktop.org/pulseaudio/pulseaudio and it does NOT even work > > on hotplug, this seems to be because this difference in default.conf: > > > > -[Mapping iec958-surround-40] > > -device-strings = iec958:%f > > -channel-map = front-left,front-right,rear-left,rear-right > > -priority = 1 > > - > > I presume you mean without modifying it with your specific profile? Yes, I mean without the specific profile.
Created attachment 51004 [details] [review] 0001-alsa-mixer-Add-support-for-the-Microsoft-Kinect-Sens.patch The patch can be applied with "git am", I tried to follow the style used in the other commit messages, but feel free to amend the commit message, or the profile, if necessary. Thanks, Antonio
OK, thanks for that. I noticed it was missing an entry for the file in src/Makefile.am (a bit late as I'd already pushed, but at least I noticed!) so this is in master now and will be in 1.0 when we release it (shortly). Hopefully the other stuff will be solved outside of PA and things will magically start working!
(In reply to comment #16) > OK, thanks for that. I noticed it was missing an entry for the file in > src/Makefile.am (a bit late as I'd already pushed, but at least I noticed!) so > this is in master now and will be in 1.0 when we release it (shortly). > Hopefully the other stuff will be solved outside of PA and things will > magically start working! Thanks Colin for catching the Makefile.am issue, I didn't try the installation. Regards, Antonio
I'd like to note that the removal of the iec958-surround-40 profile affected other mics too, notably the Playstation 3 Eye which no longer works with pulseaudio in recent Ubuntu distributions and which is affecting a lot of people, including me. https://bugs.launchpad.net/ubuntu/+source/pulseaudio/+bug/889624 I've seen similar reports for Arch Linux and Fedora. This should not be a special-case profile, it should be in default.conf. There are several 4-channel microphone arrays on the market usually packaged in webcams. Just googling I find the following 4-channel mic array webcams at bestbuy you can walk in and buy: http://www.bestbuy.com/site/Creative+-+Live!+Cam+inPerson+HD+Webcam/3613018.p?id=1218463912602&skuId=3613018 http://www.bestbuy.com/site/FreeTalk+-+Skype+Conference+Webcam/2477095.p?id=1218329600068&skuId=2477095
(In reply to comment #18) > I'd like to note that the removal of the iec958-surround-40 profile affected > other mics too, notably the Playstation 3 Eye which no longer works with > pulseaudio in recent Ubuntu distributions and which is affecting a lot of > people, including me. This should have been a separate bug. I've filed one for you this time to continue the discussion (bug #45813). Marking this one RESO FIXED since the comments suggest that everything needed has been committed.
Use of freedesktop.org services, including Bugzilla, is subject to our Code of Conduct. How we collect and use information is described in our Privacy Policy.