Bug 97442 - [HSW HDMI] Audio with 44.1 khz not working when 1920x1080p screen refreshrate > 30hz
Summary: [HSW HDMI] Audio with 44.1 khz not working when 1920x1080p screen refreshrate...
Status: CLOSED FIXED
Alias: None
Product: DRI
Classification: Unclassified
Component: DRM/Intel (show other bugs)
Version: XOrg git
Hardware: x86-64 (AMD64) Linux (All)
: medium normal
Assignee: Intel GFX Bugs mailing list
QA Contact: Intel GFX Bugs mailing list
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2016-08-22 20:17 UTC by Mark Wilczynski
Modified: 2017-07-24 22:40 UTC (History)
5 users (show)

See Also:
i915 platform: HSW
i915 features: display/audio, display/HDMI


Attachments
Limit the depth of the display pipeline to the framebuffer (2.18 KB, patch)
2016-08-26 18:31 UTC, Chris Wilson
no flags Details | Splinter Review
DMESG output from NUC6i7KYK (54.99 KB, text/plain)
2016-09-05 18:52 UTC, Chris
no flags Details
Fixes missing 44.1 kHz audio with 1920x1080p modes > 30Hz. (2.23 KB, patch)
2016-09-10 05:53 UTC, Mark Wilczynski
no flags Details | Splinter Review
Fixes skipping frames when running deep-color 1920x1080p @59.940 (390 bytes, patch)
2016-09-10 05:54 UTC, Mark Wilczynski
no flags Details | Splinter Review

Note You need to log in before you can comment on or make changes to this bug.
Description Mark Wilczynski 2016-08-22 20:17:36 UTC
This is either the same or similar bug that was originally reported here:
https://bugs.freedesktop.org/show_bug.cgi?id=75038

Additional information for my setup:

Hardware:
Asus Vivo Mini PC using Celeron 2957U
Samsung PN51F8500 TV directly connected via HDMI

The only difference for me is that I wasn't seeing the bug until I upgraded kernel from 4.1 to 4.3.  All future kernels up to 4.7 are also broken for me.

44.1 Khz output over HDMI works fine on this TV from a Raspberry Pi and also worked fine on the older kernels so I don't think it's the TV at fault.  Audio also works fine when I boot this computer into Windows 10 and use the same 1920x1080p @59.94Hz screen mode and 44.1 or 48 kHz audio.  44.1 Khz also works in Linux if I use my Denon AVR instead of the TV speakers (regardless of kernel).

Debug log from Linux 4.7.0:
http://sprunge.us/PPDj

Debug log from older Linux 4.1.10 (for reference when audio was still working):
http://sprunge.us/adcM
Comment 1 Mark Wilczynski 2016-08-22 20:53:12 UTC
These lines look suspect to me:

4.7.0 kernel:
[   19.311673] [drm:drm_mode_debug_printmodeline] Modeline 0:"" 0 148352 1920 2008 2052 2200 1080 1084 1089 1125 0x0 0x5
[   19.311678] [drm:intel_dump_crtc_timings] crtc timings: 148352 1920 2008 2052 2200 1080 1084 1089 1125, type: 0x0 flags: 0x5
[   19.311681] [drm:intel_dump_pipe_config] port clock: 222528

4.1.10 kernel:
[   13.995054] [drm:drm_mode_debug_printmodeline] Modeline 0:"" 0 148352 1920 2008 2052 2200 1080 1084 1089 1125 0x0 0x5
[   13.995057] [drm:intel_dump_pipe_config] adjusted mode:
[   13.995062] [drm:drm_mode_debug_printmodeline] Modeline 0:"" 0 148352 1920 2008 2052 2200 1080 1084 1089 1125 0x0 0x5
[   13.995068] [drm:intel_dump_crtc_timings] crtc timings: 148352 1920 2008 2052 2200 1080 1084 1089 1125, type: 0x0 flags: 0x5
[   13.995071] [drm:intel_dump_pipe_config] port clock: 148352

As you can see, the port clock of 222528 is probably wrong on the newer kernel.  On the older kernel, it matches the pixel clock of 148352 as requested by the Modeline.  Both logs are when screen was set to 1920x1080p @59.94 refresh.  Incidentally, I also noticed skipping video frames approximately every 20 seconds when playing back 59.940 fps video files on the 4.7 kernel.  No skipped video frames on 4.1.10.
Comment 2 Chris Wilson 2016-08-23 19:07:19 UTC
The port clock is right, it's chosen a 12 bits per channel pipe:

[    0.950689] [drm:intel_hdmi_compute_config] picking bpc to 12 for HDMI output
[    0.950690] [drm:intel_hdmi_compute_config] forcing pipe bpc to 36 for HDMI
[    0.950693] [drm:intel_modeset_pipe_config] hw max bpp: 36, pipe bpp: 36, dithering: 0

which accounts for the 50% increase in bw required. As to why it picks 36-bpp, there doesn't seem to be a good reason just that it is picking the highest supported data rate. I would have expected it to filter the bpp to be no larger than the incoming framebuffer (but there might some argument that increased precision improve gamma).

HDMI Audio works by using spare bandwidth in the vblank. At 222.5MHz, there isn't much spare - but there should be more than 50KHz! (More like 2.5MHz.)

Fwiw,

diff --git a/drivers/gpu/drm/i915/intel_hdmi.c b/drivers/gpu/drm/i915/intel_hdmi.c
index 1a116a6..5c1be29 100644
--- a/drivers/gpu/drm/i915/intel_hdmi.c
+++ b/drivers/gpu/drm/i915/intel_hdmi.c
@@ -1345,6 +1345,7 @@ bool intel_hdmi_compute_config(struct intel_encoder *encoder,
         * within limits.
         */
        if (pipe_config->pipe_bpp > 8*3 && pipe_config->has_hdmi_sink &&
+           !pipe_config->has_audio &&
            hdmi_port_clock_valid(intel_hdmi, clock_12bpc, true) == MODE_OK &&
            hdmi_12bpc_possible(pipe_config)) {
                DRM_DEBUG_KMS("picking bpc to 12 for HDMI output\n");
Comment 3 Mark Wilczynski 2016-08-23 20:08:14 UTC
I can't tell from these logs if the N / CTS values that are used to scale the pixel clock in order to specify audio sampling rate are correct.  Is there a way to verify that those constants have been adjusted correctly for the 50% increased clock rate?
Comment 4 Mark Wilczynski 2016-08-24 09:26:02 UTC
This is strange.  I built my Linux distribution (LibreElec) from scratch using their latest master and can't reproduce the audio bug.

I'm still seeing lots of skipped video frames indicating there is some kind of mismatch between audio and video clocks.  If I disable the 12 bit color support using your patch, the skipping frames disappear.  Audio continues to work regardless.

Will do some more digging to figure out why the regular publicly available nightly builds of LibreElec and OpenElec have broken audio on my system.
Comment 5 Mark Wilczynski 2016-08-25 03:24:09 UTC
Figured out why I couldn't reproduce the bug.  Had nothing to do with the Linux distro.  I had the HDMI signal from the PC passing through my Denon AVR on its way to the TV.  The AVR was turned off and in "pass-through" mode so I didn't think it would have any effect.  But this connection made the TV speakers work fine for 44.1 kHz audio.  After looking at the logs, I realized the AVR was replacing the TV EDID and forcing the HDMI signal to 24-bit color from the previous 36-bit.

Once I connected the PC directly to the TV (without AVR) I was back to broken 44.1 kHz audio regardless of distro.

Applying your 4.7 kernel patch to disable 12 bpc, fixes the 44.1 kHz audio on this TV.  That explains why the 8 bpc EDID of the AVR also fixed the problem.

For some reason this TV does not work correctly when sending 36-bit color signal over HDMI.  Unfortunately, I don't have another display available with a 36-bit color EDID.  All the rest report max 24-bit.  I suspect this might be why not many other users have reported this problem.

I did some experimentation by hard-coding a custom 'N' value which in theory was correct for a 222528 TMDS.  This fixed the audio problem with 36-bit video but file playback remained jumpy as a result of the skipping video frames.  Either the video pixel clock is running slightly too slow or the audio clock is running slightly too fast compared to 148352 TMDS.  Maybe the CTS value is still wrong but I wasn't sure hot to set that.  I guess it's derived automatically from the pre-defined pixel clock values stored in the AUD_CONFIG register?
Comment 7 Mark Wilczynski 2016-08-26 05:35:17 UTC
Had to merge the patch manually into the 4.7 kernel driver files because of some additional dependencies you're using.  It didn't make any difference for this bug.

It's probably going to be too difficult to merge any complex patches unless I somehow switch to one of your custom kernel builds that has all the code you're using.  Someone would need to guide me through that process.
Comment 8 Jani Nikula 2016-08-26 11:53:13 UTC
It is largely a prerequisite in the upstream bugzilla that you're able to run latest kernels and try patches.

https://kernelnewbies.org/KernelBuild and your distro's relevant pages might be helpful, but this bugzilla is not the support channel for kernel building.
Comment 9 Peter Frühberger 2016-08-26 17:49:27 UTC
@Jani: It's totally not clear to which base this patch corresponds. It neither applies on 4.8-rc3 nor on intel-drm-nightly.

I think examining intel dev trees and guessing which one to base it on is not a requirement for kernel newbies, right?
Comment 10 Chris Wilson 2016-08-26 17:56:13 UTC
The patches applied to dinq cleanly, I've put them here:

https://cgit.freedesktop.org/~ickle/linux-2.6/log/?h=hdmi-audio
Comment 11 Chris Wilson 2016-08-26 18:31:50 UTC
Created attachment 126057 [details] [review]
Limit the depth of the display pipeline to the framebuffer

Compile tested only!
Comment 12 Mark Wilczynski 2016-08-27 02:30:27 UTC
I'm primarily a Windows developer so I temporarily moved my testing over there.  The Windows 10 results were surprising.

When I played a 44.1 kHz audio test while running a full-screen-exclusive D3D app with 36-bit scan-out, the sound DID NOT work.  Same results as in Linux.  48 kHz audio worked fine with 36-bit color output.  One difference was that I didn't see any dropped or skipped video frames like I do in Linux with 36-bit color enabled.  This means the audio/video clocks are in proper sync for 59.940 content and 59.940 screen mode.

So either 36-bit color at 1920x1080@60 + 44.1 kHz audio is broken in both Windows and Linux or this TV does not accept that combination despite listing it in the EDID.  When I plug it into my Denon AVR, everything works fine using 36-bit color in Windows but it's no longer using the TV internal speakers.  The AVR reports that it is receiving 36-bit color and 44.1 kHz audio so I assume it is still passing 36-bit color to the TV.

I will try to get a second opinion by using a non-Intel GPU with this TV to see if it has the same problem.
Comment 13 Mark Wilczynski 2016-08-27 03:38:17 UTC
Did some more Windows testing using an Nvidia GeForce 210.  No problem connected directly to the TV.  36-bit Color and 44.1 kHz audio work fine using the TV speakers.

So it looks like Intel has a hardware or software issue when it comes to generating 100% compatible signals for this HDMI setting.  As you can see in the original bug, this issue also affects people with other TVs and/or AVR.

Maybe you can also pass this bug onto the Windows driver team since I don't know how to access their bug tracker.
Comment 14 Mark Wilczynski 2016-08-27 21:36:16 UTC
(In reply to Chris Wilson from comment #11)
> Created attachment 126057 [details] [review] [review]
> Limit the depth of the display pipeline to the framebuffer
> 
> Compile tested only!

If I understand your intent correctly, I think there's a logic bug in this patch.

+	if (plane_state->fb->depth > bpp)
+		bpp = 8*3;

Shouldn't that say:

+	if (plane_state->fb->depth < bpp)
+		bpp = 8*3;

That is if the FB bit depth is less than the supported bpp of the display (36-bit in my case), then clamp the bpp to that of the frame buffer (24-bit).
Comment 15 Chris Wilson 2016-08-28 08:38:31 UTC
Yes, unfortunately the test is backwards.
Comment 16 Mark Wilczynski 2016-09-01 05:50:25 UTC
I figured out why I was seeing video frame skipping when playing 59.940 fps content using 1920x1080@59.940 video mode.

When the pipeline is configured for regular 8-bit output, the refresh rate measures 59.94005994.  This is virtually perfect for the requested modeline:

 Modeline 0:"" 0 148352 1920 2008 2052 2200 1080 1084 1089 1125 0x0 0x5

When the pipeline is configured for 12=bit output, the refresh rate measures 59.89304812.  This causes a frame to be skipped every 21.2 seconds (1/(59.94005994-59.89304812)) in order to stay in sync with the expected 59.940 rate.

I measured the actual vsync rate using the GPU counters in the PIPE_FRMCNT and PIPE_FRMTMSTMP registers.

In Windows 10, the refresh rate is correct at both 8-bit and 12-bit output so there must be some error in the Linux driver when setting up the pixel clock or its dividers.
Comment 17 Chris 2016-09-03 15:39:40 UTC
This patch causes issue with my NUC6i7KYK and OLED 65E6 TV. It causes the TV to say "no signal" when ever I start a video, any video.

The patch was included in LibreELEC beta.

Here are links to software used and how patch was implemented:
http://nmacleod.com/public/oebuild/patches/00_fix_chris_wilson_126057.txt
https://bugs.freedesktop.org/show_bug.cgi?id=97442#c15
http://forum.kodi.tv/showthread.php?tid=269815&pid=2403575#pid2403575
http://milhouse.libreelec.tv/builds/master/Generic/LibreELEC-Generic.x86_64-8.0-Milhouse-20160829015023-%230828-g7cef50d.tar
Comment 18 Chris 2016-09-03 15:45:32 UTC
Sorry. Forgot to add that it has been confirmed by me that after they removed the patch everything worked just fine. As before.
Comment 19 Chris 2016-09-03 19:48:12 UTC
DMESG logs:
http://sprunge.us/IjiQ
Comment 20 Jani Nikula 2016-09-05 15:07:38 UTC
(In reply to Chris from comment #19)
> DMESG logs:
> http://sprunge.us/IjiQ

Please always attach logs to the bugzilla. Thanks.
Comment 21 Chris 2016-09-05 18:52:57 UTC
Created attachment 126229 [details]
DMESG output from NUC6i7KYK
Comment 22 Peter Frühberger 2016-09-08 14:43:41 UTC
After discussing with vsyrjala on IRC, could you please share your current findings / implementation approach concerning the 12-bit modes and audio in here?
Comment 23 Mark Wilczynski 2016-09-09 17:16:30 UTC
(In reply to Peter Frühberger from comment #22)
> After discussing with vsyrjala on IRC, could you please share your current
> findings / implementation approach concerning the 12-bit modes and audio in
> here?

My fix for the video frame skipping every 21 seconds looks good but I need more time to test the 44.1 kHz audio solution.  Don't have much content in that format but want to make sure there are no audio drop-outs.  Quick 10 minute test sounded fine.  Also need to try it on my AVR and other TV.  Hope to get my patches posted here this weekend.
  
I suspect these bugs also affect some 4K/UHD TV output modes but I can't work on those without a 4K TV.  I only fixed 1920x1080p @50, @60, and @59.940 Hz.
Comment 24 Peter Frühberger 2016-09-09 21:16:17 UTC
@Mark: Can you please post what you already have, so that the intel-devs responsible for this bugreport can support you and this process can get up to speed?
Comment 25 Mark Wilczynski 2016-09-10 05:50:59 UTC
From the HDMI spec, we know that the audio frequency is determined via regeneration from the TMDS clock using the N and CTS constants.

The formula is: 128 * fS = fTMDS_clock * N / CTS

N is restricted to:

128 * fS / 1500Hz ≤ N ≤ 128 * fS / 300Hz

We need to pass correct N and CTS values to the display via audio clock regeneration packets.  Tracing through the driver code I found that it never actually sets N or CTS anywhere.  The AUD_CONFIG register always stays with the default N value of 0x7FA6 for HDMI.  The code only changes the "Pixel Clock HDMI" value of the AUD_CONFIG register depending on pixel clock (max allowed value 148.5 Mhz).  I assume this means that under typical conditions, the N and CTS values are automatically computed by some hardware.  Since I can't see the actual N/CTS values sent to the sink device, I can only assume that they are wrong or out-of-range when pixel-clock != TMDS clock.  This only happens when running in deep-color modes like 36-bit color because TMDS is now 1.5x pixel_clock.

I fixed the missing 44.1 kHz audio on my TV by forcing specific N values into the AUD_CONFIG register when running in deep-color 1920x1080p modes over HDMI.  Since I don't know what CTS value is being sent, I had to play around with the N values to find ones that worked well for my TV.  I ended up needing a different N for 148.352 and 148.5 pixel clock modes.  See the first attached patch for intel_audio.c

The second bug with deep-color modes was skipping video frames as a result of the vertical refresh rate not matching the requested modeline pixel clock.  When requesting 148.352, the WRPLL_CTL register which holds the TMDS clock frequency was being set to 0xB01C0411.  When you decode the bits, and apply the formula from the docs ( TMDS Frequency = (Reference Frequency / Reference Divider) * (Feedback Divider / Post Divider)), you end up with:

(2700/17 * 100) * (28/4) * 2 = 222352.9411764706 port clock.

Dividing by 1.5 to convert from 12-bit to 8-bit pixels, the pixel clock ends up being 148235.2941176471

The actual vertical refresh rate ends up being: 148235.2941176471 * 1000 / (2200 * 1125 total pixels per frame) = 59.8930481283422 Hz.

59.89 is exactly the incorrect refresh rate I measured in one of my earlier posts.  It should be 59.940.  The reason this happens is because there is not enough precision in the 148352 number that is passed as an integer throughout the driver code.  The actual number should be 148351.6483516484.  This round-off error is magnified in deep color modes because of the 1.5x scaling to 222528.  The correctly scaled TMDS clock should be 222525.  The function which computes clock multipliers and dividers has some presets for common/exact clocks like 222525 but not for 222528.  It ends up searching through various combinations of dividers and comes up with those n,r,p (28,17,4) values which produce 222352 TMDS clock.

I ended up fixing this bug by patching intel_hdmi.c so that it replaces the truncated port clock with the full precision value of 222525.  This ends up producing n,r,p values of 150,91,4.  With those values inside WRPLL_CTL register, I get perfect 59.94005994005994 refresh rate.
Comment 26 Mark Wilczynski 2016-09-10 05:53:32 UTC
Created attachment 126450 [details] [review]
Fixes missing 44.1 kHz audio with 1920x1080p modes > 30Hz.

This patch only fixes the audio portion of the bug.
Comment 27 Mark Wilczynski 2016-09-10 05:54:33 UTC
Created attachment 126451 [details] [review]
Fixes skipping frames when running deep-color 1920x1080p @59.940

This patch only fixes the skipping video frames.
Comment 28 Mark Wilczynski 2016-09-10 06:03:36 UTC
Both patches were tested only on stock 4.7 kernel.  The intel_audio.c patch will probably not merge cleanly into newer kernels because of other changes and patches you guys made.  The main ideas should be easy to port.

Tested on TV and AVR for about 1 hour each when running in 36-bit color and 44.1 kHz audio at 1920x1080p @59.940.  I did some regression testing with 48 kHz audio and 1920x1080p @60 and @50 to make sure they still worked correctly.

8-bit output modes should not be affected by these patches.
Comment 29 Jani Nikula 2016-09-13 10:49:03 UTC
See also https://patchwork.freedesktop.org/series/11252/
Comment 30 Mark Wilczynski 2016-09-13 19:36:28 UTC
(In reply to Jani Nikula from comment #29)
> See also https://patchwork.freedesktop.org/series/11252/

That patch is similar to my solution.  It just needs to be expanded to deal with a couple high-clock 1920x1080p modes instead of just 4K resolutions.

The incorrect TMDS clock for 1920x1080@59.940 is an unrelated bug but should be addressed as well because it may affect the selection of proper N values required to fix 44.1 kHz audio.

BTW, the CTS (or M) values in my patched table are not correct.  I didn't fill those in because the code didn't use them for HDMI.  Only the N value is used.
Comment 31 Libin Yang 2016-09-23 06:30:50 UTC
(In reply to Mark Wilczynski from comment #30)
> (In reply to Jani Nikula from comment #29)
> > See also https://patchwork.freedesktop.org/series/11252/
> 
> That patch is similar to my solution.  It just needs to be expanded to deal
> with a couple high-clock 1920x1080p modes instead of just 4K resolutions.
> 
> The incorrect TMDS clock for 1920x1080@59.940 is an unrelated bug but should
> be addressed as well because it may affect the selection of proper N values
> required to fix 44.1 kHz audio.
> 
> BTW, the CTS (or M) values in my patched table are not correct.  I didn't
> fill those in because the code didn't use them for HDMI.  Only the N value
> is used.

Yes, we didn't use CTS for HDMI. We found if N is set correctly, HW can handle it automatically. To reduce the bugs introduced by sw, we skipped setting the CTS value. However we set the M value for DP now.

For this bug, I'm thinking whether we need calculate the n/cts/m value automatically instead of using the table as the clock may vary with mode?

Regards,
Libin
Comment 32 Jani Nikula 2016-10-26 09:45:37 UTC
Please try current drm-intel-nightly branch of http://cgit.freedesktop.org/drm-intel
Comment 33 keqiao 2017-02-03 02:42:37 UTC
This issue can not be reproduced with latest drm-tip branch(0f01216949002d20b9dc6d300c82df5ffa59e9a7), I tried different refresh rate(30Hz,50Hz,60Hz), audio with 44.1 KHz working fine under 1920x1080P resolution. 

hi Mark, 
Please help to retest this bug and close it if this issue is also gone on your side.

Thanks~
Comment 34 Ricardo 2017-03-03 17:14:50 UTC
Mark it was confirm the issue has been resoled, I'm changing the status of this bug to resolved. 

try to reproduce, if the issue is fixed please set but to closed, if the bug is back please set the bug to reopen and attach recent logs
Comment 35 keqiao 2017-03-05 11:38:14 UTC
(In reply to Ricardo from comment #34)
> Mark it was confirm the issue has been resoled, I'm changing the status of
> this bug to resolved. 
> 
> try to reproduce, if the issue is fixed please set but to closed, if the bug
> is back please set the bug to reopen and attach recent logs

Thanks Richardo for your confirmation.


Use of freedesktop.org services, including Bugzilla, is subject to our Code of Conduct. How we collect and use information is described in our Privacy Policy.