Bug 107476

commit 55bc60db5988c8366751d3d04dd690698a53412c
Author: Ville Syrjälä <ville.syrjala@linux.intel.com>
Date:   Thu Jan 17 16:31:29 2013 +0200

    drm/i915: Add "Automatic" mode for the "Broadcast RGB" property
    
    Add a new "Automatic" mode to the "Broadcast RGB" range property.
    When selected the driver automagically selects between full range and
    limited range output.
    
    Based on CEA-861 [1] guidelines, limited range output is selected if the
    mode is a CEA mode, except 640x480. Otherwise full range output is used.
    Additionally DVI monitors should most likely default to full range
    always.
    
    As per DP1.2a [2] DisplayPort should always use full range for 18bpp, and
    otherwise will follow CEA-861 rules.
    
    NOTE: The default value for the property will now be "Automatic"
    so some people may be affected in case they're relying on the
    current full range default.
    
    [1] CEA-861-E - 5.1 Default Encoding Parameters
    [2] VESA DisplayPort Ver.1.2a - 5.1.1.1 Video Colorimetry
    
    v2: Use has_hdmi_sink to check if a HDMI monitor is present
    v3: Add information about relevant spec chapters
    
    Reviewed-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
    Signed-off-by: Ville Syrjälä <ville.syrjala@linux.intel.com>
    Signed-off-by: Daniel Vetter <daniel.vetter@ffwll.ch>

My apologies, this indeed seems to be the case in the VESA documentation. But I am unsure why the function call "drm_default_rgb_quant_range(adjusted_mode)" seems to always return the value corresponding to "HDMI_QUANTIZATION_RANGE_LIMITED" on every DisplayPort connector of each monitor I try. That is perhaps the real question. I get the code conforms to spec, what I don't get is why I always end up with washed out limited RGB when I should be getting full RGB. It's not my computer, it has native DP 1.2 ports connected to the HD 630 i7-7700 iGPU. I seem to be experiencing the same problem as others in this active bug report https://bugs.freedesktop.org/show_bug.cgi?id=100023

Nicholas, on that other bugzilla entry you mentined that with same monitor, same machine the ranges are differente with different connectors (DP vs HDMI), right?
If, so, could you provide logs of both cases?

(In reply to Rodrigo Vivi from comment #3)
> Nicholas, on that other bugzilla entry you mentined that with same monitor,
> same machine the ranges are differente with different connectors (DP vs
> HDMI), right?
> If, so, could you provide logs of both cases?

Rodrigo, thank you for your consideration and for moving the discussion here, as I agree the primary issue is related to incorrect automatic range detection for DisplayPort. Yes, you are right: on the same machine, HDMI range detection works correctly on every display I have tried, but never works correctly with DisplayPort (on Linux, I should add. Intel's Windows drivers always correctly select full-range RGB over DisplayPort for the displays I have tried).

Michal's comment from https://bugs.freedesktop.org/show_bug.cgi?id=100023 is useful here because I think it summarizes the issue quite well: 
(In reply to Michał Kopeć from comment 21)
> DisplayPort output defaults to 16-235 color range, while any other outputs
> on the laptop will correctly output 0-255 full RGB. There is currently no
> way to set the driver to default to Full RGB regardless of what the display
> EDID is.

In accordance with https://01.org/linuxgraphics/documentation/how-report-bugs, I can provide the information needed. As this is a 'DRM Kernel' bug, I will provide logs from my system running the latest drm-tip branch of the kernel.

Created attachment 141017 [details]
dmesg-dp

The following information is for the DisplayPort connection which incorrectly defaults to limited range RGB 16-235 on the same monitor (Samsung CF591) and same machine. The latest drm-tip kernel with the parameters "drm.debug=0x1e log_buf_len=10M" was used. Attached file "dmesg-dp.txt" contains the full output of dmesg using DisplayPort truncated at one minute after boot (the file is large and most of the useful i915 debug info is much closer to the beginning of the file). "xrandr --verbose" output is also attached as "xrandr-dp.txt". 

-- Occurrence: Color range detection is never correct over DP, always defaults to limited-range RGB
-- Chipset: Intel® Core™ i7-7700T CPU using Intel® HD Graphics 630 (Kaby Lake GT2)
-- System architecture: x86_64
-- Kernel version (drm-tip) 4.18.0-994-generic
-- Linux distribution: Ubuntu 18.04.1 LTS (also confirmed on Fedora 28, appears distro-independent)
-- Machine: HP Elitedesk 800 G3 DM 35W
-- Display connector: DisplayPort

Created attachment 141018 [details]
xrandr-dp

Created attachment 141019 [details]
dmesg-hdmi

The following information is for the HDMI connection which correctly defaults to full-range RGB 0-255 on the same monitor (Samsung CF591) and same machine. The latest drm-tip kernel with the parameters "drm.debug=0x1e log_buf_len=10M" was used. Attached file "dmesg-hdmi.txt" contains the full output of dmesg using HDMI truncated at one minute after boot (the file is large and most of the useful i915 debug info is much closer to the beginning of the file). "xrandr --verbose" output is also attached as "xrandr-hdmi.txt".  

-- Occurrence: Color range detection is always correct over HDMI
-- Chipset: Intel® Core™ i7-7700T CPU using Intel® HD Graphics 630 (Kaby Lake GT2)
-- System architecture: x86_64
-- Kernel version (drm-tip) 4.18.0-994-generic
-- Linux distribution: Ubuntu 18.04.1 LTS (also confirmed on Fedora 28, appears distro-independent)
-- Machine: HP Elitedesk 800 G3 DM 35W
-- Display connector: HDMI

Created attachment 141020 [details]
xrandr-hdmi

Created attachment 141021 [details]
dmesg-lspcon

This limited default RGB problem extends to DP-LSPCON HDMI connections, which is the scenario many of those affected are also posting about. My computer happens to have a full-size 'native-looking' HDMI port option board attached to the motherboard through LSPCON. This helpful information from Intel i915 dev Imre Deak cleared up my confusion about why a hardware HDMI port is recognized as a DisplayPort (DP-3 in my case): "There are two ways to connect HDMI to the APL RVPs: via the DDI1 DP++ plug with an DP->HDMI dongle, or via the DDI0 HDMI plug which is connected to the SoC through the LSPCON converter. You seem to be using the second scenario with LSPCON being in the protocol converter mode (configured as such by BIOS). In that case the connection will show up as a DP connector." Anyway, it appears that the incorrect color mode selection extends to DP-LSPCON.

The following information is for the DP-LSPCON to HDMI connection which, like regular DisplayPort, incorrectly defaults to limited range RGB 16-235 on the same monitor (Samsung CF591) and same machine. The latest drm-tip kernel with the parameters "drm.debug=0x1e log_buf_len=10M" was used. Attached file "dmesg-lspcon-hdmi.txt" contains the full output of dmesg using HDMI truncated at one minute after boot (the file is large and most of the useful i915 debug info is much closer to the beginning of the file). "xrandr --verbose" output is also attached as "xrandr-lspcon-hdmi.txt".  

-- Occurrence: Color range detection is never correct over DP-LSPCON to HDMI, always defaults to limited-range RGB
-- Chipset: Intel® Core™ i7-7700T CPU using Intel® HD Graphics 630 (Kaby Lake GT2)
-- System architecture: x86_64
-- Kernel version (drm-tip) 4.18.0-994-generic
-- Linux distribution: Ubuntu 18.04.1 LTS (also confirmed on Fedora 28, appears distro-independent)
-- Machine: HP Elitedesk 800 G3 DM 35W
-- Display connector: DP-LSPCON to HDMI

Created attachment 141022 [details]
xrandr-lspcon

That should be all the information necessary. Thank you for looking into this, I'm honestly not sure what's going wrong with DP and DP-LSPCON automatic color space selection.

Or rather, I should correct my previous comment and use the phrase "automatic RGB color range detection" to avoid ambiguity. The terminology gets a bit confusing for me. I trust that the problem is clearer now, given the information provided.

But I am unsure why the function call "drm_default_rgb_quant_range(adjusted_mode)" seems to always return the value corresponding to "HDMI_QUANTIZATION_RANGE_LIMITED" on every DisplayPort connector of each monitor I try.

I told you the reason already. It's because it determines the range to be used base on the "mode" of the monitor. Here mode means resolution / refresh rate combinations.

This whole mess didn't started by Intel, but VESA, because they wanted to *replace/succeed* HDMI (completely, not in the alt mode sense). As HDMI was proprosed by the big bosses in AV world (who produces bluray player, AV Amps and such), they have to make limited range as their first class citizen because of historical/technical reasons.

The stupidity started when VESA thought DP could really become domination of both the AV world and the computer/general-purpose world by ignoring the technical and finanical reality. They simply copy the part of HDMI that try to compatible with the computer world as much as possible instead of actually having the spec make sense to the world it should have targeted. That's why I called DP some cheap braindead copycat work.

Then the monitor vendors started to produce monitors that has a CEA mode as optimal mode because of the popularity of "Full HD", while at the same time they didn't care much about color range (vendors are always lazy and ignorant about spec conformance, and to be fair limited range simply doesn't make sense in devices that are supposed to be general-purpose instead "av-specific").

Then Linux-part Intel decided that they should be spec-conforming no matter how ridiculous the spec is. Then it put the attribute/magic in kernel. Then it decided that the only interface to override it should be in X instead of in kernel. Tada, here comes the shit.

And end of story, coz no one would give a damn and they just continue to earn their salary for nothing (or worse), or move on to ruin the world even further in other company/organization for a even higher pay.

Tom, please do not add disrespectful comments here like the last sentence of comment #13 in particular. Much about what you pointed out regarding the oddity of the VESA spec may be the case, but at the very least allow for the fact Intel is actually looking into the issue. Intel's developers are doing what they think is best under the circumstances, as automatic mode is needed by some AV devices. I respectfully ask of you to 'stand down' before this entire thing is closed and nothing gets changed. I am trying my level best to work with what we have here constructively.

Please adhere to the FDO CoC https://www.freedesktop.org/wiki/CodeOfConduct/. Thank you.

Regarding the topic, "DisplayPort and LSPCON always defaults to incorrect limited-range RGB", do you observe limited-range RGB on non-CEA modes? See [1] for a table of CEA modes.

[1] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/gpu/drm/drm_edid.c#n681

The only displays I have available to test natively output 1920x1080@60Hz which I suppose matches the entry in the table /* 0x52 - 1920x1080@60Hz */. I should point out that hundreds, if not thousands of computer monitors (not televisions or av equipment) output 1920x1080@60Hz. The odd thing here is that on each 1920x1080@60Hz monitor, over HDMI, full-range RGB 0-255 is always correctly used, but over DisplayPort, limited-range RGB 16-235 is always incorrectly used. 

That doesn't make sense, especially for displays with greater color bit depth like commonplace 24bpp displays. These displays don't expect anything other than full range color, and look awful without it. Intel's Windows drivers for iGPUs correctly default to full range RGB on these displays over DisplayPort. AMD and NVIDIA graphics cards drivers also correctly default to full range RGB output for computer monitors on Windows and Linux. The discrepancy between Intel's i915 driver module selecting different color ranges over HDMI and DP/LSPCON is the problem here. 

The vast majority of monitors on the market also fit the modes in the table you mentioned, even those with 4k HDR, so the logic behind the same 24bpp+ displays *not* automatically receiving full-range color appropriately from Intel's Linux driver over DP and LSPCON doesn't make sense, especially when they do correctly receive full-range color over HDMI. I don't think this issue is about CEA/VESA specifications. Rather, it concerns the subpar reality of incorrect, limited-range, washed-out color on non-av equipment (computer monitors, namely) over DP and LSPCON.

Created attachment 141060 [details] [review]
drm/i915: set DP Main Stream Attribute for color range on DDI platforms

Please try this patch on top of an otherwise unmodified kernel. Untested, but I'm feeling optimistic enough about it that I wrote a proper commit message. Fingers crossed.

(In reply to Jani Nikula from comment #17)
> Created attachment 141060 [details] [review] [review]
> drm/i915: set DP Main Stream Attribute for color range on DDI platforms
> 
> Please try this patch on top of an otherwise unmodified kernel. Untested,
> but I'm feeling optimistic enough about it that I wrote a proper commit
> message. Fingers crossed.

Jani, this really works! Fantastic! I applied the patch directly onto today's unmodified drm-tip kernel from git://anongit.freedesktop.org/drm-tip and it completely works now! I tried this on three different DisplayPort monitors, and I am delighted to report that the problem is resolved. Full-range RGB color is appropriately selected on my 24bpp True Color monitors over DisplayPort. Your optimism was completely warranted, this totally fixes my DisplayPort woes on Linux! You have my sincerest praise, I'm so glad you resolved this issue. 

And to think, that "After all these years, the fix boils down to flipping one bit." I am absolutely overjoyed here. I would like to thank Rodrigo Vivi as well for his help. Thank you so very, very much for looking into this bug and fixing the unexpected root of the problem. I expect to see your fix applied upstream as soon as possible. :)

Jani Nikula's patch works great for native DisplayPort connections and doesn't negatively impact native HDMI connections either, which is fantastic. Unfortunately, I'm not having much luck with DP-LSPCON->HDMI. I am still unable to get automatic full-range color over DP-LSPCON->HDMI on my test monitors that work fine with native HDMI and native DP after the patch. I'm afraid I cannot accurately test LSPCON as I happen to be afflicted by this separate, unfortunate native-resolution bug: https://bugs.freedesktop.org/show_bug.cgi?id=107503 I will move the LSPCON discussion back to https://bugs.freedesktop.org/show_bug.cgi?id=100023 to be resolved separately, and edit the title of this bug to reflect the change, limiting the issue at hand to just native DP. 

The primary bug, and the one for which I originally filed, is native DisplayPort RGB automatic color-range detection on Intel integrated graphics, which has been resolved beautifully in Jani's patch. I can finally use my native DisplayPorts as intended. All things considered, the main problem was resolved effectively. Remaining concerns about LSPCON I will redirect to the other open bug thread, perhaps others will have different results over LSPCON with the patch? Overall, I think this was a *great* step forward. Automatic RGB color-range detection over native DisplayPort is finally working properly. Please merge Jani's patch fixing native DP color-range handling.

Thanks for testing! \o/

I submitted the patch for review, with the commit message amended about the LSPCON [1]. I expect this to get merged to Linus' upstream within a few weeks, trickling down to stable kernels after that.

Agreed on repurposing this bug report to be about native DP, and keeping bug 10023 open about LSPCON.

[1] http://patchwork.freedesktop.org/patch/msgid/20180814060001.18224-1-jani.nikula@intel.com

(In reply to Jani Nikula from comment #20)
> Agreed on repurposing this bug report to be about native DP, and keeping bug
> 10023 open about LSPCON.

Bug 100023.

Decreasing priority to high/major.

commit dc5977da99ea28094b8fa4e9bacbd29bedc41de5
Author: Jani Nikula <jani.nikula@intel.com>
Date:   Tue Aug 14 09:00:01 2018 +0300

    drm/i915: set DP Main Stream Attribute for color range on DDI platforms

I'm actually running into the very same problem, but with already patched kernel 4.18.16-200. I look into this several hours and seems to be similar problem.

With resolution 1920x1080 color range is trimmed, but with lover resolution (e.g. 1600x900), colors seems to be pretty fine.

Whats the next step, please? Should I reopen this one, or new one?
I can provide all necessary logs, etc, if needed.

Thank you.

> Whats the next step, please? Should I reopen this one, or new one?
> I can provide all necessary logs, etc, if needed.
There was some changes done recently related to "DisplayPort always output limited range RGB" can you please verify this issue with latest drm-tip or latest stable kernel i.e 4.19? This will provide more information about the bug. 
Latest drm-tip https://cgit.freedesktop.org/drm-tip

I did testing with latest Fedora Rawhide kernel (4.20) and problem still persists, unfortunately. Behavior is very same, as with 4.18 kernel.

Unfortunately 4.19 kernel is not in the repos, so I can't test it as stable. However, patch should be included in 4.20 already, if I understand correctly?

> However, patch should be included in 4.20 already, if I understand correctly?
Yes.

So that's mean, that this is another very similar bug, which is not covered in any patch. Whats the next step please? New ticket with same logs, as in this ticket?

I'll do my best, but I'm not developer, just simple user.

Thank you.

Can you attach the dmesg from 4.20 kernel with kernel parameters drm.debug=0x1e log_buf_len=4M. Dmesg from boot is needed.

I did requested, I'm attaching dmesg ouput after the boot. Hope I did it right, thank you.

Created attachment 142444 [details]
Dmesg.log - Ivan Vlk

I forgot the log :-)

(In reply to Ivan Vlk from comment #31)
> Created attachment 142444 [details]
> Dmesg.log - Ivan Vlk
> 
> I forgot the log :-)

Please file a new bug, and ensure you attach the dmesg from boot. The one here is incomplete.

Also, you seem to have dual GPU. Are you sure you're attaching the display to i915?

Anyway, let's not mess up this bug here, there was a clear bug that was fixed. If it doesn't work for you, you have something else.

I have created new bug report as requested: https://bugs.freedesktop.org/show_bug.cgi?id=108821

Attached new dmesg with 10MB buffer as requested and yes, I'm using Intel card.

Thank you.