Bug 110830

CC: Ben Skeggs

With a kernel with TU117 support, boot with

nouveau.config=NvChipset=0x167

And see if it Just Works (tm). (Note that the NvChipset thing is quite new, and is in the commit right before TU117 support... otherwise you could go down to TU102, which appears to be ~the same.)

I managed to build 5.2.0-0.rc3.git2.1.fc30.x86_64 on Fedora 30 and boot with overridden NvChipset (0x167). The nouveau module is loaded, the card is visible by vgaswitcheroo, but I cannot switch to it (nor force to turn it on).

> # lsmod | grep nouveau
> nouveau              2293760  1
> ttm                   122880  1 nouveau
> i2c_algo_bit           16384  2 i915,nouveau
> drm_kms_helper        225280  2 i915,nouveau
> mxm_wmi                16384  1 nouveau
> drm                   577536  15 drm_kms_helper,i915,ttm,nouveau
> video                  53248  2 i915,nouveau
> wmi                    36864  4 intel_wmi_thunderbolt,wmi_bmof,mxm_wmi,nouveau

> # cat /sys/kernel/debug/vgaswitcheroo/switch
> 0:IGD:+:Pwr:0000:00:02.0
> 1:DIS: :DynPwr:0000:01:00.0
> 2:DIS-Audio: :DynOff:0000:01:00.1

On:
> echo ON > /sys/kernel/debug/vgaswitcheroo/switch

there is no effect. On:

> echo DIS > /sys/kernel/debug/vgaswitcheroo/switch

I see an error in the system logs:
> kernel: vga_switcheroo: client 0 refused switch

The initialization logs on boot:
> kernel: MXM: GUID detected in BIOS
> kernel: ACPI Warning: \_SB.PCI0.GFX0._DSM: Argument #4 type mismatch - Found [Buffer], ACPI requires [Package] (20190509/nsargume>
> kernel: ACPI Warning: \_SB.PCI0.PEG0.PEGP._DSM: Argument #4 type mismatch - Found [Buffer], ACPI requires [Package] (20190509/nsa>
> kernel: pci 0000:01:00.0: optimus capabilities: enabled, status dynamic power, hda bios codec supported
> kernel: VGA switcheroo: detected Optimus DSM method \_SB_.PCI0.PEG0.PEGP handle
> kernel: nouveau: detected PR support, will not use DSM
> kernel: nouveau 0000:01:00.0: enabling device (0106 -> 0107)
> kernel: checking generic (90000000 7e9000) vs hw (a0000000 10000000)
> kernel: checking generic (90000000 7e9000) vs hw (b0000000 2000000)
> kernel: nouveau 0000:01:00.0: CHIPSET OVERRIDE: 168000a1 -> 167000a1
> kernel: nouveau 0000:01:00.0: NVIDIA TU117 (167000a1)
> kernel: nvme nvme0: missing or invalid SUBNQN field.
> kernel: nvme nvme0: Shutdown timeout set to 8 seconds
> kernel: nvme nvme0: 12/0/0 default/read/poll queues
> kernel: checking generic (90000000 7e9000) vs hw (90000000 10000000)
> kernel: fb0: switching to inteldrmfb from EFI VGA
> kernel: Console: switching to colour dummy device 80x25
> kernel: i915 0000:00:02.0: vgaarb: deactivate vga console
> kernel: nouveau 0000:01:00.0: bios: version 90.16.26.00.11
> kernel: nouveau 0000:01:00.0: fb: 6144 MiB GDDR6
> kernel:  nvme0n1: p1 p2 p3
> kernel: [drm] Supports vblank timestamp caching Rev 2 (21.10.2013).
> kernel: [drm] Driver supports precise vblank timestamp query.
> kernel: i915 0000:00:02.0: vgaarb: changed VGA decodes: olddecodes=io+mem,decodes=none:owns=io+mem
> kernel: [drm] Finished loading DMC firmware i915/kbl_dmc_ver1_04.bin (v1.4)
> systemd-udevd[578]: Using default interface naming scheme 'v240'.
> kernel: vga_switcheroo: enabled
> kernel: [TTM] Zone  kernel: Available graphics memory: 8042942 KiB
> kernel: [TTM] Zone   dma32: Available graphics memory: 2097152 KiB
> kernel: [TTM] Initializing pool allocator
> kernel: [TTM] Initializing DMA pool allocator
> kernel: nouveau 0000:01:00.0: DRM: VRAM: 6144 MiB
> kernel: nouveau 0000:01:00.0: DRM: GART: 536870912 MiB
> kernel: nouveau 0000:01:00.0: DRM: BIT table 'A' not found
> kernel: nouveau 0000:01:00.0: DRM: BIT table 'L' not found
> kernel: nouveau 0000:01:00.0: DRM: TMDS table version 2.0
> kernel: nouveau 0000:01:00.0: DRM: DCB version 4.1
> kernel: nouveau 0000:01:00.0: DRM: DCB outp 00: 02002f52 00020010
> kernel: nouveau 0000:01:00.0: DRM: DCB outp 01: 04814f76 04600010
> kernel: nouveau 0000:01:00.0: DRM: DCB outp 02: 04814f72 00020010
> kernel: nouveau 0000:01:00.0: DRM: DCB conn 02: 00010261
> kernel: nouveau 0000:01:00.0: DRM: DCB conn 04: 01000446
> kernel: nouveau 0000:01:00.0: DRM: failed to create kernel channel, -22
> kernel: nouveau 0000:01:00.0: DRM: MM: using COPY for buffer copies
> kernel: [drm] Supports vblank timestamp caching Rev 2 (21.10.2013).
> kernel: [drm] Driver supports precise vblank timestamp query.
> kernel: [drm] Initialized i915 1.6.0 20190417 for 0000:00:02.0 on minor 0
> kernel: [drm] Cannot find any crtc or sizes
> kernel: ACPI: Video Device [PEGP] (multi-head: no  rom: yes  post: no)
> kernel: input: Video Bus as /devices/LNXSYSTM:00/LNXSYBUS:00/PNP0A08:00/device:00/LNXVIDEO:00/input/input12
> kernel: [drm] Initialized nouveau 1.3.1 20120801 for 0000:01:00.0 on minor 1
> kernel: ACPI: Video Device [GFX0] (multi-head: yes  rom: no  post: no)
> kernel: input: Video Bus as /devices/LNXSYSTM:00/LNXSYBUS:00/PNP0A08:00/LNXVIDEO:01/input/input13
> kernel: fbcon: i915drmfb (fb0) is primary device
> kernel: Console: switching to colour frame buffer device 240x67
> kernel: [drm] Cannot find any crtc or sizes
> kernel: i915 0000:00:02.0: fb0: i915drmfb frame buffer device
> kernel: psmouse serio2: synaptics: queried max coordinates: x [..5658], y [..4722]
> kernel: [drm] Cannot find any crtc or sizes

There are also no reported providers in xrandr:
> $ xrandr --listproviders
> Providers: number: 0

I don't know if it is related to the chipset itself or to my laptop configuration. The same situation is with with 0x167 and 0x162.

Of course it didn't make sense to turn it on, as it was already turned on (DynPwr) in my case :) ). I can switch it off with vgaswitcheroo (or at least to be reported as DynOff - I will check it later with powertop) when no external output is connected. And what I missed in my previous comment - the external monitor works out-of-box, which is nice progress.

Later on I will try to play with DRI_PRIME to enable also rendering with the NVidia card.

What could be the side effects of:
> kernel: nouveau 0000:01:00.0: DRM: failed to create kernel channel, -22
?

And, in general, thanks for you suggestion with NvChipset :). If you would like to test anything else on that hardware, please let me know.

(In reply to Marcin Zajaczkowski from comment #4)
> Of course it didn't make sense to turn it on, as it was already turned on
> (DynPwr) in my case :) ). I can switch it off with vgaswitcheroo (or at
> least to be reported as DynOff - I will check it later with powertop) when
> no external output is connected. And what I missed in my previous comment -
> the external monitor works out-of-box, which is nice progress.
> 
> Later on I will try to play with DRI_PRIME to enable also rendering with the
> NVidia card.
> 
> What could be the side effects of:
> > kernel: nouveau 0000:01:00.0: DRM: failed to create kernel channel, -22
That's expected at this point. We don't have firmware available to enable acceleration yet. 
 Support for Turing is limited to modesetting currently. 

> ?
> 
> And, in general, thanks for you suggestion with NvChipset :). If you would
> like to test anything else on that hardware, please let me know.

(In reply to Marcin Zajaczkowski from comment #4)
> Of course it didn't make sense to turn it on, as it was already turned on
> (DynPwr) in my case :) ). I can switch it off with vgaswitcheroo (or at
> least to be reported as DynOff - I will check it later with powertop) when
> no external output is connected. And what I missed in my previous comment -
> the external monitor works out-of-box, which is nice progress.

vgaswitcheroo explicit control is for hard muxes. These were popular 2005 to 2010 or so. You just have 2 GPUs. vgaswitcheroo reports whether they're on or off, but that control is performed dynamically by the driver based on usage.

> 
> Later on I will try to play with DRI_PRIME to enable also rendering with the
> NVidia card.
> 
> What could be the side effects of:
> > kernel: nouveau 0000:01:00.0: DRM: failed to create kernel channel, -22
> ?

No acceleration. This means that DRI_PRIME will not do anything.

> There are also no reported providers in xrandr:
> > $ xrandr --listproviders
> > Providers: number: 0

This is incredibly odd -- there must always be at least 1! Are you running Xwayland or something? If so, the displays would be controlled through your wayland compositor. You can check that kms is working:

grep . /sys/class/drm/card*-*/status

You should see some card0-* and card1-* entries.

(In reply to Ilia Mirkin from comment #6)
> (In reply to Marcin Zajaczkowski from comment #4)
> > Of course it didn't make sense to turn it on, as it was already turned on
> > (DynPwr) in my case :) ). I can switch it off with vgaswitcheroo (or at
> > least to be reported as DynOff - I will check it later with powertop) when
> > no external output is connected. And what I missed in my previous comment -
> > the external monitor works out-of-box, which is nice progress.
> 
> vgaswitcheroo explicit control is for hard muxes. These were popular 2005 to
> 2010 or so. You just have 2 GPUs. vgaswitcheroo reports whether they're on
> or off, but that control is performed dynamically by the driver based on
> usage.

You are right. Afew seconds after an external monitor is disconnected the discrete card becomes DynOff.

I have a hard mux in my (quite) old Asus. I can use only the Intel card to save energy and the Nvidia card is used only occasionally for CUDA and gaming. I wonder why in multiple laptops nowadays the outputs are wired to one card limiting the aforementioned use case? What are the benefits of that (assuming the modern Intel can handle 3 displays enabled at the same time)?

> > There are also no reported providers in xrandr:
> > > $ xrandr --listproviders
> > > Providers: number: 0
> 
> This is incredibly odd -- there must always be at least 1! Are you running
> Xwayland or something? If so, the displays would be controlled through your
> wayland compositor. You can check that kms is working:

Yes, it was Wayland. With Xorg I see two providers:

> Providers: number : 2
> Provider 0: id: 0x6b cap: 0xf, Source Output, Sink Output, Source Offload, Sink Offload crtcs: 3 outputs: 3 associated providers: 1 name:modesetting
> Provider 1: id: 0x45 cap: 0x2, Sink Output crtcs: 4 outputs: 2 associated providers: 1 name:modesetting

Hmm, why there are 3 and 4 outputs associated to them? Is it possible that DP output is accessible with the Intel card only (I don't have a connector right here to test it organoleptically)?

> 
> grep . /sys/class/drm/card*-*/status
> 
> You should see some card0-* and card1-* entries.

Yes, I see different outputs.
> /sys/class/drm/card0-DP-1/status:disconnected
> /sys/class/drm/card0-eDP-1/status:connected
> /sys/class/drm/card0-HDMI-A-1/status:disconnected
> /sys/class/drm/card1-DP-2/status:disconnected
> /sys/class/drm/card1-HDMI-A-2/status:disconnected

I know about a lack of acceleration, but I wanted to test offloading with NVidia GPU. I doesn't seem to work:

> $ glxinfo | grep "OpenGL vendor string"
> OpenGL vendor string: Intel Open Source Technology Center
> $ DRI_PRIME=1 glxinfo | grep "OpenGL vendor string"
> libGL error: failed to create dri screen
> libGL error: failed to load driver: nouveau
> OpenGL vendor string: VMware, Inc.

Can it be specific to my graphic card or rather some Xorg/Wayland related stuff?

(In reply to Marcin Zajaczkowski from comment #7)
> > Providers: number : 2
> > Provider 0: id: 0x6b cap: 0xf, Source Output, Sink Output, Source Offload, Sink Offload crtcs: 3 outputs: 3 associated providers: 1 name:modesetting
> > Provider 1: id: 0x45 cap: 0x2, Sink Output crtcs: 4 outputs: 2 associated providers: 1 name:modesetting
> 
> Hmm, why there are 3 and 4 outputs associated to them? Is it possible that
> DP output is accessible with the Intel card only (I don't have a connector
> right here to test it organoleptically)?

"crtcs: 4; outputs: 2". All modern NVIDIA GPU's have 4 CRTC's. You can see the full list of outputs with that grep command you ran.

You can now slave the nvidia GPU's outputs to the primary (intel) GPU:

xrandr --setprovideroutputsource 1 0

which should allow you to drive on your NVIDIA GPU's outputs by configuring them in xrandr.

This should also be possible with a Wayland compositor, however the specifics will vary by compositor. Xwayland has no ability to control screen setup.

(In reply to Marcin Zajaczkowski from comment #8)
> I know about a lack of acceleration, but I wanted to test offloading with
> NVidia GPU. I doesn't seem to work:

Lack of acceleration = no opengl, or anything else which requires the GPU to execute things in the "graphics" engine.

(In reply to Ilia Mirkin from comment #9)
> (In reply to Marcin Zajaczkowski from comment #7)
> > > Providers: number : 2
> > > Provider 0: id: 0x6b cap: 0xf, Source Output, Sink Output, Source Offload, Sink Offload crtcs: 3 outputs: 3 associated providers: 1 name:modesetting
> > > Provider 1: id: 0x45 cap: 0x2, Sink Output crtcs: 4 outputs: 2 associated providers: 1 name:modesetting
> > 
> > Hmm, why there are 3 and 4 outputs associated to them? Is it possible that
> > DP output is accessible with the Intel card only (I don't have a connector
> > right here to test it organoleptically)?
> 
> "crtcs: 4; outputs: 2". All modern NVIDIA GPU's have 4 CRTC's. You can see
> the full list of outputs with that grep command you ran.
> 
> You can now slave the nvidia GPU's outputs to the primary (intel) GPU:
> 
> xrandr --setprovideroutputsource 1 0
> 
> which should allow you to drive on your NVIDIA GPU's outputs by configuring
> them in xrandr.

In lxqt (with Openbox) after that command and "xrandr --output HDMI-1-2 --auto --right-of eDP-1" the external monitor is turned on and I see a mouse cursor there, but no window is rendered.

In Gnome 3 (with xorg) the external monitor is detected automatically, but the effect is the same - black screen with a mouse cursor.

Can it be related to the fact that NVidia providers reports only "Sink Output", but not offloading?

> This should also be possible with a Wayland compositor, however the
> specifics will vary by compositor. Xwayland has no ability to control screen
> setup.

In Wayland the external monitor works out-of-box in Gnome 3. However, I have problem with bringing it to live with Xorg server (some tools I use don't like Wayland). Is it possible to force LXQT or Gnome 3 (on xorg) to render also on the external screen?

I've read about some issues with Windows Manager and "offloading", but though it only applies to OpenGL-based output with PRIME=1.

(In reply to Marcin Zajaczkowski from comment #10)
> (In reply to Ilia Mirkin from comment #9)
> > xrandr --setprovideroutputsource 1 0
> > 
> > which should allow you to drive on your NVIDIA GPU's outputs by configuring
> > them in xrandr.
> 
> In lxqt (with Openbox) after that command and "xrandr --output HDMI-1-2
> --auto --right-of eDP-1" the external monitor is turned on and I see a mouse
> cursor there, but no window is rendered.
> 
> In Gnome 3 (with xorg) the external monitor is detected automatically, but
> the effect is the same - black screen with a mouse cursor.

I've seen a bunch of reports of this recently. It's not any issue with nouveau kernel component itself, I think it's an Xorg issue. Try using a redirecting compositor. Or not using one. Or using modesetting ddx. Or using nouveau ddx.

The lack of acceleration on TU* might also be playing into this. Not sure.

> 
> Can it be related to the fact that NVidia providers reports only "Sink
> Output", but not offloading?
> 
> > This should also be possible with a Wayland compositor, however the
> > specifics will vary by compositor. Xwayland has no ability to control screen
> > setup.
> 
> In Wayland the external monitor works out-of-box in Gnome 3. However, I have
> problem with bringing it to live with Xorg server (some tools I use don't
> like Wayland). Is it possible to force LXQT or Gnome 3 (on xorg) to render
> also on the external screen?
> 
> I've read about some issues with Windows Manager and "offloading", but
> though it only applies to OpenGL-based output with PRIME=1.

Offloading is usually meant as "perform rendering on secondary GPU, display on primary GPU". This is what something like DRI_PRIME controls (i.e. which GPU performs rendering for a particular application).

You want the inverse -- you want your primary GPU to generate images to be displayed on a secondary GPUs outputs. In Xorg this is referred to as "reverse PRIME".

Thanks for your explanations.

In the meantime, I've tried (recommended) KWin with LXQT (both XRender nad OpenGL), but with no effects. I will try to explore it at the Xorg/Window Manager side (or stick to Wayland for awhile)...

However, I have one more question about the nouveau driver itself. I tried to switch from the "modesetting" driver to intel/nouveau in the xorg configuration. However, trying to define 'Driver "nouveau"' I end up with the error that device 167 is not known. I've seen you commit from January [1]. Is it just needed to add a new entry for my GTX 1660 in nv_driver.c? Anyway, do you think using nouveau_drv.so instead of "modesetting" can change anything?

[1] - https://cgit.freedesktop.org/nouveau/xf86-video-nouveau/commit/?id=2376d1ebf2d9a96bc2ebf21d53a9f9841ce5c15b

(In reply to Marcin Zajaczkowski from comment #12)
> [1]. Is it just needed to add a new entry for my GTX 1660 in nv_driver.c?

Mmmmaybe. You'd have to kill Accel for it too, since none of the other functions will work either.

> Anyway, do you think using nouveau_drv.so instead of "modesetting" can
> change anything?

Not in the reverse prime case -- it can only make things worse. IIRC nouveau_drv expects acceleration support for handling reverse prime.

To summarize the things. The offloading problem still needs to be examined at the software level (it works only in Wayland). However, there is a workaround. In the end, it turned out and in a model with GTX 1660 Ti, the mDP port is connected to the Intel card. The external monitor works fine with a mDP->HDMI adapter (with the NVidia card reported as DynOff).

@Ilia, thanks for your assistance!

Btw, it's too late for 5.2, but maybe GTX 1660 Ti will be natively supported with nouveau (although "nouveau.config=NvChipset=0x167" works fine as a workaround - probably the same stable as with GTX 1650 :) ).

Btw, for people with the same laptop and similar issues landing in that issue - I created a short summary how to solve some problems with Hyperbook NH5/Clevo NH55RCQ (including those with GeForce GTX 1660 Ti):
https://gist.github.com/szpak/71081b40217fb27c7a565b8c7b972067

-- GitLab Migration Automatic Message --

This bug has been migrated to freedesktop.org's GitLab instance and has been closed from further activity.

You can subscribe and participate further through the new bug through this link to our GitLab instance: https://gitlab.freedesktop.org/xorg/driver/xf86-video-nouveau/issues/488.