Bug 98502

If this is a regression can you bisect?

I'll try to bisect and see if I can find the offending commit, I'll post back when I have a result.

Are there any tests you'd like me to do or other information you'd like me to post?

After bisecting (hope I've done it right), the first bad commit is be239326aa4f9317d42ee07f0f51179c8b3d5b22

I've tried to revert this commit and test again but I haven't been able to do so.

Bisect log is attached.

Created attachment 127635 [details]
Bisect log

That's because the drmDevice API retrieves the device revision id.

IIRC that was something explicitly requested by Jammy and the only way to get the info is to parse ./config which wakes up the device.

AFAICT if using libudev one also needs to parse ./config since the kernel does not expose a revision_id file/info like the {subsystem_,}{device,vendor} ones.

I think we'd really want to change that [in the long term]. There was a patch for it, but seems like it never got sent/merged.

Short term ideas:
libdrm
 - cheat, don't parse ./config. Read the revision_id file and set to zero if missing. Default for all or toggle {config,revision_id} by envvar/API ?
 - roll revision_id-less API. Duplication of (already tad nasty) code :-(

mesa
 - go back to manual sysfs parsing in mesa
I'd like to avoid that if possible - makes BSD/other integration cumbersome, code duplication (?)


Fwiw, I'm leaning for the "cheat" since no open-source user-space (seems to) depend on the revision_id value, and aim for getting a kernel patch for revision_id upstreamed.

Opinions/other ideas ?

As a user, any band aid that will mask the problem until the proper long term fix is in place will be very much welcomed.

As to how it is implemented, I am not familiar with the mesa and/or libdrm code, if you think the cheat option will be the best stopgap measure I'll trust your judgement.

The kernel patch is out and should fit the mailing list archives in due time. 

Michel since Jammy is no longer around can you check (forward to AMDGPU-PRO team)  about the implications of the "cheat" route, please ?

I'm not sure what exactly you're asking me to do. Can you just ask on the amd-gfx mailing list directly?

Guys, the kernel (v4.10) has been updated to provide the revision field [1], at the same time libdrm (v2.4.75) has API which does not fetch it [2] and the mesa patches [3] to us the new API has been updated.

Sadly applying [3] triggers a libtool bug^Wfeature where linking fails on the Intel CI. Once we get a workaround for that I'll apply [3].


[1] https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=702ed3be1b1bf4dea05954168321741c0910c645
[2] https://cgit.freedesktop.org/mesa/drm/commit/?id=11687bf4180f7e21045ed9c4730533c40fe01ea5
[3] https://patchwork.freedesktop.org/series/19714/

I have the same problem: Qt5 wakes up dGPU on launch [1]. However, I believe that this is a kernel problem: the dGPU wakes up when a program tries to read from /sys/bus/pci/devices/<bus id>/config. This was not happening with 4.7 and when I use Arch's current LTS kernel (4.4) it also works fine [2].

I can't believe that this is intentional, because a) waking up dGPU on every program launch does not make any sense and b) when the dGPU is disable via acpi_call, any program hangs reading the same file. What puzzles me most is the following: I have two almost identical laptops (for this bug they can be treated as identical, I believe) and the second one, running Gentoo, is free from this problem with the same versions of kernel. Moreover, with kernel configured using config from the second one, the first one still shows the problematic behaviour. 

[1] https://bugs.archlinux.org/task/52452
[2] https://bugzilla.kernel.org/show_bug.cgi?id=194721

(In reply to Eugene Shalygin from comment #11)
> I have the same problem: Qt5 wakes up dGPU on launch [1]. However, I believe
> that this is a kernel problem: the dGPU wakes up when a program tries to
> read from /sys/bus/pci/devices/<bus id>/config. This was not happening with
> 4.7 and when I use Arch's current LTS kernel (4.4) it also works fine [2].

As soon as all the patches landed (from comment #10) there's no need to check /sys/bus/pci/devices/<bus id>/config anymore, because there's a new API to get that info that doesn't need to wake up the device and the problem goes away. But right now the mesa patches are not applied as far as I can tell.

> 
> I can't believe that this is intentional [...]

It's not :-)

(In reply to Tobias Droste from comment #12)
> (In reply to Eugene Shalygin from comment #11)
> > I have the same problem: Qt5 wakes up dGPU on launch [1]. However, I believe
> > that this is a kernel problem: the dGPU wakes up when a program tries to
> > read from /sys/bus/pci/devices/<bus id>/config. This was not happening with
> > 4.7 and when I use Arch's current LTS kernel (4.4) it also works fine [2].
> 
> As soon as all the patches landed (from comment #10) there's no need to
> check /sys/bus/pci/devices/<bus id>/config anymore, because there's a new
> API to get that info that doesn't need to wake up the device and the problem
> goes away. But right now the mesa patches are not applied as far as I can
> tell.
> 

Aren't most patches in mesa and libdrm by now? At least the versions provided by Arch already seem to have the patches (or at least some of them).

The only missing piece seems to be the kernel patch which should land once kernel 4.10 leaves testing.

However 4.9 does not have the patch and as it an LTS release it will stick around for a while, it would be nice if the sysfs patch was also included in 4.9.

> > 
> > I can't believe that this is intentional [...]
> 
> It's not :-)

(In reply to Tobias Droste from comment #12)
> (In reply to Eugene Shalygin from comment #11)
> > I have the same problem: Qt5 wakes up dGPU on launch [1]. However, I believe
> > that this is a kernel problem: the dGPU wakes up when a program tries to
> > read from /sys/bus/pci/devices/<bus id>/config. This was not happening with
> > 4.7 and when I use Arch's current LTS kernel (4.4) it also works fine [2].
> 
> As soon as all the patches landed (from comment #10) there's no need to
> check /sys/bus/pci/devices/<bus id>/config anymore, because there's a new
> API to get that info that doesn't need to wake up the device and the problem
> goes away. But right now the mesa patches are not applied as far as I can
> tell.
> 
Yes and no - yes the patches in comment #10 will help from graphics POV, but no they won't help with the 'generic case' that Eugene is talking about. With "generic case" being anyone reading the ./config file will wake up the device.

Eugene if behaviour has changed across kernel versions [i.e. you're confident that with kernel X the dGPU was powered off and reading ./config did not wake it up] feel free to bisect and report it.

(In reply to Mauro Santos from comment #13)

> Aren't most patches in mesa and libdrm by now? At least the versions
> provided by Arch already seem to have the patches (or at least some of them).
> 
You know that everything is open-source right ;-)

Arch has also gone full r****d mode and is patching mesa for EGL GLVND, but none of the above patches are included. Yet the kernel and libdrm are vanilla.

> Eugene if behaviour has changed across kernel versions...

I'm confident that from 2014 and up to 4.10.0 on my Gentoo machine this was never the case. I don't quite understand what do I need to bisect, because the same kernel sources (4.9.11) compiled using the same config give different results on two identical laptops. May it be that a difference in environment (at compile-time or run-time) causes this problem?

To clarify: lspci was never waking up the dGPU. I assume that it was always reading the device config file.

(In reply to Eugene Shalygin from comment #17)
> To clarify: lspci was never waking up the dGPU. I assume that it was always
> reading the device config file.

The kernel behaviour change(?) does sound surprising. I suggest checking that the device was powered off in both cases - leaning that it might not have been.

That said, it sounds like the root of your issue is different than the one covered here - pardon for dragging you :-\

(In reply to Emil Velikov from comment #18)
> (In reply to Eugene Shalygin from comment #17)
> > To clarify: lspci was never waking up the dGPU. I assume that it was always
> > reading the device config file.
> 
> The kernel behaviour change(?) does sound surprising. I suggest checking
> that the device was powered off in both cases - leaning that it might not
> have been.
 
I do have to say that I see the same behavior, although at the time what made me notice the change is behavior was a mesa update. The only change for the following tests was booting with a different kernel.

With 4.4.52 lspci says:  
03:00.0 Display controller [0380]: Advanced Micro Devices, Inc. [AMD/ATI] Mars [Radeon HD 8670A/8670M/8750M] [1002:6600] (rev ff)

With 4.9.11 lspci says:
03:00.0 Display controller [0380]: Advanced Micro Devices, Inc. [AMD/ATI] Mars [Radeon HD 8670A/8670M/8750M] [1002:6600] (rev 81)

Notice that the revision is different so I would bet the kernel was not behaving properly before.

Also /sys/kernel/debug/dri/1/radeon_pm_info says the following right after using lspci:
With kernel 4.4.52:
PX asic powered off

With kernel 4.9.11:
uvd    vclk: 0 dclk: 0
power level 0    sclk: 30000 mclk: 15000 vddc: 900 vddci: 0 pcie gen: 2

> That said, it sounds like the root of your issue is different than the one
> covered here - pardon for dragging you :-\

(In reply to Mauro Santos from comment #19)
> With 4.4.52 lspci says:  
> 03:00.0 Display controller [0380]: Advanced Micro Devices, Inc. [AMD/ATI]
> Mars [Radeon HD 8670A/8670M/8750M] [1002:6600] (rev ff)
> 
> With 4.9.11 lspci says:
> 03:00.0 Display controller [0380]: Advanced Micro Devices, Inc. [AMD/ATI]
> Mars [Radeon HD 8670A/8670M/8750M] [1002:6600] (rev 81)
> 
> Notice that the revision is different so I would bet the kernel was not
> behaving properly before.

Previously lspci would just read back from pci config space for stuff that was not cached which resulted in reading back all ones if the device was powered off, hence the rev ff.  The pci revision id is now cached so it's no longer read back from pci config space which is why is shows the proper value (rev 81).

(In reply to Alex Deucher from comment #20)
> Previously lspci would just read back from pci config space for stuff that
> was not cached which resulted in reading back all ones if the device was
> powered off, hence the rev ff.  The pci revision id is now cached so it's no
> longer read back from pci config space which is why is shows the proper
> value (rev 81).

You lost me here, I assume that by caching you mean the sysfs patch that adds the revision field. I have tested with 4.4.52 and 4.9.11 and those do not have that patch.

The difference between both is that with 4.4.52 the device stays powered off when using lspci but with 4.9.11 the device is automatically powered on. This is using all the same software versions, the only difference is booting into different kernel versions.

(In reply to Mauro Santos from comment #21)
> The difference between both is that with 4.4.52 the device stays powered off
> when using lspci but with 4.9.11 the device is automatically powered on.
> This is using all the same software versions, the only difference is booting
> into different kernel versions.

I don't quite understand the situation too. I observe two opposite behaviours with the same kernel version (4.9): lspci wakes up dGPU on ArchLinux or not (Gentoo). Which one is bugged?

(In reply to Eugene Shalygin from comment #22)

> I don't quite understand the situation too. I observe two opposite
> behaviours with the same kernel version (4.9): lspci wakes up dGPU on
> ArchLinux or not (Gentoo). Which one is bugged?

As I said earlier - first, you want to ensure that the dGPU is powered off in _both_ cases (Arch vs Gentoo). Assuming you're OK with building your own kernel a couple of printfs is all you need.

If the state is not the same - check you can the conditions under which the device should be powered-off - powertop toggles, something using your dGPU, etc.

If they are - check with the lspci/other version and thus code used on both systems.

In either case all this sounds unrelated to this bug, so I'd suggest keeping it separate.

Back to the original issue:

kernel, libdrm and mesa changes are in v4.10, v2.4.75 and [to be announced] V17.1.0 respectively.

Just found that dGPU wakes up when I connect an Android phone via USB. And what is worse, the dGPU does not power off after that until reboot.

(In reply to Eugene Shalygin from comment #25)
> Just found that dGPU wakes up when I connect an Android phone via USB.

No, any USB device makes that. kernel 4.10.8.

(In reply to Eugene Shalygin from comment #26)
> (In reply to Eugene Shalygin from comment #25)
> > Just found that dGPU wakes up when I connect an Android phone via USB.
> 
> No, any USB device makes that. kernel 4.10.8.

While the side effects are similar, the issue seems unrelated. There are some suggestions in comment 23, that you might want to explode.

In either case, please keep it a separate bug report. Thank you!