Bug 69928

Created attachment 86799 [details]
3.11.0 boots fine

Your 3.12-rc2 lod looks incomplete - note the timestamps.

AFAICS the only commit that should be affecting you is
commit a27e56996687e79416d69a7e6dc26f9d8fe06059
Author: Lucas Stach <dev@lynxeye.de>
Date:   Wed Aug 28 02:00:50 2013 +0200

    drm/nouveau: use MSI interrupts
    
    MSIs were only problematic on some old, broken chipsets. But now that we
    already see systems where PCI legacy interrupts are somewhat flaky, it's
    really time to move to MSIs.
    
    v2 (Ben Skeggs): blacklist BR02 boards


You should be able to toggle MSI off using "nouveau.config=NvMSI=0" appended to your kernel command line. Can you give it a try and attach the resulting dmesg

Created attachment 86806 [details]
dmesg 3.12-rc2 boots successful with nouveau.config=NvMSI=0

> note the timestamps
I noticed but that was my 1st usage of netconsole - maybe it was some issue with target system (a VM), router or something else.

> You should be able to toggle MSI off using "nouveau.config=NvMSI=0" appended to > your kernel command line. Can you give it a try and attach the resulting dmesg

see new attachment

Thanks for confirming. The author of the patch has requested information wrt MSI on nvidia boards. I'm hoping that we can get reply soon(ish) otherwise we may have to disable MSI default on all boards.

Confirmed with 3.12-rc4. 

System:
 ASUS P6T SE, i7
 Nvidia GF100 [GeForce GTX 470] (rev a3).
 Ubuntu 13.04.

Details: No error with 3.10 kernel.   Boots / hangs in the text mode before unity-greater with 3.12-rc4.     Boots succesfully / works fine with 3.12-rc4 with nouveau.config=NvMSI=0 in the boot command line.


I would suggest rising a priority on this one.

(In reply to comment #5)
> Confirmed with 3.12-rc4. 
> 
> System:
>  ASUS P6T SE, i7
>  Nvidia GF100 [GeForce GTX 470] (rev a3).
>  Ubuntu 13.04.
> 
> Details: No error with 3.10 kernel.   Boots / hangs in the text mode before
> unity-greater with 3.12-rc4.     Boots succesfully / works fine with
> 3.12-rc4 with nouveau.config=NvMSI=0 in the boot command line.
> 
> 
> I would suggest rising a priority on this one.

MSI will be disabled by default for 3.12 as a result of this and a few other related regressions.

To help fix this in the future, would you be able to get me a trace of the nvidia binary driver?

Instructions of how to do so are here: http://nouveau.freedesktop.org/wiki/MmioTrace/

Thanks!

Created attachment 87677 [details]
mmiotrace loading module nvidia 304

i hope kernel version doesn't play a role in mmio tracing

trace was done from fresh ubuntu 13.04 install (on usb stick)
Linux vm1304 3.8.0-19-generic #30-Ubuntu SMP Wed May 1 16:36:13 UTC 2013 i686 athlon i686 GNU/Linux

with nvidia blobs from ubuntu repos (nvidia-304)

additional mmio traces:

simple xinit with sleep:
mmiotrace_xinit.log.xz (3.1 MB)
https://mega.co.nz/#!u4RwFLjK!Yzi4UVujupmRdEIMFZtbQBOtjyoJFR1jrXheO1RUpew

starting glxgears:
mmiotrace_glxgears.log.xz (3.4 MB)
https://mega.co.nz/#!744wlQ7b!CBnbnEYfsIwOuXKFojgy3kuecVkC6U43Bye4nZkKMUk

starting nvidia-settings - since dual screen is active already just browse through options and then quit:
mmiotrace_dualsettings.log.xz (2.7 MB)
https://mega.co.nz/#!L1ZR3ApZ!XEuYymYmvLyQSQdpwQ0HaiBEipp2CBHz9y3PHNBNn6Q


glxgears posted because: https://bugs.freedesktop.org/show_bug.cgi?id=69952

Can you also show the contents of /proc/interrupts while X is running with the NVIDIA binary driver?

Thanks!

Created attachment 87714 [details]
output /proc/interrupts every 1 second with nvidia-304

/proc/interrupts is attached from one "test" run of 
with a parallel mmiotrace

xinit nvidia-settings
(browse through some menus, then quit)
(somehow xinit "sleep 20" wasnt really working this time because some shell issue )

from the recovery console
- Kernel was 3.8.0-19 default provided by Ubuntu 13.04 (like above)
like before

Do you need proc/interrupts from a normal user desktop too ?

Created attachment 87715 [details]
mmiotrace of proc/interrupts run

mmiotrace of 
xinit nvidia-settings of the proc/interrupts run



with nvidia-304, Kernel 3.8.0-19 Ubuntu 13.04 default (32 bit)

The request to see /proc/interrupts was just to see if it was using MSI or not, not for the actual interrupt counts.

 20:       5187   IO-APIC-fasteoi   ehci_hcd:usb2, nvidia

Sadly it's not using MSI. (If it were, it'd say PCI-MSI-edge or something like that.) Would it be possible for you to get a version that enables MSI? I believe 325 and newer do this. If it does enable MSI, Ben could use a trace.

(In reply to comment #12)
> The request to see /proc/interrupts was just to see if it was using MSI or
> not, not for the actual interrupt counts.
> 
>  20:       5187   IO-APIC-fasteoi   ehci_hcd:usb2, nvidia
> 
> Sadly it's not using MSI. (If it were, it'd say PCI-MSI-edge or something
> like that.) Would it be possible for you to get a version that enables MSI?
> I believe 325 and newer do this. If it does enable MSI, Ben could use a
> trace.

MSI is indeed enabled in 325.15

Creating a test system (ubuntu 13.10) took a while my setup is:
- apply a patch to NVIDIA drivers because Kernel 3.11 is used (via google)
- disable modeset (kernel cmdline: nomodeset )
- blacklist vesafb, nouveau (modprobe.d entries and cmdline: rdblacklist=nouveau  nouveau.blacklist=1  nouveau.modeset=0)
- enable vga console (cmdline: video=vesa:off vga=normal )
- install nvidia custom driver (created by --apply-patch patch311.patch)
- reboot

two mmiotraces are attached

Created attachment 87876 [details]
mmiotrace modprobe nvidia

mmiotrace when doing modprobe nvidia

mmiotrace of xinit "sleep 10" is too big (>3000kB xz compressed)

System is a clean Ubuntu 13.10 with Nvidia 325.15 with the needed small patch


--- a/kernel/nv-linux.h
+++ b/kernel/nv-linux.h
@@ -957,7 +957,11 @@ static inline int nv_execute_on_all_cpus
 #endif
 
 #if !defined(NV_VMWARE)
+#if LINUX_VERSION_CODE >= KERNEL_VERSION(3, 11, 0)
+#define NV_NUM_PHYSPAGES                get_num_physpages()
+#else
 #define NV_NUM_PHYSPAGES                num_physpages
+#endif
 #define NV_GET_CURRENT_PROCESS()        current->tgid
 #define NV_IN_ATOMIC()                  in_atomic()
 #define NV_LOCAL_BH_DISABLE()           local_bh_disable()


here is the link:
mmiotrace_xinit_sleep10.log.xz (3.0 MB)
https://mega.co.nz/#!2hRgDRqb!BwPChW79v7SdYK0rvxbtT3EPmHVM300Flqfuw65Xpbs

without disabling of vesafb and enabling of vga console (see kernel cmdline in comment above) there will be lockups after logging into X (there is a NVRM warning about that in dmesg)

FTR, the trace contains:

[1] 698.683063 MMIO32 R 0x088068 0x00810005 PPCI.MSI_HEAD => { CAP_ID = MSI | NEXT_CAP_PTR = 0 | ENABLE | QMASK = 0 | QSIZE = 0 | 64BIT }
[1] 698.683072 MMIO8 W 0x088068 0x000000ff PPCI.MSI_HEAD <= { CAP_ID = 0xff | NEXT_CAP_PTR = 0 | QMASK = 0 | QSIZE = 0 }

and then later

[1] 699.766366 MMIO32 R 0x000100 0x80000000 PMC.INTR_HOST => { SOFTWARE }
[1] 699.766375 MMIO32 W 0x000100 0x00000000 PMC.INTR_HOST <= { 0 }
[1] 699.766434 MMIO8 W 0x088068 0x000000ff PPCI.MSI_HEAD <= { CAP_ID = 0xff | NEXT_CAP_PTR = 0 | QMASK = 0 | QSIZE = 0 }
[1] 699.766443 MMIO32 W 0x000140 0x00000000 PMC.INTR_EN_HOST <= { 0 }

etc. So the blob does enable MSI on NVAA.

Created attachment 106850 [details] [review]
Add some writes to 0x100c14

Does this patch help (applying it to a recent kernel code would be better)? I'm not sure it will, but well, maybe worth the try.

Is this still an issue using kernel 3.19-rc4?

No retest in over a year. Marking invalid.