Created attachment 131576 [details] i915 crash dump On Linux 4.12-rc3 with GVT-g enabled, MDEV device attached to a VM and running an application using VA-API to watch H264 video I got the following GPU hang: [71153.016783] [drm] GPU HANG: ecode 9:0:0x85dffffb, reason: Hang on rcs, action: reset [71153.016784] [drm] GPU hangs can indicate a bug anywhere in the entire gfx stack, including userspace. [71153.016784] [drm] Please file a _new_ bug report on bugs.freedesktop.org against DRI -> DRM/Intel [71153.016785] [drm] drm/i915 developers can then reassign to the right component if it's not a kernel issue. [71153.016785] [drm] The gpu crash dump is required to analyze gpu hangs, so please always attach it. [71153.016786] [drm] GPU crash dump saved to /sys/class/drm/card0/error [71154.896616] asynchronous wait on fence i915:systemd-logind[482]/0:de4af timed out [71154.896625] asynchronous wait on fence i915:systemd-logind[482]/0:de4af timed out [71154.903792] drm/i915: Resetting chip after gpu hang [71154.904718] [drm] RC6 on [71156.927851] WARN_ON_ONCE(!(offset >= 0 && offset < gvt->device_info.mmio_size)) [71156.927880] ------------[ cut here ]------------ [71156.927897] WARNING: CPU: 3 PID: 1199 at drivers/gpu/drm/i915/gvt/mmio.c:293 intel_vgpu_emulate_mmio_write+0x606/0x630 [i915] [71156.927897] Modules linked in: vhost_net vhost tap tun ebtable_filter ebtables ip6table_filter ip6_tables md4 nls_utf8 cifs dns_resolver fscache ctr ccm bonding bridge stp llc ipheth iptable_filter ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack joydev iTCO_wdt iTCO_vendor_support arc4 snd_hda_codec_hdmi snd_hda_codec_realtek snd_hda_codec_generic iwlmvm mac80211 ext4 jbd2 fscrypto mbcache snd_soc_skl snd_soc_skl_ipc iwlwifi snd_soc_sst_ipc snd_soc_sst_dsp snd_hda_ext_core snd_soc_sst_match snd_soc_core snd_compress rtsx_pci_ms intel_rapl snd_pcm_dmaengine cfg80211 x86_pkg_temp_thermal memstick intel_powerclamp ac97_bus coretemp kvm_intel intel_cstate intel_rapl_perf pcspkr psmouse mousedev evdev input_leds mac_hid e1000e i2c_i801 [71156.927925] ptp pps_core snd_hda_intel snd_hda_codec snd_hda_core snd_hwdep snd_pcm shpchp snd_timer mei_me mei intel_pch_thermal uvcvideo btusb btrtl videobuf2_vmalloc btbcm videobuf2_memops videobuf2_v4l2 btintel videobuf2_core bluetooth videodev media ecdh_generic crc16 kvmgt vfio_mdev mdev vfio_iommu_type1 vfio kvm irqbypass i915 cdc_ether option usbnet usb_wwan usbserial mii thinkpad_acpi nvram wmi snd drm_kms_helper soundcore rfkill led_class battery drm ac intel_gtt syscopyarea sysfillrect sysimgblt fb_sys_fops i2c_algo_bit video button thermal tpm_tis tpm_tis_core tpm sch_fq_codel ip_tables x_tables xfs libcrc32c crc32c_generic algif_skcipher af_alg hid_generic usbhid hid dm_crypt dm_mod dax sd_mod rtsx_pci_sdmmc mmc_core serio_raw atkbd libps2 crct10dif_pclmul crc32_pclmul crc32c_intel [71156.927958] ghash_clmulni_intel pcbc aesni_intel aes_x86_64 crypto_simd glue_helper cryptd ahci libahci xhci_pci libata xhci_hcd rtsx_pci scsi_mod usbcore usb_common i8042 serio [71156.927967] CPU: 3 PID: 1199 Comm: CPU 1/KVM Tainted: G W 4.12.0-rc3-mainline #1 [71156.927967] Hardware name: LENOVO 20F6007RGE/20F6007RGE, BIOS R02ET48W (1.21 ) 06/01/2016 [71156.927968] task: ffff8801102e5880 task.stack: ffffc90003d98000 [71156.927979] RIP: 0010:intel_vgpu_emulate_mmio_write+0x606/0x630 [i915] [71156.927980] RSP: 0018:ffffc90003d9b940 EFLAGS: 00010286 [71156.927981] RAX: 0000000000000043 RBX: 0000000000000008 RCX: ffffffff81a55a08 [71156.927982] RDX: 0000000000000000 RSI: 0000000000000082 RDI: 0000000000000247 [71156.927983] RBP: ffffc90003d9b998 R08: 0000000000000043 R09: 0000000000008165 [71156.927983] R10: ffffc90003d9b920 R11: 0000000000000000 R12: ffff880235eec000 [71156.927984] R13: ffff8801102d2810 R14: ffffc900012cd000 R15: ffffc9000a8fb010 [71156.927985] FS: 00007fe0781ff700(0000) GS:ffff880242580000(0000) knlGS:00000024879ab000 [71156.927986] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [71156.927986] CR2: ffffd10de9b8e000 CR3: 00000001100f2000 CR4: 00000000003426e0 [71156.927987] Call Trace: [71156.927992] ? __check_object_size+0xae/0x18b [71156.927994] kvmgt_page_track_write+0x64/0x70 [kvmgt] [71156.928001] kvm_page_track_write+0x7e/0xb0 [kvm] [71156.928008] emulator_write_phys+0x3b/0x50 [kvm] [71156.928015] write_emulate+0xe/0x10 [kvm] [71156.928021] emulator_read_write_onepage+0x19f/0x320 [kvm] [71156.928027] emulator_read_write+0xcd/0x180 [kvm] [71156.928033] emulator_write_emulated+0x15/0x20 [kvm] [71156.928040] segmented_write+0x59/0x80 [kvm] [71156.928046] writeback+0x12d/0x210 [kvm] [71156.928052] x86_emulate_insn+0x82a/0xf10 [kvm] [71156.928058] ? x86_decode_insn+0x50a/0x1270 [kvm] [71156.928064] x86_emulate_instruction+0x1df/0x720 [kvm] [71156.928072] kvm_mmu_page_fault+0xa2/0x120 [kvm] [71156.928075] handle_ept_violation+0x9e/0x150 [kvm_intel] [71156.928077] vmx_handle_exit+0xad/0x1420 [kvm_intel] [71156.928084] ? apic_set_eoi+0xbc/0x200 [kvm] [71156.928090] ? kvm_lapic_sync_from_vapic+0xcd/0x190 [kvm] [71156.928097] kvm_arch_vcpu_ioctl_run+0x8e2/0x1690 [kvm] [71156.928103] ? kvm_arch_vcpu_load+0x62/0x240 [kvm] [71156.928109] kvm_vcpu_ioctl+0x339/0x630 [kvm] [71156.928114] ? kvm_vcpu_ioctl+0x339/0x630 [kvm] [71156.928117] ? vfio_device_fops_read+0x24/0x30 [vfio] [71156.928119] do_vfs_ioctl+0xa3/0x5f0 [71156.928121] ? __fget+0x6e/0x90 [71156.928122] SyS_ioctl+0x79/0x90 [71156.928124] entry_SYSCALL_64_fastpath+0x1a/0xa5 [71156.928125] RIP: 0033:0x7fe084c340d7 [71156.928126] RSP: 002b:00007fe0781fe8e8 EFLAGS: 00000246 ORIG_RAX: 0000000000000010 [71156.928127] RAX: ffffffffffffffda RBX: 000000000000ae80 RCX: 00007fe084c340d7 [71156.928128] RDX: 0000000000000000 RSI: 000000000000ae80 RDI: 0000000000000011 [71156.928128] RBP: 00007fe078234000 R08: 0000558fd2aa7030 R09: 00000000000000ff [71156.928129] R10: 0000000000004260 R11: 0000000000000246 R12: 0000000000000000 [71156.928129] R13: 00007fe08bc87000 R14: 0000000000000000 R15: 00007fe078234000 [71156.928131] Code: 8b 54 05 fc 89 54 05 c4 e9 ba fd ff ff 48 c7 c6 90 41 6c a0 48 c7 c7 52 a8 6a a0 4c 89 4d c0 c6 05 9f aa 05 00 01 e8 de 90 b0 e0 <0f> ff 4c 8b 4d c0 e9 e8 fe ff ff 89 d8 41 0f b7 54 05 fe 66 89 [71156.928153] ---[ end trace 57fca0ef330078ee ]--- i915 module is loaded with enable_gvt=1 and MDEV created in i915-GVTg_V5_2
Hello Maik, Could you try to replicate with 4.12-rc7, and attach dmesg log with the parameter "drm.debug=0xe" on grub? Thank you.
Hi Elizabeth, I cannot reproduce this anymore with 4.12-rc7. I've tried it for a couple of times as GPU hang occured very fast after using VA-API on the host. The difference is newer kernel on host and I've updated IGFX driver in the Windows 10 VM to latest one. --Maik
(In reply to Maik Broemme from comment #2) > Hi Elizabeth, > > I cannot reproduce this anymore with 4.12-rc7. I've tried it for a couple of > times as GPU hang occured very fast after using VA-API on the host. The > difference is newer kernel on host and I've updated IGFX driver in the > Windows 10 VM to latest one. > > --Maik Hello Maik, Thanks for the answer, since it seems fixed I'm going to change the status to RESOLVE for now. If the same problem occurs again, please get the logs, attach them and change the status to REOPEN. Cheers.
(In reply to Elizabeth from comment #3) > (In reply to Maik Broemme from comment #2) > > Hi Elizabeth, > > > > I cannot reproduce this anymore with 4.12-rc7. I've tried it for a couple of > > times as GPU hang occured very fast after using VA-API on the host. The > > difference is newer kernel on host and I've updated IGFX driver in the > > Windows 10 VM to latest one. > > > > --Maik > > Hello Maik, > Thanks for the answer, since it seems fixed I'm going to change the status > to RESOLVE for now. If the same problem occurs again, please get the logs, > attach them and change the status to REOPEN. > Cheers. Changing the status to close. Thank you.
Created attachment 132650 [details] I915 crash dump - kernel 4.13.0 This bug seems to reoccur in version 4.13.0 of the kernel on RHEL7 as well as all prior kernel versions.
Created attachment 132651 [details] dmesg - kernel 4.13.0 Attachment with relevant dmesg output
(In reply to jacob from comment #5) > Created attachment 132650 [details] > I915 crash dump - kernel 4.13.0 > > This bug seems to reoccur in version 4.13.0 of the kernel on RHEL7 as well > as all prior kernel versions. Good day Jacob, I noticed kernel 4.13 can't be found on https://www.kernel.org/ , could you please share your source? Thank you.
https://koji.fedoraproject.org/koji/buildinfo?buildID=916481 This is where the kernel was acquired from. The source RPM can be gotten here as well as the exact commit in the linux tree this kernel was built from.
(In reply to jacob from comment #8) > https://koji.fedoraproject.org/koji/buildinfo?buildID=916481 > > This is where the kernel was acquired from. The source RPM can be gotten > here as well as the exact commit in the linux tree this kernel was built > from. Hello again, Could you please try to replicate with drm-tip https://cgit.freedesktop.org/drm-tip instead, this has our latest changes included. Thanks.
Hello Jacob, this is a Mesa problem. Changing product to Mesa. Thank you. From batch: 0xfd5123cc: 0x7b000005: 3DPRIMITIVE: fail sequential 0xfd5123d0: 0x00000006: vertex count 0xfd5123d4: 0x00000004: start vertex 0xfd5123d8: 0x00000000: instance count 0xfd5123dc: 0x00000001: start instance 0xfd5123e0: 0x00000000: index bias
We tried it again on drm-tip on commit 0bc6369eb653e3c6c38c8c12c48c73840f5e1434 and these are the error messages we had: Sep 13 10:19:54 ctrgsl-01 kernel: [drm] GPU HANG: ecode 9:0:0x85dffffb, in ugraf [2286], reason: Hang on rcs0, action: reset Sep 13 10:19:54 ctrgsl-01 kernel: [drm] GPU hangs can indicate a bug anywhere in the entire gfx stack, including userspace. Sep 13 10:19:54 ctrgsl-01 kernel: [drm] Please file a _new_ bug report on bugs.freedesktop.org against DRI -> DRM/Intel Sep 13 10:19:54 ctrgsl-01 kernel: [drm] drm/i915 developers can then reassign to the right component if it's not a kernel issue. Sep 13 10:19:54 ctrgsl-01 kernel: [drm] The gpu crash dump is required to analyze gpu hangs, so please always attach it. Sep 13 10:19:54 ctrgsl-01 kernel: [drm] GPU crash dump saved to /sys/class/drm/card0/error Sep 13 10:19:54 ctrgsl-01 kernel: i915 0000:00:02.0: Resetting rcs0 after gpu hang Sep 13 10:20:02 ctrgsl-01 kernel: i915 0000:00:02.0: Resetting rcs0 after gpu hang Sep 13 10:20:32 ctrgsl-01 kernel: i915 0000:00:02.0: Resetting rcs0 after gpu hang Sep 13 10:20:40 ctrgsl-01 kernel: i915 0000:00:02.0: Resetting rcs0 after gpu hang Sep 13 10:20:48 ctrgsl-01 kernel: i915 0000:00:02.0: Resetting rcs0 after gpu hang Sep 13 10:53:33 ctrgsl-01 kernel: i915 0000:00:02.0: Resetting rcs0 after gpu hang Sep 13 10:53:41 ctrgsl-01 kernel: i915 0000:00:02.0: Resetting rcs0 after gpu hang Sep 13 10:55:16 ctrgsl-01 kernel: i915 0000:00:02.0: Resetting rcs0 after gpu hang Sep 13 10:55:24 ctrgsl-01 kernel: i915 0000:00:02.0: Resetting rcs0 after gpu hang Sep 13 10:55:32 ctrgsl-01 kernel: i915 0000:00:02.0: Resetting rcs0 after gpu hang
Created attachment 134199 [details] GPU crash dump from drm-tip Attached GPU crash dump from /sys/class/drm/card0/error
You're hitting the same error than bug 99720. Are you using 4K resolution??
No, our resolution is 1024x768.
Are there any more logs that we could provide that might be useful or any tests that we could do that might be useful?
Hello Jonathan, if the problem is reproducible, you can help the Mesa team by providing an apitrace file having the latest Mesa release and if possible the latest kernel release. Also information as to if it is desktop dependent (KDE/XFCE/Unity/Gnome), steps to reproduce, sna/modesetting xorg configuration may help. From dmesg: [ 0.000000] NUMA: Warning: node 0 [mem 0x00000000-0x77ffffff] overlaps with itself [mem 0x00100000-0x46614fff] ... [ 19.254936] ====================================================== [ 19.254937] WARNING: possible circular locking dependency detected [ 19.254938] 4.13.0-0.rc0.git3.1.fc27.x86_64 #1 Tainted: G U [ 19.254939] ------------------------------------------------------ [ 19.254940] tuned/1671 is trying to acquire lock: [ 19.254941] (cpu_hotplug_lock.rw_sem){++++++}, at: [<ffffffff9c7c42ea>] store+0x2a/0x80 [ 19.254947] but task is already holding lock: [ 19.254948] (s_active#86){++++.+}, at: [<ffffffff9c36dccc>] kernfs_fop_write+0x12c/0x1e0 [ 19.254953] which lock already depends on the new lock. [ 19.254954] the existing dependency chain (in reverse order) is: [ 19.254955] -> #2 (s_active#86){++++.+}: [ 19.254961] lock_acquire+0xa3/0x1f0 [ 19.254962] __kernfs_remove+0x26b/0x310 [ 19.254963] kernfs_remove+0x23/0x40 [ 19.254965] sysfs_remove_dir+0x51/0x60 [ 19.254968] kobject_del.part.3+0x13/0x40 [ 19.254970] kobject_put+0x6e/0x1a0 [ 19.254972] cpufreq_policy_free+0xe3/0x150 [ 19.254974] cpufreq_online+0xef/0x7a0 [ 19.254975] cpufreq_add_dev+0x51/0x80 [ 19.254977] subsys_interface_register+0xe1/0x160 [ 19.254978] cpufreq_register_driver+0x15d/0x230 [ 19.254982] iw_cm_accept+0x12f/0x150 [iw_cm] [ 19.254984] do_one_initcall+0x50/0x1a0 [ 19.254986] do_init_module+0x5f/0x1e8 [ 19.254989] load_module+0x24b0/0x2b50 [ 19.254991] SYSC_init_module+0x194/0x1d0 [ 19.254993] SyS_init_module+0xe/0x10 [ 19.254995] entry_SYSCALL_64_fastpath+0x1f/0xbe [ 19.254996] -> #1 (subsys mutex#5){+.+.+.}: [ 19.255000] lock_acquire+0xa3/0x1f0 [ 19.255003] __mutex_lock+0x86/0x9f0 [ 19.255005] mutex_lock_nested+0x1b/0x20 [ 19.255007] subsys_interface_register+0x7d/0x160 [ 19.255009] cpufreq_register_driver+0x15d/0x230 [ 19.255011] iw_cm_accept+0x12f/0x150 [iw_cm] [ 19.255013] do_one_initcall+0x50/0x1a0 [ 19.255014] do_init_module+0x5f/0x1e8 [ 19.255016] load_module+0x24b0/0x2b50 [ 19.255018] SYSC_init_module+0x194/0x1d0 [ 19.255020] SyS_init_module+0xe/0x10 [ 19.255248] entry_SYSCALL_64_fastpath+0x1f/0xbe [ 19.255249] -> #0 (cpu_hotplug_lock.rw_sem){++++++}: [ 19.255252] __lock_acquire+0x1367/0x13b0 [ 19.255254] lock_acquire+0xa3/0x1f0 [ 19.255255] cpus_read_lock+0x42/0x90 [ 19.255257] store+0x2a/0x80 [ 19.255258] sysfs_kf_write+0x42/0x60 [ 19.255259] kernfs_fop_write+0x151/0x1e0 [ 19.255261] __vfs_write+0x37/0x170 [ 19.255263] vfs_write+0xc6/0x1c0 [ 19.255264] SyS_write+0x58/0xc0 [ 19.255265] do_syscall_64+0x6c/0x1c0 [ 19.255267] return_from_SYSCALL_64+0x0/0x7a [ 19.255267] other info that might help us debug this: [ 19.255269] Chain exists of: cpu_hotplug_lock.rw_sem --> subsys mutex#5 --> s_active#86 [ 19.255273] Possible unsafe locking scenario: [ 19.255274] CPU0 CPU1 [ 19.255274] ---- ---- [ 19.255275] lock(s_active#86); [ 19.255277] lock(subsys mutex#5); [ 19.255492] lock(s_active#86); [ 19.255494] lock(cpu_hotplug_lock.rw_sem); [ 19.255496] *** DEADLOCK *** [ 19.255497] 4 locks held by tuned/1671: [ 19.255498] #0: (&f->f_pos_lock){+.+.+.}, at: [<ffffffff9c2f3d4c>] __fdget_pos+0x4c/0x60 [ 19.255501] #1: (sb_writers#3){.+.+.+}, at: [<ffffffff9c2cc343>] vfs_write+0x193/0x1c0 [ 19.255505] #2: (&of->mutex){+.+.+.}, at: [<ffffffff9c36dcc3>] kernfs_fop_write+0x123/0x1e0 [ 19.255508] #3: (s_active#86){++++.+}, at: [<ffffffff9c36dccc>] kernfs_fop_write+0x12c/0x1e0 [ 19.255511] stack backtrace: [ 19.255513] CPU: 1 PID: 1671 Comm: tuned Tainted: G U 4.13.0-0.rc0.git3.1.fc27.x86_64 #1 [ 19.255514] Hardware name: HP ProLiant m710x Server Cartridge/ProLiant m710x Server Cartridge, BIOS H07 05/23/2016 [ 19.255515] Call Trace: [ 19.255517] dump_stack+0x8e/0xcd [ 19.255519] print_circular_bug+0x1b6/0x210 [ 19.255521] __lock_acquire+0x1367/0x13b0 [ 19.255524] ? debug_lockdep_rcu_enabled+0x1d/0x30 [ 19.255526] lock_acquire+0xa3/0x1f0 [ 19.255528] ? lock_acquire+0xa3/0x1f0 [ 19.255530] ? store+0x2a/0x80 [ 19.255533] cpus_read_lock+0x42/0x90 [ 19.255535] ? store+0x2a/0x80 [ 19.255537] store+0x2a/0x80 [ 19.255539] sysfs_kf_write+0x42/0x60 [ 19.255541] kernfs_fop_write+0x151/0x1e0 [ 19.255544] __vfs_write+0x37/0x170 [ 19.255545] ? rcu_read_lock_sched_held+0x79/0x80 [ 19.255547] ? rcu_sync_lockdep_assert+0x2c/0x60 [ 19.255548] ? __sb_start_write+0x135/0x190 [ 19.255550] ? vfs_write+0x193/0x1c0 [ 19.255551] vfs_write+0xc6/0x1c0 [ 19.255553] SyS_write+0x58/0xc0 [ 19.255555] do_syscall_64+0x6c/0x1c0 [ 19.255556] entry_SYSCALL64_slow_path+0x25/0x25 [ 19.255558] RIP: 0033:0x7f809f81bcad [ 19.255559] RSP: 002b:00007f808bffd510 EFLAGS: 00000293 ORIG_RAX: 0000000000000001 [ 19.255560] RAX: ffffffffffffffda RBX: 000000000000000b RCX: 00007f809f81bcad [ 19.255561] RDX: 000000000000000b RSI: 00007f80a09f1000 RDI: 000000000000000c [ 19.255562] RBP: 00007f80a09f1000 R08: 00007f808002d5c0 R09: 00007f808bfff700 [ 19.255563] R10: 000000000000006a R11: 0000000000000293 R12: 00007f808002d4e0 [ 19.255564] R13: 000000000000000b R14: 0000000001932490 R15: 00000000018dad40
Hello, sorry for the long delay. Some important fixes for gpu hangs have been added in the latest mesa 17.3.6 release. Could you give it a try? Thank you.
-- GitLab Migration Automatic Message -- This bug has been migrated to freedesktop.org's GitLab instance and has been closed from further activity. You can subscribe and participate further through the new bug through this link to our GitLab instance: https://gitlab.freedesktop.org/mesa/mesa/issues/1597.
Use of freedesktop.org services, including Bugzilla, is subject to our Code of Conduct. How we collect and use information is described in our Privacy Policy.