Bug 109242

Summary: [RADV] The Witcher 3 system freeze
Product: Mesa Reporter: Dmitry <terapy-session>
Component: Drivers/Vulkan/radeonAssignee: mesa-dev
Status: RESOLVED FIXED QA Contact: mesa-dev
Severity: normal    
Priority: medium    
Version: 18.3   
Hardware: x86-64 (AMD64)   
OS: Linux (All)   
Whiteboard:
i915 platform: i915 features:
Attachments: dmesg via ssh
game save
r600_debug_info
VGT param hack
Possible fix partial_vs_wave

Description Dmitry 2019-01-07 15:57:09 UTC
The Witcher 3 freezes the system when using the RADV driver. 
Mesa: 18.3.1
Kernel driver: amdgpu
OS: Arch
GPU: RX460

There is no such problem with AMDVLK.
Comment 1 Samuel Pitoiset 2019-01-07 16:07:16 UTC
Does this is a recent regression for you?
Comment 2 Dmitry 2019-01-08 01:56:31 UTC
Before that, there were no heavy loads, and hangs during active combat.
Comment 3 Samuel Pitoiset 2019-01-08 08:11:32 UTC
Can you explain how to reproduce? Can you upload a savegame? Is there anything in dmesg when it hangs?
Comment 4 Dmitry 2019-01-08 13:37:54 UTC
Any heavy scene, for example fight with the first Griffin on a plot.
https://www.youtube.com/watch?v=0J9qnKeID0s

Yes I can. It hangs in the fight.


dmesg is not done because I do a physical reboot.
Comment 5 Samuel Pitoiset 2019-01-08 16:50:35 UTC
Where is the savegame? Also what kernel and llvm versions are you using? The ones provided by Arch Linux?

I have just tried to reproduce GPU hangs on my side by playing around few minutes, no success.
Comment 6 Dmitry 2019-01-08 18:00:27 UTC
Saving the game is not important, I started the game first and realized that hangs in any battles. Problem new.

4.20.0-arch1-1
LLVM 7.0.1-1

I understand this is a problem with the RX460. On bug tracker has several reports about The Witcher 3. And even with my RX460
https://bugs.freedesktop.org/show_bug.cgi?id=101731
Comment 7 Roman Elshin 2019-01-09 02:21:23 UTC
Created attachment 143024 [details]
dmesg via ssh
Comment 8 Roman Elshin 2019-01-09 02:22:54 UTC
Created attachment 143025 [details]
game save
Comment 9 Roman Elshin 2019-01-09 02:23:41 UTC
It looks like i have a same issue with rx560.
Comment 10 Samuel Pitoiset 2019-01-09 09:26:28 UTC
Just installed your savegame, when do you have a GPU hang? When fighting with that creature?
Comment 11 Roman Elshin 2019-01-09 09:49:00 UTC
Yes, usually  hangs happens after just the first strike to it.
Comment 12 Samuel Pitoiset 2019-01-09 10:05:23 UTC
I tried multiple times, no hangs so far. What graphics settings are you using?

I tested on my RX480 with LLVM 7.0.1 and Mesa 18.3.1.
Comment 13 Dmitry 2019-01-09 10:36:37 UTC
Medium preset, 1080p, HairWorks off
Comment 14 Samuel Pitoiset 2019-01-09 10:41:13 UTC
Can you attach the output of "R600_DEBUG=info glxgears" ?
Comment 15 Dmitry 2019-01-09 10:54:57 UTC
Created attachment 143033 [details]
r600_debug_info
Comment 16 Samuel Pitoiset 2019-01-09 11:21:50 UTC
Is the GPU hang 100% reproducible for you guys? Maybe, it only affects Polaris11 and Polaris12. (I only have Polaris10)
Comment 17 Dmitry 2019-01-09 12:00:43 UTC
Yes, always.
On AMDVLK hangs too, but then returns to normal operation.
Comment 18 Samuel Pitoiset 2019-01-09 12:41:53 UTC
In your first comment you said "There is no such problem with AMDVLK."... So, it also hangs with AMDVLK but recovery happens while with RADV the whole machine freezes and you need a hard reboot?
Comment 19 Dmitry 2019-01-09 12:45:28 UTC
C AMDVLK hangs only the game client, and with RADV the whole system. But it is possible and the system hangs for seconds, have not checked Alt + Tab.
Comment 20 Samuel Pitoiset 2019-01-09 13:54:26 UTC
Created attachment 143035 [details] [review]
VGT param hack

Can you try the attached patch? I really doubt it will fix the problem but it's worth trying.
Comment 21 Roman Elshin 2019-01-09 15:38:25 UTC
It looks like it fixes problem, thanks!
Comment 22 Dmitry 2019-01-09 16:01:05 UTC
I don't know how to use it.
Comment 23 Samuel Pitoiset 2019-01-09 16:05:16 UTC
(In reply to Dmitry from comment #22)
> I don't know how to use it.

You need to build mesa from git [1]

https://www.mesa3d.org/install.html

That said, it's actually unexpected that this hack fixes the problem.
Comment 24 Dmitry 2019-01-09 17:00:56 UTC
Mesa-git I can take from AUR. But I still don't understand how to use hook.
Comment 25 Roman Elshin 2019-01-09 17:11:16 UTC
But it definitely works here (just tested several other places with fights - no hangs any more), are there any side effects expected?

I  think must be a way to add applying this patch in PKGBUILD (I am not familiar with arch)
Comment 26 Bas Nieuwenhuizen 2019-01-12 11:52:39 UTC
Setting those options when they are not needed might slow things down.

I think this is the proper fix:

https://gitlab.freedesktop.org/mesa/mesa/merge_requests/109/diffs?commit_id=70aa51f2bf7a4f64df166e4c94936d49814d482f
Comment 27 Roman Elshin 2019-01-12 14:05:56 UTC
>I think this is the proper fix:
>
>https://gitlab.freedesktop.org/mesa/mesa/merge_requests/109/diffs?commit_id=70aa51f2bf7a4f64df166e4c94936d49814d482f

It looks like this not fix a problem (but makes it less reproducible).
Comment 28 Bas Nieuwenhuizen 2019-01-12 20:26:33 UTC
Created attachment 143082 [details] [review]
Possible fix partial_vs_wave

So it seems, looks like it is random for me :(

I tried the attached patch 10 times without hanging on the boss from the savefile, could you test whether this fixes it for you?
Comment 29 Roman Elshin 2019-01-13 05:37:01 UTC
>Created attachment 143082 [details] [review] [review]
>Possible fix partial_vs_wave
>
>So it seems, looks like it is random for me :(

Yes, these hangs looks very random now (or it other hangs which also hides with  "VGT param hack").
Comment 30 Roman Elshin 2019-01-13 10:22:42 UTC
>https://gitlab.freedesktop.org/mesa/mesa/merge_requests/109/diffs?commit_id=70aa51f2bf7a4f64df166e4c94936d49814d482f
>Created attachment 143082 [details] [review] [review]

Are both patches must be applied? (it was tested only first or second, not together).
Comment 31 Roman Elshin 2019-01-13 19:04:55 UTC
>Are both patches must be applied? (it was tested only first or second, not >together).

With both patches applied i can't reproduce this issue, thanks!
Comment 32 Samuel Pitoiset 2019-01-30 09:42:28 UTC
Should be fixed with https://cgit.freedesktop.org/mesa/mesa/commit/?id=568e7a2998e39263c41ec70ec67bcc5601477781

Can we close the issue?
Comment 33 Roman Elshin 2019-01-31 16:26:55 UTC
I think so (mesa from git works fine for me here).
Comment 34 madcatx 2019-02-01 10:10:09 UTC
I hate to make noise about a closed issue but I believe I have been seeing a GPU hang very similar to what is described here in Witcher 3 fights. I can reproduce it only when the game is run through classic WINE D3D->OpenGL translation, DXVK runs flawlessly for me.

Since this fix is for RADV only, is it possible that a similar fix may be applicable for OpenGL path as well?

FTR, I have an ASUS RX570.
Comment 35 Bas Nieuwenhuizen 2019-02-01 10:18:53 UTC
Not sure the fix from the bug would affect a rx 570, but this has been
fixed in radeonsi as well:

commit 5183e794affbbbf7dc959332619b0852c21536d6
Author: Marek Olšák <marek.olsak@amd.com>
Date:   Mon Jan 14 18:24:08 2019 -0500

    radeonsi: also apply the GS hang workaround to draws without tessellation

    ported from AMDVLK.

    Cc: 18.3 <mesa-stable@lists.freedesktop.org>
    Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>

On Fri, Feb 1, 2019 at 11:10 AM <bugzilla-daemon@freedesktop.org> wrote:
>
> Comment # 34 on bug 109242 from madcatx@atlas.cz
>
> I hate to make noise about a closed issue but I believe I have been seeing a
> GPU hang very similar to what is described here in Witcher 3 fights. I can
> reproduce it only when the game is run through classic WINE D3D->OpenGL
> translation, DXVK runs flawlessly for me.
>
> Since this fix is for RADV only, is it possible that a similar fix may be
> applicable for OpenGL path as well?
>
> FTR, I have an ASUS RX570.
>
> ________________________________
> You are receiving this mail because:
>
> You are the assignee for the bug.
> You are the QA Contact for the bug.
>
> _______________________________________________
> mesa-dev mailing list
> mesa-dev@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/mesa-dev
Comment 36 Samuel Pitoiset 2019-02-11 10:17:35 UTC
Closing, both RADV and RadeonSI should now be fixed.

Use of freedesktop.org services, including Bugzilla, is subject to our Code of Conduct. How we collect and use information is described in our Privacy Policy.