Bug 112169

QoS; you can poke around with the timeout setting in config (via sysfs in later patches) to try and determine the right ballpark. Current was chosen by looking at piglit and declaring the test that tried to keep preemption off for 1s a dos.

(In reply to Chris Wilson from comment #1)
> QoS; you can poke around with the timeout setting in config (via sysfs in
> later patches)

When do you think those config patches will land, and from which sysfs file one can configure it?


> to try and determine the right ballpark. Current was chosen
> by looking at piglit and declaring the test that tried to keep preemption
> off for 1s a dos.

While these tests don't have slowest frame updates, they probably have slowest individual draw calls.

GpuTest is public, so it can be easily tested.  Piano & Volplosion are slowest tests in it, and slow linearly down with larger resolutions.

Marking this for all platforms I have, as it happens on all of them.

So far with a bump to 250ms, gputest fails to cause iris to loop unpreemptibly. The only other question is whether or not iris itself can avoid such long denial of service.

(In reply to Chris Wilson from comment #3)
> So far with a bump to 250ms, gputest fails to cause iris to loop
> unpreemptibly. The only other question is whether or not iris itself can
> avoid such long denial of service.

Why that would be a problem for Iris, when it AFAIK has never been a problem for i965?

(In reply to Eero Tamminen from comment #4)
> (In reply to Chris Wilson from comment #3)
> > So far with a bump to 250ms, gputest fails to cause iris to loop
> > unpreemptibly. The only other question is whether or not iris itself can
> > avoid such long denial of service.
> 
> Why that would be a problem for Iris, when it AFAIK has never been a problem
> for i965?

It's a problem in both (unbounded execution latency affecting higher priority clients); I was just using iris as an example as being more convenient to play with.

./scripts/config --set-val CONFIG_DRM_I915_PREEMPT_TIMEOUT 250

On TGL, this happens in many additional tests, including 2 different Automotive customer benchmarks.

In the SynMark benchmark, it happens in the following tests (when using last evening drm-tip):
[  778.513869] Iteration 1/3: synmark2 OglCSCloth
[  778.849082] i915 0000:00:02.0: Resetting rcs0 for preemption time out
[  778.898748] Iteration 2/3: synmark2 OglCSCloth
[  779.097064] i915 0000:00:02.0: Resetting rcs0 for preemption time out
[  779.134308] Iteration 3/3: synmark2 OglCSCloth
[  779.329093] i915 0000:00:02.0: Resetting rcs0 for preemption time out
[  779.374631] Iteration 1/3: synmark2 OglCSDof
[  780.000965] i915 0000:00:02.0: Resetting rcs0 for preemption time out
[  780.764332] Iteration 2/3: synmark2 OglCSDof
[  781.001096] i915 0000:00:02.0: Resetting rcs0 for preemption time out
[  781.699362] Iteration 3/3: synmark2 OglCSDof
[  781.928953] i915 0000:00:02.0: Resetting rcs0 for preemption time out
...
[ 1445.167415] Iteration 1/3: synmark2 OglPSPom
[ 1445.377107] i915 0000:00:02.0: Resetting rcs0 for preemption time out
[ 1470.346698] Iteration 2/3: synmark2 OglPSPom
[ 1470.553096] i915 0000:00:02.0: Resetting rcs0 for preemption time out
[ 1495.481206] Iteration 3/3: synmark2 OglPSPom
[ 1495.681099] i915 0000:00:02.0: Resetting rcs0 for preemption time out
...
[ 1876.058405] Iteration 3/3: synmark2 OglTerrainFlyTess
[ 1898.265122] i915 0000:00:02.0: Resetting rcs0 for preemption time out
[ 1901.588742] Iteration 1/3: synmark2 OglTerrainPanTess
[ 1927.116433] Iteration 2/3: synmark2 OglTerrainPanTess
[ 1952.646441] Iteration 3/3: synmark2 OglTerrainPanTess
[ 1969.121125] i915 0000:00:02.0: Resetting rcs0 for preemption time out
[ 1969.233106] i915 0000:00:02.0: Resetting rcs0 for preemption time out
[ 1969.345105] i915 0000:00:02.0: Resetting rcs0 for preemption time out
...
[ 2074.882611] Iteration 2/3: synmark2 OglVSDiffuse8
[ 2093.057110] i915 0000:00:02.0: Resetting rcs0 for preemption time out
[ 2093.169108] i915 0000:00:02.0: Resetting rcs0 for preemption time out
[ 2093.281111] i915 0000:00:02.0: Resetting rcs0 for preemption time out

(On ICL, Terrain tests run at >100FPS, CSCloth & VSDiffuse8 at >200FPS, so it seems unlikely that draw calls in these test would be particularly slow on TGL either, unless there's some other TGL specific bug in upstream Mesa / drm-tip kernel.  SynMark above is Wayland version because X server crashes on TGL when something tries to use it.)

GPU reclocking is dysfunctional on Tigerlake (at least as far as observation goes and the randomly reported frequencies), so not too surprising as other observations back up that Tigerlake is clocking lower than intended.

(In reply to Chris Wilson from comment #7)
> GPU reclocking is dysfunctional on Tigerlake (at least as far as observation
> goes and the randomly reported frequencies), so not too surprising as other
> observations back up that Tigerlake is clocking lower than intended.

Looking at:
- /sys/class/drm/card0/gt_act_freq_mhz
- i915:intel_gpu_freq_change tracepoint values

For first minutes of running 3D benchmarks without a pause, tracepoint reports i915 to have requested 300MHz speed from GPU, but sampling gt_act_freq_mhz tells GPU to run at 400MHz.

After that tracepoint tells 900Mhz to be requested, but gt_act_freq_mhz tells GPU to run at 700MHz.

That indeed seems bogus power management behavior, but some of the tests where the TGL pre-emption reset issues are seen, are still running at > 100FPS, so they're definitely not too slow from the user point of view.

Did you ever manage to try with the 250ms preemption reset timeout? That cleared up the benchmarks you highlighted for me, but my sample size is limited.

(In reply to Chris Wilson from comment #5)
> ./scripts/config --set-val CONFIG_DRM_I915_PREEMPT_TIMEOUT 250

(In reply to Chris Wilson from comment #9)
> Did you ever manage to try with the 250ms preemption reset timeout? That
> cleared up the benchmarks you highlighted for me, but my sample size is
> limited.

~/intel-gpu-tools$ git pull
Already up to date.
~/intel-gpu-tools$ ls  scripts/
convert_itp.py  intel-gfx-trybot  list-workarounds  Makefile.am  media-bench.pl  quick-testlist.py  run-tests.sh  throttle.py  trace.pl  who.sh

-> there's no script called "config" that I could use for trying that.

My apologies, I was referring to the kernel build config helper as currently the control knob (CONFIG_DRM_I915_PREEMPT_TIMEOUT) is only configurable at build time.

(In reply to Chris Wilson from comment #11)
> My apologies, I was referring to the kernel build config helper as currently
> the control knob (CONFIG_DRM_I915_PREEMPT_TIMEOUT) is only configurable at
> build time.

Just adding this to config:
CONFIG_DRM_I915_PREEMPT_TIMEOUT=250

wasn't enough.  After olddefconfig, values had reverted to:
CONFIG_DRM_I915_USERFAULT_AUTOSUSPEND=250
CONFIG_DRM_I915_HEARTBEAT_INTERVAL=2500
CONFIG_DRM_I915_PREEMPT_TIMEOUT=100
CONFIG_DRM_I915_STOP_TIMEOUT=100

What else is needed to get new preempt timeout through olddefconfig?

(olddefconfig is what our build machinery uses for kernel build, I can't change that.)

I would have though olddefconfig would have worked. Nevermind, just patch the default then:

diff --git a/drivers/gpu/drm/i915/Kconfig.profile b/drivers/gpu/drm/i915/Kconfig.profile
index 1799537a3228..082c409134af 100644
--- a/drivers/gpu/drm/i915/Kconfig.profile
+++ b/drivers/gpu/drm/i915/Kconfig.profile
@@ -25,7 +25,7 @@ config DRM_I915_HEARTBEAT_INTERVAL
 
 config DRM_I915_PREEMPT_TIMEOUT
        int "Preempt timeout (ms, jiffy granularity)"
-       default 100 # milliseconds
+       default 250 # milliseconds
        help
          How long to wait (in milliseconds) for a preemption event to occur
          when submitting a new context via execlists. If the current context

From quick test on SKL GT2, 250ms fixes hangs on MemBW blit & blend tests, but not in the texture BW test.  It fixes hang in GpuTest Volplosion, but not in the slower Piano test.

I'm currently trying few longer timeouts on J4205 (i.e. slower device than GT2), and a larger set of tests.

On SKL GT2, 450ms is enough for everything, but on J4205, GpuTest Piano still GPU resets with 450ms.

(450ms should have been enough for Piano based on average frame timings, but maybe some draws / frames are considerably longer than the average.)

With 550ms everything goes through without resets also on J4205.  Note that I'm running GpuTest Piano test in 1366x768 resolution.  With FullHD resolution (i.e. 2x more pixels), I assume it would still be reseted.

PS. I have hard time testing timeouts on J4205, because it system hangs so easily with drm-tip (in GpuTest Piano, in SynMark Trrain tests...).

Applied

commit 5766a5ffc6a69595903865518c43636bde0e4ac4 (HEAD -> drm-intel-next-queued, drm-intel/drm-intel-next-queued)
Author: Chris Wilson <chris@chris-wilson.co.uk>
Date:   Mon Nov 25 16:27:37 2019 +0000

    drm/i915: Default to a more lenient forced preemption timeout
    
    Based on a sampling of a number of benchmarks across platforms, by
    default opt for a much more lenient timeout so that we should not
    adversely affect existing "good" clients.
    
    640ms ought to be enough for anyone.

to keep the wolves at bay. Thanks for the feedback. I'll leave this as NEEDINFO for a bit in case we need to refine that value again.

(In reply to Chris Wilson from comment #16)
> I'll leave this as NEEDINFO for a bit in case we need to refine that value again.

(In reply to Eero Tamminen from comment #15)
> On SKL GT2, 450ms is enough for everything, but on J4205, GpuTest Piano
> still GPU resets with 450ms.
...
> With 550ms everything goes through without resets also on J4205.  Note that
> I'm running GpuTest Piano test in 1366x768 resolution.

Tested also 500ms on J4205.  There are no GPU reset in GpuTest tests with 500ms either, when running test in that resolution (or in other tests I run regularly).

As this 3D benchmark is EU & memory bandwidth bound, BXT with minimal 12EUs instead 18EUs in J4205, would need larger timeout, but the new 640 value is probably enough for that too.

However, FullHD resolution would need 2x larger timeout, as it has 2x more pixels and GpuTest Piano is pixel shader bound.

While I'm not running GpuTest (Piano) in that large resolution, Phoronix does:
https://www.phoronix.com/scan.php?page=article&item=intel-iris-uhd620&num=4

(I'm not sure how much 3D benchmarks Phoronix runs on low end iGPU machines though.)

-- GitLab Migration Automatic Message --

This bug has been migrated to freedesktop.org's GitLab instance and has been closed from further activity.

You can subscribe and participate further through the new bug through this link to our GitLab instance: https://gitlab.freedesktop.org/drm/intel/issues/562.