Bug 111788 - [i915] i915/vcs1-sema exceeds i915/vcs1-busy value
Summary: [i915] i915/vcs1-sema exceeds i915/vcs1-busy value
Status: NEW
Alias: None
Product: DRI
Classification: Unclassified
Component: DRM/Intel (show other bugs)
Version: XOrg git
Hardware: x86-64 (AMD64) Linux (All)
: not set normal
Assignee: Intel GFX Bugs mailing list
QA Contact: Intel GFX Bugs mailing list
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2019-09-23 16:49 UTC by andrey.orlov
Modified: 2019-09-25 09:50 UTC (History)
2 users (show)

See Also:
i915 platform:
i915 features:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description andrey.orlov 2019-09-23 16:49:14 UTC
Values I got from perf tool (very often we see i915/vcs1-sema > i915/vcs1-busy):
{code}
# ./perf stat -I 100 -a -e i915/vcs1-busy/,i915/vcs1-sema/,i915/vcs1-wait/ ./sample_multi_transcode -par 0002_FHD23i_1.par

libva info: VA-API version 1.6.0
libva info: va_getDriverName() returns 0
libva info: User requested driver 'iHD'
libva info: Trying to open /opt/intel/mediasdk/lib64/iHD_drv_video.so
libva info: Found init function __vaDriverInit_1_6
libva info: va_openDriver() returns 0
#           time             counts unit events
     0.100160810         34,209,689 ns   i915/vcs1-busy/                                             
     0.100160810         15,046,281 ns   i915/vcs1-sema/                                             
     0.100160810                  0 ns   i915/vcs1-wait/                                             
     0.200365878         81,586,469 ns   i915/vcs1-busy/                                             
     0.200365878         75,075,011 ns   i915/vcs1-sema/                                             
     0.200365878                  0 ns   i915/vcs1-wait/                                             
     0.300568998         85,209,534 ns   i915/vcs1-busy/                                             
     0.300568998         44,998,752 ns   i915/vcs1-sema/                                             
     0.300568998                  0 ns   i915/vcs1-wait/                                             
     0.400733002         85,725,483 ns   i915/vcs1-busy/                                             
     0.400733002         90,037,698 ns   i915/vcs1-sema/                                             
     0.400733002                  0 ns   i915/vcs1-wait/                                             
     0.500939741         84,982,430 ns   i915/vcs1-busy/                                             
     0.500939741         80,097,984 ns   i915/vcs1-sema/                                             
     0.500939741                  0 ns   i915/vcs1-wait/                                             
     0.601132522         85,518,925 ns   i915/vcs1-busy/                                             
     0.601132522         90,071,194 ns   i915/vcs1-sema/                                             
     0.601132522                  0 ns   i915/vcs1-wait/                                             
     0.701333967         83,735,465 ns   i915/vcs1-busy/                                             
     0.701333967         85,106,697 ns   i915/vcs1-sema/                                             
     0.701333967                  0 ns   i915/vcs1-wait/                                             
     0.801501148         84,875,175 ns   i915/vcs1-busy/                                             
     0.801501148         45,109,484 ns   i915/vcs1-sema/                                             
     0.801501148                  0 ns   i915/vcs1-wait/                                             
     0.901658275         85,498,038 ns   i915/vcs1-busy/                                             
     0.901658275         30,055,031 ns   i915/vcs1-sema/                                             
     0.901658275                  0 ns   i915/vcs1-wait/                                             
     1.001859013         86,232,348 ns   i915/vcs1-busy/                                             
     1.001859013         89,996,108 ns   i915/vcs1-sema/                                             
     1.001859013                  0 ns   i915/vcs1-wait/                                             
     1.102076724         85,695,796 ns   i915/vcs1-busy/                                             
     1.102076724         94,994,151 ns   i915/vcs1-sema/                                             
     1.102076724                  0 ns   i915/vcs1-wait/                                             
     1.202270626         85,122,835 ns   i915/vcs1-busy/                                             
     1.202270626         69,968,796 ns   i915/vcs1-sema/                                             
     1.202270626                  0 ns   i915/vcs1-wait/                                             
     1.302443248         85,881,238 ns   i915/vcs1-busy/                                             
     1.302443248         95,000,911 ns   i915/vcs1-sema/                                             
     1.302443248                  0 ns   i915/vcs1-wait/                                             
     1.402619770         86,196,687 ns   i915/vcs1-busy/                                             
     1.402619770         80,003,731 ns   i915/vcs1-sema/                                             
     1.402619770                  0 ns   i915/vcs1-wait/                                             
     1.502802220         84,797,945 ns   i915/vcs1-busy/                                             
     1.502802220        100,000,569 ns   i915/vcs1-sema/                                             
     1.502802220                  0 ns   i915/vcs1-wait/                                             
     1.603005224         84,732,615 ns   i915/vcs1-busy/                                             
     1.603005224        100,000,407 ns   i915/vcs1-sema/                                             
     1.603005224                  0 ns   i915/vcs1-wait/                                             
     1.703178374         84,980,237 ns   i915/vcs1-busy/                                             
     1.703178374         85,010,242 ns   i915/vcs1-sema/                                             
     1.703178374                  0 ns   i915/vcs1-wait/                                             
     1.803356979         84,558,765 ns   i915/vcs1-busy/                                             
     1.803356979         35,035,493 ns   i915/vcs1-sema/                                             
     1.803356979                  0 ns   i915/vcs1-wait/                                             
     1.903561910         85,307,891 ns   i915/vcs1-busy/                                             
     1.903561910         80,113,763 ns   i915/vcs1-sema/                                             
     1.903561910                  0 ns   i915/vcs1-wait/                                             
     2.003732306         85,147,056 ns   i915/vcs1-busy/                                             
     2.003732306         40,049,492 ns   i915/vcs1-sema/                                             
     2.003732306                  0 ns   i915/vcs1-wait/                                             
     2.103939868         85,385,397 ns   i915/vcs1-busy/                                             
     2.103939868          5,002,618 ns   i915/vcs1-sema/                                             
     2.103939868                  0 ns   i915/vcs1-wait/                                             
     2.204145664         86,464,111 ns   i915/vcs1-busy/                                             
     2.204145664         34,987,714 ns   i915/vcs1-sema/                                             
     2.204145664                  0 ns   i915/vcs1-wait/                                             
     2.304341045         85,311,024 ns   i915/vcs1-busy/                                             
     2.304341045         94,976,653 ns   i915/vcs1-sema/                                             
     2.304341045                  0 ns   i915/vcs1-wait/                                             
     2.404520380         85,852,106 ns   i915/vcs1-busy/                                             
     2.404520380         34,996,772 ns   i915/vcs1-sema/                                             
     2.404520380                  0 ns   i915/vcs1-wait/                                             
     2.504729578         85,753,160 ns   i915/vcs1-busy/                                             
     2.504729578         44,995,847 ns   i915/vcs1-sema/                                             
     2.504729578                  0 ns   i915/vcs1-wait/                                             
#           time             counts unit events
     2.604946693         83,258,753 ns   i915/vcs1-busy/                                             
     2.604946693         74,990,707 ns   i915/vcs1-sema/                                             
     2.604946693                  0 ns   i915/vcs1-wait/                                             
     2.705148192         85,853,133 ns   i915/vcs1-busy/                                             
     2.705148192         85,146,362 ns   i915/vcs1-sema/                                             
     2.705148192                  0 ns   i915/vcs1-wait/                                             
     2.805344742         85,636,079 ns   i915/vcs1-busy/                                             
     2.805344742         25,033,601 ns   i915/vcs1-sema/                                             
     2.805344742                  0 ns   i915/vcs1-wait/                                             
     2.905514029         83,719,125 ns   i915/vcs1-busy/                                             
     2.905514029         65,043,622 ns   i915/vcs1-sema/                                             
     2.905514029                  0 ns   i915/vcs1-wait/                                             
     3.005722647         85,310,040 ns   i915/vcs1-busy/                                             
     3.005722647         30,037,930 ns   i915/vcs1-sema/                                             
     3.005722647                  0 ns   i915/vcs1-wait/                                             
     3.105932982         85,260,192 ns   i915/vcs1-busy/                                             
     3.105932982         30,031,303 ns   i915/vcs1-sema/                                             
     3.105932982                  0 ns   i915/vcs1-wait/                                             
     3.206102810         85,288,760 ns   i915/vcs1-busy/                                             
     3.206102810         50,078,485 ns   i915/vcs1-sema/                                             
     3.206102810                  0 ns   i915/vcs1-wait/                                             
     3.306279745         85,587,333 ns   i915/vcs1-busy/                                             
     3.306279745         75,027,713 ns   i915/vcs1-sema/                                             
     3.306279745                  0 ns   i915/vcs1-wait/                                             
     3.406493454         85,765,888 ns   i915/vcs1-busy/                                             
     3.406493454         95,011,091 ns   i915/vcs1-sema/                                             
     3.406493454                  0 ns   i915/vcs1-wait/                                             
     3.506691238         85,091,538 ns   i915/vcs1-busy/                                             
     3.506691238         85,025,489 ns   i915/vcs1-sema/                                             
     3.506691238                  0 ns   i915/vcs1-wait/                                             
     3.606898304         85,016,700 ns   i915/vcs1-busy/                                             
     3.606898304         94,998,820 ns   i915/vcs1-sema/                                             
     3.606898304                  0 ns   i915/vcs1-wait/                                             
     3.707108156         85,224,696 ns   i915/vcs1-busy/                                             
     3.707108156         55,041,010 ns   i915/vcs1-sema/                                             
     3.707108156                  0 ns   i915/vcs1-wait/                                             
     3.807313350         84,935,598 ns   i915/vcs1-busy/                                             
     3.807313350         99,971,882 ns   i915/vcs1-sema/                                             
     3.807313350                  0 ns   i915/vcs1-wait/                                             
     3.907520124         86,416,994 ns   i915/vcs1-busy/                                             
     3.907520124         60,218,944 ns   i915/vcs1-sema/                                             
     3.907520124                  0 ns   i915/vcs1-wait/                                             
     4.007730814         85,646,119 ns   i915/vcs1-busy/                                             
     4.007730814         50,073,911 ns   i915/vcs1-sema/                                             
     4.007730814                  0 ns   i915/vcs1-wait/                                             
     4.107904563         85,931,492 ns   i915/vcs1-busy/                                             
     4.107904563         80,000,899 ns   i915/vcs1-sema/                                             
     4.107904563                  0 ns   i915/vcs1-wait/                                             
     4.208073659         85,295,903 ns   i915/vcs1-busy/                                             
     4.208073659         84,978,887 ns   i915/vcs1-sema/                                             
     4.208073659                  0 ns   i915/vcs1-wait/                                             
     4.308284750         85,051,978 ns   i915/vcs1-busy/                                             
     4.308284750         99,963,217 ns   i915/vcs1-sema/                                             
     4.308284750                  0 ns   i915/vcs1-wait/                                             
     4.408481753         86,220,293 ns   i915/vcs1-busy/                                             
     4.408481753        100,000,425 ns   i915/vcs1-sema/                                             
     4.408481753                  0 ns   i915/vcs1-wait/                                             
     4.508689997         85,022,693 ns   i915/vcs1-busy/                                             
     4.508689997         65,139,017 ns   i915/vcs1-sema/                                             
     4.508689997                  0 ns   i915/vcs1-wait/                                             
     4.608895935         84,892,627 ns   i915/vcs1-busy/                                             
     4.608895935         50,069,240 ns   i915/vcs1-sema/                                             
     4.608895935                  0 ns   i915/vcs1-wait/                                             
     4.709103951         85,038,229 ns   i915/vcs1-busy/                                             
     4.709103951         80,109,594 ns   i915/vcs1-sema/                                             
     4.709103951                  0 ns   i915/vcs1-wait/                                             
     4.809279557         85,325,514 ns   i915/vcs1-busy/                                             
     4.809279557         85,050,025 ns   i915/vcs1-sema/                                             
     4.809279557                  0 ns   i915/vcs1-wait/                                             
     4.909499959         85,595,340 ns   i915/vcs1-busy/                                             
     4.909499959         29,999,102 ns   i915/vcs1-sema/                                             
     4.909499959                  0 ns   i915/vcs1-wait/                                             
     5.009715840         86,633,730 ns   i915/vcs1-busy/                                             
     5.009715840         94,996,584 ns   i915/vcs1-sema/                                             
     5.009715840                  0 ns   i915/vcs1-wait/                                             
#           time             counts unit events
     5.109927801         85,274,181 ns   i915/vcs1-busy/                                             
     5.109927801         94,968,127 ns   i915/vcs1-sema/                                             
     5.109927801                  0 ns   i915/vcs1-wait/                                             
     5.210138711         84,449,606 ns   i915/vcs1-busy/                                             
     5.210138711         40,170,670 ns   i915/vcs1-sema/                                             
     5.210138711                  0 ns   i915/vcs1-wait/                                             
     5.310351659         85,629,044 ns   i915/vcs1-busy/                                             
     5.310351659         45,012,072 ns   i915/vcs1-sema/                                             
     5.310351659                  0 ns   i915/vcs1-wait/                                             
     5.410568218         86,679,014 ns   i915/vcs1-busy/                                             
     5.410568218         99,994,658 ns   i915/vcs1-sema/                                             
     5.410568218                  0 ns   i915/vcs1-wait/                                             
     5.510780105         85,682,007 ns   i915/vcs1-busy/                                             
     5.510780105                  0 ns   i915/vcs1-sema/                                             
     5.510780105                  0 ns   i915/vcs1-wait/                                             
     5.610987172         84,631,229 ns   i915/vcs1-busy/                                             
     5.610987172         50,027,357 ns   i915/vcs1-sema/                                             
     5.610987172                  0 ns   i915/vcs1-wait/                                             
     5.711192665         85,008,656 ns   i915/vcs1-busy/                                             
     5.711192665         80,007,406 ns   i915/vcs1-sema/                                             
     5.711192665                  0 ns   i915/vcs1-wait/                                             
     5.811410934         84,773,228 ns   i915/vcs1-busy/                                             
     5.811410934         35,052,545 ns   i915/vcs1-sema/                                             
     5.811410934                  0 ns   i915/vcs1-wait/                                             
     5.911616718         84,900,639 ns   i915/vcs1-busy/                                             
     5.911616718         30,006,841 ns   i915/vcs1-sema/                                             
     5.911616718                  0 ns   i915/vcs1-wait/                                             
     6.011827360         86,234,742 ns   i915/vcs1-busy/                                             
     6.011827360         89,983,850 ns   i915/vcs1-sema/                                             
     6.011827360                  0 ns   i915/vcs1-wait/                                             
     6.081667753         38,056,952 ns   i915/vcs1-busy/                                             
     6.081667753         40,003,467 ns   i915/vcs1-sema/                                             
     6.081667753                  0 ns   i915/vcs1-wait/                                             
{code}

We have performed one another experiment with GPU frequency (only gt_max_freq_mhz, gt_boost_freq_mhz): default (950), 900, 800, 700, 600, 500, 400.
Example:
echo 800 > /sys/class/drm/card0/gt_max_freq_mhz
echo 800 > /sys/class/drm/card0/gt_boost_freq_mhz
Other settings - system defaults.

And only one case, where I see big amount of bad samples - 700 MHz.

As already was discussed in our internal bug, the fix for more-precise-sampling helps:
{code}
From 0a1252963a958aefb09d0113afd0caf3359a4462 Mon Sep 17 00:00:00 2001
From: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
Date: Wed, 11 Sep 2019 17:52:31 +0100
Subject: [PATCH] more precise sampling

Signed-off-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
---
 drivers/gpu/drm/i915/i915_pmu.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/gpu/drm/i915/i915_pmu.c b/drivers/gpu/drm/i915/i915_pmu.c
index 623ad32303a1..9f0bb3b85401 100644
--- a/drivers/gpu/drm/i915/i915_pmu.c
+++ b/drivers/gpu/drm/i915/i915_pmu.c
@@ -17,7 +17,7 @@
 #include "intel_pm.h"
 
 /* Frequency for the sampling timer for events which need it. */
-#define FREQUENCY 200
+#define FREQUENCY 1000
 #define PERIOD max_t(u64, 10000, NSEC_PER_SEC / FREQUENCY)
 
 #define ENGINE_SAMPLE_MASK \
-- 
2.20.1
{code}
Comment 1 Chris Wilson 2019-09-23 20:27:10 UTC
If you disable busy-stats, what's the output then? I'm curious whether the apparent cap of 85% busy is genuine.
Comment 2 Tvrtko Ursulin 2019-09-24 08:28:29 UTC
85% is real, Andrey run that test with 1000Hz sampling. Note that this is with GPU fixed at 700MHz and with default settings utilization is lower (~72% cap but I think more variance than then fixed scenario). We probably shouldn't read too much from this since we don't know how the other engines are looking. One of them could easily be the limiting factor.

But it is interesting that the sampling error is largest in the 700Mhz case. Must be some unlucky latching of batch periods vs 200Hz sampling.

I suggested to clamp sema to busy in the custom monitoring tool to start with.

Question is whether we can afford going to 1000Hz in upstream. It would require testing for CPU usage, or maybe even going to like 500Hz would help.
Comment 3 Chris Wilson 2019-09-24 08:48:18 UTC
Not mentioned is the baseline kernel. I don't think preempt-to-busy has an impact here, as busy-stats is computed from the schedule-in/out events which are separate from unwind(). Just curious though.
Comment 4 Chris Wilson 2019-09-25 09:50:04 UTC
Put together https://patchwork.freedesktop.org/series/67183/ to help investigate this. I think it is just the sampling issue, but that should also be independent of gpu clock frequency.

To be certain of reasonably accuracy with 100ms sample intervals, our estimate is that we want a 1000Hz timer.


Use of freedesktop.org services, including Bugzilla, is subject to our Code of Conduct. How we collect and use information is described in our Privacy Policy.