Created attachment 144954 [details]
Screenshot from callgraph excerpt
With Linux 5.3-rc3 and pm-graph’s `sleepgraph.py`  measuring suspend times on the Dell OptiPlex 5040 with an external AMD graphics card, the driver needs 2.1 seconds to suspend the device, which is too long.
$ lspci -nn -s 01:
01:00.0 VGA compatible controller : Advanced Micro Devices, Inc. [AMD/ATI] Oland XT [Radeon HD 8670 / R7 250/350] [1002:6610] (rev 81)
01:00.1 Audio device : Advanced Micro Devices, Inc. [AMD/ATI] Cape Verde/Pitcairn HDMI Audio [Radeon HD 7700/7800 Series] [1002:aab0]
Here is the time:
0000:01:00.0 suspend (2170.017 ms @ 5037.143762 to 5039.313779)
Looking into the call graph, 1.4 s are spent in `ttm_bo_evict_mm`.
ttm_bo_evict_mm [ttm] (1438.050 ms @ 5037.601941)
As you can see in the attached screenshot, there are some `ttm_bo_handle_move_mem` call which take several milliseconds adding up to the long time.
- ttm_mem_evict_first [ttm] (304.860 ms @ 5037.705809)
mutex_trylock (0.000 ms @ 5037.705809)
+ ttm_bo_del_from_lru [ttm]
- ttm_bo_evict [ttm] (304.857 ms @ 5037.705810)
amdgpu_evict_flags [amdgpu] (0.000 ms @ 5037.705810)
ttm_bo_mem_space [ttm] (0.001 ms @ 5037.705811)
ttm_bo_handle_move_mem [ttm] (304.853 ms @ 5037.705812)
The contents of vram have to be moved to system memory on suspend since vram is powered off. Depending on general memory pressure at suspend time it may take a while to get the contexts of vram into system ram.
(In reply to Alex Deucher from comment #1)
> The contents of vram have to be moved to system memory on suspend since vram
> is powered off. Depending on general memory pressure at suspend time it may
> take a while to get the contexts of vram into system ram.
Just to clarify, the VRAM on the external graphics device is powered off, correct?
Are there any tools to analyze these delays?
(In reply to Paul Menzel from comment #2)
> Just to clarify, the VRAM on the external graphics device is powered off,
> Are there any tools to analyze these delays?
I guess profiling the relevant functions in ttm. See if we are waiting on pages, etc.