Bug 109538 - VAAPI HEVC encoding is unstable and produces garbled output
Summary: VAAPI HEVC encoding is unstable and produces garbled output
Alias: None
Product: DRI
Classification: Unclassified
Component: DRM/AMDgpu (show other bugs)
Version: DRI git
Hardware: x86-64 (AMD64) Linux (All)
: medium major
Assignee: Default DRI bug account
QA Contact:
Depends on:
Reported: 2019-02-03 12:57 UTC by tempel.julian
Modified: 2019-11-19 09:12 UTC (History)
1 user (show)

See Also:
i915 platform:
i915 features:


Description tempel.julian 2019-02-03 12:57:55 UTC

ffmpeg -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format vaapi -i 1080p_h.264_60fps.mp4 -c:v hevc_vaapi -qp 22 -tier high -c:a copy output.mp4

Several times and browse the web with Firefox while doing so. Sooner or later (rather sooner) the encoding will either hang or the driver crashes completely and I need to perform a hard reset.
The encoding settings don't seem to matter as long -c:v hevc_vaapi is used.
-c:v h264_vaapi seems to be totally stable.

The encoding result of -c:v hevc_vaapi also shows a garbled line at the bottom of the video, while -c:v h264_vaapi looks fine.

This is with a RX 580
mesa-git 19.1.0_devel.107212.ab940b0d972
linux 4.20.6 & drm-next-5.1-wip f9028b9278422fdf186f1b88662e28ed24e13df8
libva 2.4
ffmpeg-git b429c86d84cfc964c84bc68ec56799802de2b25a

There seems to be a difference between linux 4.20.6 & drm-next-5.1-wip though. I couldn't make the stable kernel crash entirely, instead just the encoding process started to hang.
Comment 1 gregory shu 2019-07-10 12:54:21 UTC
I'm doing it the other way around hevc_vaapi 10 bit to h264_vaapi and have similar results, garbled green output

ffmpeg -threads 4 \
-init_hw_device vaapi=amd:/dev/dri/renderD128 -hwaccel vaapi -hwaccel_output_format vaapi -hwaccel_device amd -filter_hw_device amd \
-i nexp.mkv'     \
-vf "scale_vaapi=format=nv12" \
-c:v h264_vaapi -profile:v 578 \
-c:a copy  -c:s copy \
-f mpegts -y plop.mkv

libva info: VA-API version 1.6.0
libva info: va_getDriverName() returns 0
libva info: Trying to open /usr/lib64/va/drivers/radeonsi_drv_video.so
libva info: Found init function __vaDriverInit_1_6
libva info: va_openDriver() returns 0
vainfo: VA-API version: 1.6 (libva 2.6.0.pre1)
vainfo: Driver version: Mesa Gallium driver 19.2.0-devel for AMD RAVEN (DRM 3.32.0, 5.2.0-gentoo, LLVM 8.0.0)
vainfo: Supported profile and entrypoints
      VAProfileMPEG2Simple            : VAEntrypointVLD
      VAProfileMPEG2Main              : VAEntrypointVLD
      VAProfileVC1Simple              : VAEntrypointVLD
      VAProfileVC1Main                : VAEntrypointVLD
      VAProfileVC1Advanced            : VAEntrypointVLD
      VAProfileH264ConstrainedBaseline: VAEntrypointVLD
      VAProfileH264ConstrainedBaseline: VAEntrypointEncSlice
      VAProfileH264Main               : VAEntrypointVLD
      VAProfileH264Main               : VAEntrypointEncSlice
      VAProfileH264High               : VAEntrypointVLD
      VAProfileH264High               : VAEntrypointEncSlice
      VAProfileHEVCMain               : VAEntrypointVLD
      VAProfileHEVCMain               : VAEntrypointEncSlice
      VAProfileHEVCMain10             : VAEntrypointVLD
      VAProfileJPEGBaseline           : VAEntrypointVLD
      VAProfileVP9Profile0            : VAEntrypointVLD
      VAProfileVP9Profile2            : VAEntrypointVLD
Comment 2 tempel.julian 2019-08-02 20:10:54 UTC
Doesn't crash anymore, but there is still a garbled line at the bottom of video frames.
Comment 3 Martin Peres 2019-11-19 09:12:16 UTC
-- GitLab Migration Automatic Message --

This bug has been migrated to freedesktop.org's GitLab instance and has been closed from further activity.

You can subscribe and participate further through the new bug through this link to our GitLab instance: https://gitlab.freedesktop.org/drm/amd/issues/689.

Use of freedesktop.org services, including Bugzilla, is subject to our Code of Conduct. How we collect and use information is described in our Privacy Policy.