Bug 110673 - amdgpu hevc encoding problems: segment fault and contents of garbage
Summary: amdgpu hevc encoding problems: segment fault and contents of garbage
Status: RESOLVED FIXED
Alias: None
Product: Mesa
Classification: Unclassified
Component: Other (show other bugs)
Version: unspecified
Hardware: All Linux (All)
: medium normal
Assignee: mesa-dev
QA Contact: mesa-dev
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2019-05-14 03:19 UTC by zhoulei
Modified: 2019-07-10 12:47 UTC (History)
1 user (show)

See Also:
i915 platform:
i915 features:


Attachments

Description zhoulei 2019-05-14 03:19:01 UTC
I have tested amdgpu hevc encoding via libva few days ago.
And I found two problems.

Hardware: R5 2400g(VCN), rx580/vega64(UVD_ENC)

1. intra_period must >= 8 otherwise segment fault in mesa.

I found these codes in mesa using gdb:

radeon_uvd_enc.c/radeon_vcn_enc.c:

enc->enc_pic.max_poc = pic->seq.intra_period;
for (int i = enc->enc_pic.max_poc; i != 0; enc->enc_pic.log2_max_poc++)
    i = (i >> 1);

radeon_uvd_enc_1_1.c/radeon_vcn_enc_1_2.c:

radeon_uvd_enc_code_ue(enc, enc->enc_pic.log2_max_poc - 4);
radeon_enc_code_ue(enc, enc->enc_pic.log2_max_poc - 4);

If intra_period < 8, (enc->enc_pic.log2_max_poc - 4) is negative.
cs array will overflow in radeon_uvd_enc_code_ue or radeon_enc_code_ue.

ffmpeg can reproduce this problem by changing "-g" parameter value.

ffmpeg -vaapi_device /dev/dri/renderD128 -i input.avi -vf 'format=nv12,hwupload' -c:v hevc_vaapi -bf 0 -g 7 output.mp4

2. Hevc encoding only works correctly in CQP mode.

I have tested CBR and VBR, both of them produce contents of garbage, green or black mostly.

ffmpeg can reproduce this problem.

ffmpeg -vaapi_device /dev/dri/renderD128 -i input.avi -vf 'format=nv12,hwupload' -c:v hevc_vaapi -bf 0 -b:v 5M output.mp4
Comment 1 Boyuan Zhang 2019-05-17 23:59:48 UTC
Hi Zhoulei,

Thanks for pointing out these 2 issues.

I reproduced issue#1, and have a fix submitted for code review: https://lists.freedesktop.org/archives/mesa-dev/2019-May/219392.html

Please give a try.

And I will investigate issue#2.
Comment 2 zhoulei 2019-05-18 07:12:59 UTC
(In reply to Boyuan Zhang from comment #1)
> Hi Zhoulei,
> 
> Thanks for pointing out these 2 issues.
> 
> I reproduced issue#1, and have a fix submitted for code review:
> https://lists.freedesktop.org/archives/mesa-dev/2019-May/219392.html
> 
> Please give a try.
> 
> And I will investigate issue#2.

Hi Boyuan,

I have tried the patch in my R5 2400G(ArchLinux, mesa-19.0.4).

Segment fault is gone, but new issue is found.

reproduce: 

No more segment fault when using ffmpeg to produce hevc stream.

ffmpeg -vaapi_device /dev/dri/renderD128 -i input.avi -vf 'format=nv12,hwupload' -c:v hevc_vaapi -bf 0 -g 7 output.mp4

But Some error messages is shown when decoding the encoded stream.

$ mpv output.mp4 
Playing: output.mp4
 (+) Video --vid=1 (*) (hevc 1920x1080 23.976fps)
 (+) Audio --aid=1 (*) (aac 2ch 48000Hz)
Using hardware decoding (vaapi).
AO: [pulse] 48000Hz stereo 2ch float
VO: [vaapi] 1920x1088 vaapi[nv12]
AV: 00:00:00 / 00:00:04 (4%) A-V:  0.006 DS: 2.857/0
[ffmpeg/video] hevc: Could not find ref with POC 7
AV: 00:00:00 / 00:00:04 (11%) A-V:  0.004 DS: 2.643/0
[ffmpeg/video] hevc: Could not find ref with POC 14
AV: 00:00:00 / 00:00:04 (12%) A-V:  0.004 DS: 2.667/0
[ffmpeg/video] hevc: Could not find ref with POC -1
AV: 00:00:00 / 00:00:04 (17%) A-V:  0.004 DS: 2.619/0
[ffmpeg/video] hevc: Could not find ref with POC 5
AV: 00:00:01 / 00:00:04 (24%) A-V:  0.005 DS: 2.571/0
[ffmpeg/video] hevc: Could not find ref with POC 12
AV: 00:00:01 / 00:00:04 (26%) A-V:  0.005 DS: 2.581/0
[ffmpeg/video] hevc: Could not find ref with POC -1
AV: 00:00:01 / 00:00:04 (30%) A-V:  0.005 DS: 2.571/0
[ffmpeg/video] hevc: Could not find ref with POC 3
AV: 00:00:01 / 00:00:04 (37%) A-V:  0.006 DS: 2.548/0
[ffmpeg/video] hevc: Could not find ref with POC 10
AV: 00:00:01 / 00:00:04 (41%) A-V:  0.006 DS: 2.532/0
[ffmpeg/video] hevc: Could not find ref with POC -1
AV: 00:00:01 / 00:00:04 (43%) A-V:  0.006 DS: 2.531/0
[ffmpeg/video] hevc: Could not find ref with POC 1
AV: 00:00:02 / 00:00:04 (50%) A-V:  0.005 DS: 2.536/0
[ffmpeg/video] hevc: Could not find ref with POC 8
AV: 00:00:02 / 00:00:04 (56%) A-V:  0.005 DS: 2.524/0
[ffmpeg/video] hevc: Duplicate POC in a sequence: 0.
[ffmpeg/video] hevc: Error parsing NAL unit #1.
Error while decoding frame!
AV: 00:00:02 / 00:00:04 (62%) A-V:  0.005 DS: 2.565/0
[ffmpeg/video] hevc: Could not find ref with POC 6
AV: 00:00:03 / 00:00:04 (69%) A-V:  0.005 DS: 2.566/0
[ffmpeg/video] hevc: Could not find ref with POC 13
AV: 00:00:03 / 00:00:04 (71%) A-V:  0.004 DS: 2.564/0
[ffmpeg/video] hevc: Could not find ref with POC -1
AV: 00:00:03 / 00:00:04 (75%) A-V:  0.004 DS: 2.554/0
[ffmpeg/video] hevc: Could not find ref with POC 4
AV: 00:00:03 / 00:00:04 (82%) A-V:  0.004 DS: 2.556/0
[ffmpeg/video] hevc: Could not find ref with POC 11
AV: 00:00:03 / 00:00:04 (86%) A-V:  0.004 DS: 2.553/0
[ffmpeg/video] hevc: Could not find ref with POC -1
AV: 00:00:04 / 00:00:04 (88%) A-V:  0.003 DS: 2.546/0
[ffmpeg/video] hevc: Could not find ref with POC 2
AV: 00:00:04 / 00:00:04 (93%) A-V:  0.000 DS: 2.545/0
Comment 3 Boyuan Zhang 2019-05-27 18:50:50 UTC
Hi Zhoulei,

Please use the latest patch to test again: https://lists.freedesktop.org/archives/mesa-dev/2019-May/219613.html

Regards,
Boyuan
Comment 4 zhoulei 2019-05-28 01:10:50 UTC
(In reply to Boyuan Zhang from comment #3)
> Hi Zhoulei,
> 
> Please use the latest patch to test again:
> https://lists.freedesktop.org/archives/mesa-dev/2019-May/219613.html
> 
> Regards,
> Boyuan

I have tested this patch on RX580, it works.

Issue#1 has been fixed. 

Thanks.
Comment 5 Boyuan Zhang 2019-06-17 19:11:04 UTC
Hi Zhoulei,

I fixed the second issue you reported (cbr/vbr corruption for hevc encoding).

Please give a try using this patch:
https://lists.freedesktop.org/archives/mesa-dev/2019-June/220304.html

Thanks,
Boyuan
Comment 6 zhoulei 2019-06-19 07:58:55 UTC
(In reply to Boyuan Zhang from comment #5)
> Hi Zhoulei,
> 
> I fixed the second issue you reported (cbr/vbr corruption for hevc encoding).
> 
> Please give a try using this patch:
> https://lists.freedesktop.org/archives/mesa-dev/2019-June/220304.html
> 
> Thanks,
> Boyuan

Hi Boyuan:

I have tested these patches on RX580, it works.

In addition:

Changes in radeon_vcn_enc_1_2.c cannot applied.

It looks like a typo.

RENC_UVD_RATE_CONTROL_METHOD_NONE, radeon_uvd_enc_code_fixed_bits and

radeon_uvd_enc_code_ue are defined in radeon_uvd_enc.h.

shoud use marco and funtions defined in radeon_vcn_enc.h.
Comment 7 Boyuan Zhang 2019-06-19 21:08:26 UTC
(In reply to zhoulei from comment #6)
> (In reply to Boyuan Zhang from comment #5)
> > Hi Zhoulei,
> > 
> > I fixed the second issue you reported (cbr/vbr corruption for hevc encoding).
> > 
> > Please give a try using this patch:
> > https://lists.freedesktop.org/archives/mesa-dev/2019-June/220304.html
> > 
> > Thanks,
> > Boyuan
> 
> Hi Boyuan:
> 
> I have tested these patches on RX580, it works.
> 
> In addition:
> 
> Changes in radeon_vcn_enc_1_2.c cannot applied.
> 
> It looks like a typo.
> 
> RENC_UVD_RATE_CONTROL_METHOD_NONE, radeon_uvd_enc_code_fixed_bits and
> 
> radeon_uvd_enc_code_ue are defined in radeon_uvd_enc.h.
> 
> shoud use marco and funtions defined in radeon_vcn_enc.h.


Thanks for your testing Zhoulei.

The typo has been fixed in the patch V2: https://lists.freedesktop.org/archives/mesa-dev/2019-June/220450.html

Regards,
Boyuan
Comment 8 zhoulei 2019-06-20 00:57:18 UTC
Issues have been fixed with these two set of patches:

https://lists.freedesktop.org/archives/mesa-dev/2019-May/219673.html

https://lists.freedesktop.org/archives/mesa-dev/2019-June/220450.html

Mark as RESOLVED.
Comment 9 gregory shu 2019-07-10 12:47:36 UTC
(In reply to zhoulei from comment #8)
> Issues have been fixed with these two set of patches:
> 
> https://lists.freedesktop.org/archives/mesa-dev/2019-May/219673.html
> 
> https://lists.freedesktop.org/archives/mesa-dev/2019-June/220450.html
> 
> Mark as RESOLVED.

Not sure if I should post new one as it seems related, I'm on 2200G I'm still getting garbled green encode output with the patches applied, when input needs conversion from 10bit.


libva info: VA-API version 1.6.0
libva info: va_getDriverName() returns 0
libva info: Trying to open /usr/lib64/va/drivers/radeonsi_drv_video.so
libva info: Found init function __vaDriverInit_1_6
libva info: va_openDriver() returns 0
vainfo: VA-API version: 1.6 (libva 2.6.0.pre1)
vainfo: Driver version: Mesa Gallium driver 19.2.0-devel for AMD RAVEN (DRM 3.32.0, 5.2.0-gentoo, LLVM 8.0.0)
vainfo: Supported profile and entrypoints
      VAProfileMPEG2Simple            : VAEntrypointVLD
      VAProfileMPEG2Main              : VAEntrypointVLD
      VAProfileVC1Simple              : VAEntrypointVLD
      VAProfileVC1Main                : VAEntrypointVLD
      VAProfileVC1Advanced            : VAEntrypointVLD
      VAProfileH264ConstrainedBaseline: VAEntrypointVLD
      VAProfileH264ConstrainedBaseline: VAEntrypointEncSlice
      VAProfileH264Main               : VAEntrypointVLD
      VAProfileH264Main               : VAEntrypointEncSlice
      VAProfileH264High               : VAEntrypointVLD
      VAProfileH264High               : VAEntrypointEncSlice
      VAProfileHEVCMain               : VAEntrypointVLD
      VAProfileHEVCMain               : VAEntrypointEncSlice
      VAProfileHEVCMain10             : VAEntrypointVLD
      VAProfileJPEGBaseline           : VAEntrypointVLD
      VAProfileVP9Profile0            : VAEntrypointVLD
      VAProfileVP9Profile2            : VAEntrypointVLD
      VAProfileNone                   : VAEntrypointVideoProc

E.g. this command:

ffmpeg -threads 4 \
-init_hw_device vaapi=amd:/dev/dri/renderD128 -hwaccel vaapi -hwaccel_output_format vaapi -hwaccel_device amd -filter_hw_device amd \
-i nexp.mkv'     \
-vf "scale_vaapi=format=nv12" \
-c:v h264_vaapi -profile:v 578 \
-c:a copy  -c:s copy \
-f mpegts -y plop.mkv

input file is a 10bit hevc


Use of freedesktop.org services, including Bugzilla, is subject to our Code of Conduct. How we collect and use information is described in our Privacy Policy.