Bug 105277 - ffmpeg using radeonsi vaapi on Polaris21 RX560 creates h264 steams not playable by gstreamer & hw players
Summary: ffmpeg using radeonsi vaapi on Polaris21 RX560 creates h264 steams not playab...
Status: RESOLVED MOVED
Alias: None
Product: Mesa
Classification: Unclassified
Component: Drivers/Gallium/radeonsi (show other bugs)
Version: git
Hardware: x86-64 (AMD64) Linux (All)
: medium normal
Assignee: Default DRI bug account
QA Contact: Default DRI bug account
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2018-02-27 20:20 UTC by Luke McKee
Modified: 2019-09-25 18:03 UTC (History)
1 user (show)

See Also:
i915 platform:
i915 features:


Attachments
Royalty free big buck bunny 30 second video file corrupted by mesa-git ;) (8.18 MB, video/mp4)
2018-02-27 20:20 UTC, Luke McKee
Details
gst-play-1.0 debug log using VAAPI - gstreamer bug #793836 is not at play. (346.61 KB, text/plain)
2018-02-27 21:40 UTC, Luke McKee
Details

Description Luke McKee 2018-02-27 20:20:04 UTC
Created attachment 137664 [details]
Royalty free big buck bunny 30 second video file corrupted by mesa-git ;)

This ticket relates to a comment on a closed bug:
https://bugs.freedesktop.org/show_bug.cgi?id=104920#c3

Hardware required to replicate: RX560, sys-kernel/linux-firmware-20180213::gentoo, ATOM BIOS: 113-C98121-M01, amd-staging latest kernel & mesa-git.

There is some corruption when creating mkv,mp4 or any container in ffmpeg/ffmpeg-git using vaapi to encode. When I used libx264 with exactly the same settings as the encoder (no bframes, constrained baseline etc) the content is playable on hardware players and gstreamer, however when vaapi is used to use the encoding the content is scrambled on hardware players (TCL TV) and gstreamer's qtdemux can not parse the stream. vlc's player always skips the first two frames too.

The error exists regardless of the scale filter, I was originally testing on much higher bitrate source material.

Sample video file used:
https://download.blender.org/peach/bigbuckbunny_movies/

What works with gstreamer/totem & hw players (x264 not relating to mesa)
ffmpeg-git -hwaccel vaapi -hwaccel_output_format vaapi -t 30 -i big_buck_bunny_720p_surround.avi -vf scale_vaapi=w=1366:h=768,hwdownload,format=nv12 -c:v libx264 -b:v 2000k -coder:v 0 -bf 0 -profile:v baseline -level 3.1 -c:a aac -ac 2 -b:a 128k -ar 48000 -movflags +faststart -x264-params opencl x264test.mp4


What doesn't:

ffmpeg-git -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format vaapi -t 30 -i big_buck_bunny_720p_surround.avi -vf scale_vaapi=w=1366:h=768 -c:v h264_vaapi -b:v 2000k -qp 20 -bf 0 -profile:v constrained_baseline -movflags +faststart -quality:v 0 -level:v 3.1 -coder:v cavlc -c:a aac -ab 128k -ar 48000 -ac 2 vaapitest.mp4

Comparing the output of ffprobe -show_format -show_streams shows the files are nearly identical.

diff x264test.txt vaapitest.txt 
27,28c27,28
< is_avc=true
< nal_length_size=4
---
> is_avc=false
> nal_length_size=0
37c37
< bit_rate=2142956
---
> bit_rate=2153505
102c102
< filename=x264test.mp4
---
> filename=vaapitest.mp4
109,110c109,110
< size=8535782
< bit_rate=2274540
---
> size=8575301
> bit_rate=2285071

I am using amd-gpu-staging-next from 2 days ago 4.15-rc4 after the old powerplay cleanup commit, and mesa-git from today.
Comment 1 Julien Isorce 2018-02-27 20:48:07 UTC
Maybe related issue: https://bugzilla.gnome.org/show_bug.cgi?id=793836
Comment 2 Luke McKee 2018-02-27 20:56:01 UTC
I have just found out the source material being matroska avc stream going in is triggering the copyrighted content is required to replicate it. The big buck bunny transcoded file works fine on totem I just found out. I assumed it would be broken too. I'll attach 10 seconds of a my little pony blueray corrupted file I transcoded for my daughter that breaks totem / hardware players, and because it was created with x264/mkvmerge I'll include the source materials encoding settings below to replicate the defect. I think only 10 seconds would be fair use. Encoded with exactly the same parameters as above.

https://mega.nz/#!nqISgDLA!rvGIBMI8wCYBL7An4Fi9Az3VCINu3AvKxOMgj5f_gQo
2.6mb vaapitest2.mkv


General

Format                                   : Matroska
Format version                           : Version 4 / Version 2
File size                                : 4.37 GiB
Duration                                 : 1 h 39 min
Overall bit rate                         : 6 302 kb/s
Encoded date                             : UTC 2017-12-30 18:42:40
Writing application                      : mkvmerge v8.5.2 ('DiAMOND') 64bit
Writing library                          : libebml v1.3.3 + libmatroska v1.4.4

Video
ID                                       : 1
Format                                   : AVC
Format/Info                              : Advanced Video Codec
Format profile                           : High@L4.1
Format settings                          : CABAC / 5 Ref Frames
Format settings, CABAC                   : Yes
Format settings, ReFrames                : 5 frames
Codec ID                                 : V_MPEG4/ISO/AVC
Duration                                 : 1 h 39 min
Bit rate                                 : 5 531 kb/s
Width                                    : 1 920 pixels
Height                                   : 808 pixels
Display aspect ratio                     : 2.40:1
Frame rate mode                          : Constant
Frame rate                               : 23.976 (24000/1001) FPS
Color space                              : YUV
Chroma subsampling                       : 4:2:0
Bit depth                                : 8 bits
Scan type                                : Progressive
Bits/(Pixel*Frame)                       : 0.149
Stream size                              : 3.75 GiB (86%)
Writing library                          : x264 core 152 r2851 ba24899
Encoding settings                        : cabac=1 / ref=5 / deblock=1:0:0 / analyse=0x3:0x113 / me=umh / subme=8 / psy=1 / psy_rd=1.00:0.00 / mixed_ref=1 / me_range=16 / chroma_me=1 / trellis=1 / 8x8dct=1 / cqm=0 / deadzone=21,11 / fast_pskip=1 / chroma_qp_offset=-2 / threads=24 / lookahead_threads=4 / sliced_threads=0 / nr=0 / decimate=1 / interlaced=0 / bluray_compat=0 / constrained_intra=0 / bframes=3 / b_pyramid=2 / b_adapt=1 / b_bias=0 / direct=3 / weightb=1 / open_gop=0 / weightp=2 / keyint=240 / keyint_min=24 / scenecut=40 / intra_refresh=0 / rc_lookahead=50 / rc=2pass / mbtree=1 / bitrate=5531 / ratetol=1.0 / qcomp=0.60 / qpmin=0 / qpmax=69 / qpstep=4 / cplxblur=20.0 / qblur=0.5 / ip_ratio=1.40 / aq=1:1.00
Language                                 : English
Default                                  : Yes
Forced                                   : No

Audio
ID                                       : 2
Format                                   : DTS
Format/Info                              : Digital Theater Systems
Codec ID                                 : A_DTS
Duration                                 : 1 h 39 min
Bit rate mode                            : Constant
Bit rate                                 : 768 kb/s
Channel(s)                               : 6 channels
Channel positions                        : Front: L C R, Side: L R, LFE
Sampling rate                            : 48.0 kHz
Frame rate                               : 93.750 FPS (512 SPF)
Bit depth                                : 24 bits
Compression mode                         : Lossy
Stream size                              : 545 MiB (12%)
Language                                 : English
Default                                  : Yes
Forced                                   : No

Text #1
ID                                       : 3
Format                                   : UTF-8
Codec ID                                 : S_TEXT/UTF8
Codec ID/Info                            : UTF-8 Plain Text
Title                                    : English regular
Language                                 : English
Default                                  : No
Forced                                   : No

Text #2
ID                                       : 4
Format                                   : UTF-8
Codec ID                                 : S_TEXT/UTF8
Codec ID/Info                            : UTF-8 Plain Text
Title                                    : English SDH
Language                                 : English
Default                                  : No
Forced                                   : No
Comment 3 Luke McKee 2018-02-27 21:04:45 UTC
(In reply to Julien Isorce from comment #1)
> Maybe related issue: https://bugzilla.gnome.org/show_bug.cgi?id=793836

You may be right. That shows you gst can't play back content it encoded itself using vaapi on amd's. I was using ffmpeg. This shows the ball is in #drm-devel's court.

gst-play-1.0 vaapitest2.mp4 
Press 'k' to see a list of keyboard shortcuts.
Now playing /mnt/tmp/movies/vaapitest2.mp4
ERROR This file contains no playable streams. for file:///mnt/tmp/movies/vaapitest2.mp4
ERROR debug information: /var/tmp/portage/media-libs/gst-plugins-good-1.12.4/work/gst-plugins-good-1.12.4/gst/isomp4/qtdemux.c(703): gst_qtdemux_post_no_playable_stream_error (): /GstPlayBin:playbin/GstURIDecodeBin:uridecodebin0/GstDecodeBin:decodebin0/GstQTDemux:qtdemux0:
no known streams found
Reached end of play list

I'll try encoding with gstreamer-1.0 and update the ticket.
Comment 4 Luke McKee 2018-02-27 21:36:28 UTC
(In reply to Julien Isorce from comment #1)
> Maybe related issue: https://bugzilla.gnome.org/show_bug.cgi?id=793836

No it's working fine here with gst-play-1.0 using omx (i hacked the priorities in gstreamer to use omx decoding over vaapi).  (media-plugins/gst-plugins-omx-1.12.4)

So now more testing for good luck..... 

<10 mins later>

I'll attach a gstreamer debug log for playing that file with omx,vaapi, and xv/ffmpeg if you are curious. That proves 100% this issue isn't related to that bug at all.


That ticket was a vaapi decoding issue. The test vector they uploaded also plays here without problems with gstreamer 1.12.4. totem seems to crash on it though but it's probably something to do with clutter and a very short file.

I made a longer file with this:
gst-launch-1.0 videotestsrc pattern=blink num-buffers=30   ! video/x-raw, framerate=30/1   ! x264enc key-int-max=1   ! mp4mux faststart=true movie-timescale=300 trak-timescale=30   ! filesink location=i-frames-non-frag.mp4
Comment 5 Luke McKee 2018-02-27 21:40:19 UTC
Created attachment 137670 [details]
gst-play-1.0 debug log using VAAPI - gstreamer bug #793836 is not at play.

I also have similar logs for xv,omx - works fine here.

gst-plugins-vaapi here is of course merged without egl support btw, because I don't think that's working yet with amd & mesa.
Comment 6 Luke McKee 2018-02-27 22:53:15 UTC
#ffmpeg dev jkqxz suggested i check a few things.

ffmpeg-git -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format vaapi -t 60 -i pony.mkv -map 0:0 -vf scale_vaapi=w=1366:h=768 -c:v h264_vaapi -b:v 2000k -qp 20 -bf 0 -profile:v constrained_baseline -sn -quality:v 0 -level:v 3.1 -coder:v cavlc -an -sn vaapitest3-intermediate.h264; ffmpeg-git -i vaapitest3-intermediate.h264 -t 60 -i pony.mkv -map 0:0 -map 1:1 -vcodec copy -movflags +faststart -c:a aac -ab 128k -ar 48000 -ac 2 -sn vaapitest3.mp4

ffmpeg said this:
"[mp4 @ 0x55a35c67acc0] Timestamps are unset in a packet for stream 0 (the video stream created by mesa-git vaapi). This is deprecated and will stop working in the future. Fix your code to set the timestamps properly"

^^^ 
Is this our problem?


This above creates an A/V file that is still broken. We let ffmpeg mux stream to a container after it is created to see if it was a container problem.

Another test:
ffmpeg-git -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format vaapi -ss 120 -t 10 -i rep-mylittleponythemovie.2017.1080p.bluray.x264.mkv -vf scale_vaapi=w=1366:h=768 -c:v h264_vaapi -b:v 2000k -qp 20 -bf 0 -profile:v constrained_baseline -quality:v 0 -level:v 3.1 -coder:v cavlc -an vaapitest5-intermediate.h264; ffmpeg -i vaapitest5-intermediate.h264 -c:v copy vaapitest5.mp4 

creates a file that is playable by gstreamer with no audio.

Now I've tested vaapitest[2,3,5].mp4 on a hardware player. The results are presently surprising for me. I've found a workaround for what I set out to achieve - to hardware encode for a tv with a usb socket. Now to set up a usb gadget mode on the router ;)

vaapitest2.mp4 (the mega.nz shared file) doesn't play on hardware player ""not supported" or gstreamer/totem.
vappitest3.mp4 (plays on the hardware player! but not gstreamer!)
vaapitest5.mp4 (plays on both, but hardware player prints warning about only one stream the whole time it's playing - no audio)
All 3 are playable my ffmpeg / chromium.

I suggest the devs look into how timestamps are handled by the h264 stream and please keep up the good work towards getting h265 working!

This may also be a possible thing to check:
"<jkqxz> The Mesa VAAPI implementation doesn't support packed headers, so you end up with no extradata in the file if you mux directly to mp4 or other format with global headers."

Please leave the ticket open. There is still something broken here, but this explains why this slipped through functional testing.

I have a haswell with integrated graphics so I will repeat the tests using intel's vaapi implementation with the same encoding settings to see if the intel vaapi encoded streams are broken too in the coming days.
Comment 7 Luke McKee 2018-02-28 02:07:27 UTC
Final note before I wash my hands of this matter and leave it to the devs to fix.

I also just tried this with the new main / high profile options and cabac enabled that is now available in the more bleeding edge amd kernels / mesa / libva / ffmpeg. 

The breakage on the hardware player and gstermer qtdemux still applies. 

new h265 vaapi support is dangerous too.
https://bugs.freedesktop.org/show_bug.cgi?id=103953#c7
Comment 8 GitLab Migration User 2019-09-25 18:03:21 UTC
-- GitLab Migration Automatic Message --

This bug has been migrated to freedesktop.org's GitLab instance and has been closed from further activity.

You can subscribe and participate further through the new bug through this link to our GitLab instance: https://gitlab.freedesktop.org/mesa/mesa/issues/1306.


Use of freedesktop.org services, including Bugzilla, is subject to our Code of Conduct. How we collect and use information is described in our Privacy Policy.