Bug 104920

Summary: Broken hardware video encoding with vaapi/ffmpeg
Product: Mesa Reporter: Sven Arvidsson <sa>
Component: Drivers/Gallium/radeonsiAssignee: Default DRI bug account <dri-devel>
Status: CLOSED NOTABUG QA Contact: Default DRI bug account <dri-devel>
Severity: normal    
Priority: medium    
Version: 17.3   
Hardware: Other   
OS: All   
Whiteboard:
i915 platform: i915 features:

Description Sven Arvidsson 2018-02-02 20:53:24 UTC
Hi,

I'm trying to encode video using ffmpeg 3.4.1 with the following command:

ffmpeg -vaapi_device /dev/dri/renderD128 -f x11grab -video_size 1920x1080 -i :0 -vf 'format=nv12,hwupload' -c:v h264_vaapi -bf 0 out.mp4

This seems to work, ffmpeg doesn't complain, and I can tell the GPU is active as the fans turn on.

But the video recorded is corrupt and ffplay reports the following errors a bunch of times: 

[h264 @ 0x7f4ae4003060] decode_slice_header error
[h264 @ 0x7f4ae4003060] no frame!
[h264 @ 0x7f4ae4003060] non-existing PPS 0 referenced

and

[mov,mp4,m4a,3gp,3g2,mj2 @ 0x56377e460000] decoding for stream 0 failed
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x56377e460000] Could not find codec parameters for stream 0 (Video: h264 (avc1 / 0x31637661), none, 1920x1080, 1159 kb/s): unspecified pixel format


System environment:
-- system architecture: 64-bit
-- Linux distribution: Debian unstable
-- GPU: TONGA
-- Model: Asus Strix R9 285 2GB
-- Display connector: DVI
-- xf86-video-amdgpu: 1.4.0
-- xserver: 1.19.0
-- mesa: 17.3.3
-- drm: 2.4.89
-- kernel: 4.14
-- dri3
Comment 1 Andy Furniss 2018-02-03 00:23:52 UTC
You need -profile:v 578.

x11grab on my old CPU at least is not the best way to go.
ffmpeg is slow doing the BGR0 -> nv12 conversion, and by default it will be 601 rather than 709.
There are ways around that, and to be a bit faster with some convoluted commands.

I don't know what versions of everything you need, but you can "now" get the GPU to do the conversion and zero copy input, which is way faster and does BGR0 -> rec 709 CSC.

The downside is you need to be root or suid the ffmpeg binary.

1920x1080 is also slightly problematic but using mp4 or mkv will hide it (I guess depending on player). mkv seems to be broken with current ffmpeg git, bah. The issue being that eg. .ts would be seen as 1088 rather than 1080.

To get GPU accel using the newer/faster way would be something like -

ffmpeg -framerate 60 -device /dev/dri/card0 -f kmsgrab -i - -vsync 0 -init_hw_device vaapi=v:/dev/dri/renderD128 -filter_hw_device v -vf 'hwmap,scale_vaapi=format=nv12' -c:v h264_vaapi -profile:v 578 -bf 0 out.mp4
Comment 2 Sven Arvidsson 2018-02-03 18:56:54 UTC
Thank you, -profile:v 578 works fine.

Looks like I had everything needed for the new way with kmsgrab too, which is even better!

Thank you so much for the tip and explanation, much appreciated!

Resolving as NOTABUG as this was a configuration problem/ignorance on my part.
Comment 3 Luke McKee 2018-02-27 19:36:20 UTC
it's -profile v: constrained_baseline --level 3.1 or 3.0 on newer ffmpeg versions.

Even with that I'm having corrupted encoding so I'll open a new ticket.

As a favour can you test your hardware encoded videos with totem (it uses gstreamer) and see if they load. vlc says the first 2 frames don't work but plays, and mpv can play it fine. Hardware players also bork at the encoded content. This is the ffmeg command line I'm using...

GST_DEBUG=3 reports this:
qtdemux qtdemux.c:9719:qtdemux_parse_trak:<qtdemux0> Track shorter than 20% (2881879/24000 vs. 5834998/1000) of the stream found, assuming preview image or something; skipping track

ffmpeg -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format vaapi -ss 120 -t 120 -i a-movie.mkv -map 0:0 -map 0:1 -c:v h264_vaapi -b:v 4000k -qp 20 -bf 0 -profile:v 578 (or constrained_baseline in ffmpeg-git) -movflags +faststart -quality:v 0 -level:v 3.1 -coder:v cavlc -c:a aac -ab 128k -ar 48000 -ac 2 vaapitest.mp4

Here's the full specs on the created ffmpeg file using ffprobe:

https://pastebin.com/TRVM9XiH
Comment 4 Luke McKee 2018-02-27 20:30:48 UTC
See #105277 for more info. People had trouble getting encoding to start working on this ticket - not a bug - but when it works the data may be corrupt and not decodable by qtdemux in gstreamer and other hardware players.

Also vaapi may have trouble decoding what it creates itself - at least in vlc it does not mpv ;)
Comment 5 Sven Arvidsson 2018-02-27 21:19:12 UTC
I'm still using ffmpeg 3.4.1 with -profile:v 578 and Totem plays the videos encoded fine here.
Comment 6 Andy Furniss 2018-02-28 10:54:11 UTC
(In reply to Luke McKee from comment #3)

> ffmpeg -hwaccel vaapi -hwaccel_device /dev/dri/renderD128
> -hwaccel_output_format vaapi -ss 120 -t 120 -i a-movie.mkv -map 0:0 -map 0:1
> -c:v h264_vaapi -b:v 4000k -qp 20 -bf 0 -profile:v 578 (or
> constrained_baseline in ffmpeg-git) -movflags +faststart -quality:v 0
> -level:v 3.1 -coder:v cavlc -c:a aac -ab 128k -ar 48000 -ac 2 vaapitest.mp4

Not related to any issue as such - and I don't have the same h/w to test, but some thoughts related to the command line.

-b:v 4000k -qp 20

I would think one or the other, not both ie. you want constant bitrate or you want constant qp (= highly variable bitrate). I didn't test what ffmpeg takes this to mean.

-coder:v cavlc

I don't think this is hooked up - you will probably get cabac or calvlc depending on h/w.

-quality:v 0

I don't thing this will do anything either on VCE (I don't know anything about the newer engines)
Comment 7 grmat 2018-05-21 16:52:53 UTC
NOTABUG?

IMHO, it's a documentation issue, *at least*. Please add the info to https://wiki.freedesktop.org/xorg/RadeonFeature/ or somewhere else.

Also, why is it possible to use radeonsi/vaapi for encoding that isn't actually supported? Isn't it possible to return a fail value? It shouldn't be the software *using* vaapi being responsible to check if the hardware supports stuff, or am I wrong?
Comment 8 Christian König 2018-05-22 13:01:50 UTC
(In reply to grmat from comment #7)
> NOTABUG?
> 
> IMHO, it's a documentation issue, *at least*. Please add the info to
> https://wiki.freedesktop.org/xorg/RadeonFeature/ or somewhere else.
> 
> Also, why is it possible to use radeonsi/vaapi for encoding that isn't
> actually supported? Isn't it possible to return a fail value? It shouldn't
> be the software *using* vaapi being responsible to check if the hardware
> supports stuff, or am I wrong?

No, that is perfectly correct. VA-API advises to the application what the hardware can do or can't do.

When the application choose to ignore that it is a problem in the application and not the driver.
Comment 9 grmat 2018-05-22 16:18:03 UTC
Thanks, that's what I was (In reply to Christian König from comment #8)
> 
> No, that is perfectly correct. VA-API advises to the application what the
> hardware can do or can't do.
> 
> When the application choose to ignore that it is a problem in the
> application and not the driver.

Thanks, that's what I was asking. So it's a ffmpeg issue, they should automatically select the correct profile for the vaapi_device.

It would still be good to have it documented on RadeonFeature, IMHO.

Use of freedesktop.org services, including Bugzilla, is subject to our Code of Conduct. How we collect and use information is described in our Privacy Policy.