Also dmesg |grep VCE: 

[drm] Found VCE firmware/feedback version 50.0.1 / 17!
[drm] VCE initialized successfully.

After fixing https://bugs.freedesktop.org/show_bug.cgi?id=100972 , 
ffmpeg -hwaccel vaapi -hwaccel_output_format vaapi -i Elephants_Dream_HD.avi -vf format=yuv420p,hwupload -threads 2 -acodec copy  -vaapi_device /dev/dri/renderD128  -bf 0 -c:v h264_vaapi -c:v h264_vaapi -profile:v 77 ~/test.mkv

seems to work (ffmpeg from git master).
Performance is atrocious though : I get about an encoding rate of about 2 FPS.

Hmm, I haven't tested yet, but if you hit the division by zero, I guess there is something special about thet file (or ffmpeg/something changed since I last looked). Before reading your other bug the only way I thought would trigger that one was to explicitly add -g 0 to the command line.

On speed - I would get rid of the yuv420p, maybe add -g 48 as for some reason things don't seem quite right with the gop, or you shouldn't hit the division by zero.

Your fix seems to change the intent of that code a little bit - I don't know if it makes any difference. That code got added to correct some corner case rate control cbr issue - ffmpeg switched to using vbr by default so may not show it anyway. I don't recall what rate (maybe it's cqp) you'll get if you don't ask for anything like your example. There may be a better place to check if gop is 0.

-profile:v 77 doesn't quite work properly IIRC (well it doesn't change anything encoding wise but the file will not show as main).

You may get more perf if you force your GPU to high.

I guess this is just a test, but for real world use, using your h/w to transcode without b-frames is not really an optimum solution and it's up to the firmware team whether b-frames ever work (They don't work on windows either AFAICT).

The file is the OSS Elephant's dream movie : https://orange.blender.org/
Purposely tested with this for easy sharing.
Contained streams (ffmpeg -i ): 
    Stream #0:0: Video: msmpeg4v2 (MP42 / 0x3234504D), yuv420p, 1920x1080, 10002 kb/s, 24 fps, 24 tbr, 24 tbn, 24 tbc
    Stream #0:1: Audio: ac3 ([0] [0][0] / 0x2000), 48000 Hz, 5.1(side), fltp, 448 kb/s

-profile:v 77  : I tested values and in the end found one in ffmpeg sources. Are these values documented anywhere?

B-Frames : so basically VCE is useless in its B-Frame-less state?
I was hoping to create a tvheadend streaming server with live hw-accelerated transcoding, is this at all possible with AMD VCE cards?

I'm not too familiar with VAAPI, but for transcoding, you really need efficient pipelining between the decode and the encode.  If there are CPU copies in the middle, performance won't be great.  We generally recommend using gstreamer with OpenMax using tunneling so that there are no extra copies between the decode and encode stages of the transcode.  I'm not sure if VAAPI supports something like this.

(In reply to Martin Bednar from comment #4)
> The file is the OSS Elephant's dream movie : https://orange.blender.org/
> Purposely tested with this for easy sharing.
> Contained streams (ffmpeg -i ): 
>     Stream #0:0: Video: msmpeg4v2 (MP42 / 0x3234504D), yuv420p, 1920x1080,
> 10002 kb/s, 24 fps, 24 tbr, 24 tbn, 24 tbc
>     Stream #0:1: Audio: ac3 ([0] [0][0] / 0x2000), 48000 Hz, 5.1(side),
> fltp, 448 kb/s

I think ffmpeg will silently fall back to s/w decode with this file as it's not normal h264.

If you don't specify a bitrate it seems ffmpeg will use cqp = 20 (which will come out quite high bitrate on some content).

> -profile:v 77  : I tested values and in the end found one in ffmpeg sources.
> Are these values documented anywhere?

Not sure about ffmpeg, but they are standard numbers in the world of h264

> B-Frames : so basically VCE is useless in its B-Frame-less state?

I wouldn't go that far, for realtime encoding it is useful and libx264 with realtime settings wouldn't use b-frames either (well depends on how much CPU you have available in practice).
For example my card can do 2160p60 realtime in an artificial test - though in practice for say, game/screen recording I would need hardware CSC which Windows may have, but linux doesn't.

> I was hoping to create a tvheadend streaming server with live hw-accelerated
> transcoding, is this at all possible with AMD VCE cards?

If the input is progressive h.264 maybe - VCE doesn't encode interlaced which could be an issue depending on what your local broadcasters use.

TV tends to be quite low bitrate anyway - if you are not reducing size then re-encoding may not be the best way to go.

gstreamer can, for my dual instance VCE card, be faster than ffmpeg, on your APU I don't know whether it would be.

gstreamer can use vaapi (I don't think a current mesa regression affects transcoding). OMX can be used, but it's cqp only, with vaapi you can target bitrates.

-- GitLab Migration Automatic Message --

This bug has been migrated to freedesktop.org's GitLab instance and has been closed from further activity.

You can subscribe and participate further through the new bug through this link to our GitLab instance: https://gitlab.freedesktop.org/mesa/mesa/issues/595.