Bug 67190

Hi Joe, this is likely a gstreamer-vaapi issue. I have not updated yet to work with Wayland 1.2.x, if anything was needed. I will check it out.

Let me know if you need help with debug, gb! ^_^

Any update on this bug? We need this fixed for demos at LinuxCon NA, Plumbers and a GENIVI event in October and our demos rely on this feature.
Ulf

Wayland 1.2, libva, intel-driver, gstreamer's 1.0 branch, and gstreamer-vaapi works fine for me with a 1280x720 video, however no video output for Sintel.2010.1080p.mp4. Maybe gstreamer_vaapi doesn't handle video cropping  (1920x1088 vs 1920x1080) correctly.

Just a note that the setup and steps to reproduce have been working fine since Wayland 1.0 and 1.1. It is only with 1.2 that this issue appeared.

Is there any update on this issue?

As stated in this bug's description, the following video streams have functioned with gstreamer-vaapi and Wayland < 1.2 for months (I ran them regularly in the course of my daily duties), but *not* with Wayland >= 1.2:

  * http://www.bigbuckbunny.org/index.php/download/
    - The 1920x1080 h264 stream

  * http://www.sintel.org/download
    - The HD1080p stream

I am able to play the 720p streams from those sources, though.

I am seeing that gst-vaapi no longer works for streams around 1080p regardless of Wayland version-- I've tried 1.2.0 and 1.0.2, and in each case gstreamer-vaapi will not play a 1920x1080-ish h264 stream. For now at least, Weston is excused from investigation.

This is given the following stack:

libva libva-1.2.1
intel-driver 1.2.0
gstreamer 1.0.9
gst-plugins-[base,good,bad] 1.0.9
gst-ffmpeg 1.0.9
gstreamer-vaapi 0.5.5

Last known good configuration-- where gst-vaapi played 1920x1080-ish h264 streams:

libva 1.2.0
intel-driver 1.2.0
gstreamer 1.0.7
gst-plugins-base,good,bad] 1.0.7
gst-ffmpeg 1.0.7
gstreamer-vaapi commit 5fe72d9 (pre-0.5.4)

Regression more severe in X than in Wayland. I cannot play back either the 1080 or 720 video streams of Sintel or Big Buck Bunny with the stack specified in comment 8, using the gst pipeline specified in the bug description. In either X or Wayland, using a pure software gst pipeline allows me to play back any of the streams.

This is the "pure software pipeline":
  gst-launch-1.0 filesrc location=Sintel.2010.1080p.mp4 ! \
      qtdemux ! avdec_h264 ! videoconvert ! < ximagesink | waylandsink >


Changing the bug title to be more specific. This would appear to be a regression in the gstreamer-vaapi component, given data collected thus far. Therefore it's deceptive to associate "wayland" with this bug at the present time.

Created attachment 83901 [details]
gst-launch debug output under wayland

Created attachment 83902 [details]
gst-launch debug output under X

Also, for gst-vaapi ext/codecparsers, this is what's checked out:

f90de0a codecparsers: h264: fix calculation of the frame cropping rectangle.

(In reply to comment #10)
> Created attachment 83901 [details]
> gst-launch debug output under wayland

In gst-libs/gst/vaapi/gstvaapiwindow_wayland.c:302, around the "XXX: use VPP to support unusual source and destination rectangles" comment, you could #if 0 / #endif the if() block to unlock you temporarily. Though, this is not something I would push, a proper solution is needed, as indicated. :)

(and this is what is to be available for the next release BTW).

An alternative is to try something in vaapisink too, but this is more involved to validate because there are multiple combinations to consider and that also impact the X11 backend. One solution would be to use the cropped size to guess-estimate the window size. Though, this is also just a workaround because there are situations where vaapisink does not control the allocation (creation) of the window. So, we end up to need a generic solution in GstVaapiWindowWayland::render().

(In reply to comment #11)
> Created attachment 83902 [details]
> gst-launch debug output under X

I believe there is a missing plug-in elements. This looks more like an installation issue on the GStreamer side. It is expected to produce the same issue in Wayland too. If not, this means that you use different GStreamer installations, and one of them has an issue because you definitely don't see that in Wayland, according to the other log.

Could you please report the output of GST_DEBUG=vaapi:5,vaapidecode:5,vaapisink:5 for this one?

Thanks.

Created attachment 84013 [details]
gst-launch additional debug output under X

This is in reply to comment #14.

In that attachment, Xorg and mutter are running.

(In reply to comment #15)
> Created attachment 84013 [details]
> gst-launch additional debug output under X
> 
> This is in reply to comment #14.

Hi, I pushed something to the git master branch. It works, though I am not fully satisfied yet.

Rationale:

- If VPP is not supported, or not available (libva >= 1.2.0, libva-intel-driver >= 1.2.0), then we fallback to displaying the raw surface. This will be wrong in most cases where video cropping or deinterlacing was needed. At least, we now display something and no longer exit.

- If VPP is supported, then an intermediate surface is used to receive the outcome of video processing. Currently, the target surface format is arbitrarily set to YUV 4:2:0 (most probably NV12, under the hoods).

A future optimization is to use another format that would be more suitable to the compositor, e.g. packed YUV 4:2:2 (YUY2). Though, we need to determine a proper threshold/balance between extra memory bandwidth (YUV 4:2:2 requires more bytes), or extra GPU/EU processing power (shaders are used for YUV 4:2:0, but direct sampling from YUV 4:2:2 is possible on most hardware).

Tested on latest upstream bits and the issue is gone.

wayland (master) 1.4.93-0-g8511544
drm (master) heads/master-0-g305478c
mesa (master) heads/master-0-g39ae284
libva (master) heads/master-0-g0337703
intel-driver (master) heads/master-0-gb48ba79
cairo (master) heads/master-0-g85b05e8
libinput (master) heads/master-0-g97af5c3
weston (master) heads/master-0-gbe803ad
gstreamer (master) heads/master-0-g7e60572
gst-plugins-base (master) heads/master-0-g1ca576c
gst-plugins-good (master) heads/master-0-g541a967
gst-plugins-bad (master) heads/master-0-gbd34e62
gst-ffmpeg (master) heads/master-0-g78b64cb
gstreamer-vaapi (master) heads/master-0-gc12d80e