this issue is caused by handling with mmco=4 error.

gstvaapi mmco doesn't comply with the latest h264 spec.Expecially for mmco=3 and mmco=6.

Created attachment 80000 [details] [review]
h264-fix-exec-reference-marking

FRExt_MMCO4_Sony_B.264
MR6_BT_B.h264
MR8_BT_B.h264

Created attachment 81555 [details] [review]
h264-fix-exec-reference-picture-marking-error-another

another new commit for h264 fix exec reference picture marking error.

Created attachment 81556 [details] [review]
h264-fix-reflist-error

core dump bug when decode MR7_BT_B.h264.It's caused by reflist error.this patch is to fix it.
And there is also some ssim low bugs with this clip.

Comment on attachment 80000 [details] [review]
h264-fix-exec-reference-marking

Review of attachment 80000 [details] [review]:
-----------------------------------------------------------------

LGTM. Thanks.

::: gst-libs/gst/vaapi/gstvaapidecoder_h264.c
@@ +2946,4 @@
>  
>      ref_picture->long_term_frame_idx = ref_pic_marking->long_term_frame_idx;
>      gst_vaapi_picture_h264_set_reference(ref_picture,
> +        GST_VAAPI_PICTURE_FLAG_LONG_TERM_REFERENCE, TRUE);

This is GST_VAAPI_PICTURE_IS_COMPLETE().

commit e95a42ea6e9e7235623c71ea2aa2b13c92380ac2
Author: Cong Zhong <congx.zhong@intel.com>
Date:   Fri May 24 19:00:54 2013 +0800

    decoder: h264: fix long-term reference picture marking process.
    
    Fix reference picture marking process with memory_management_control_op
    set to 3 and 6, i.e. assign LongTermFrameIdx to a short-term reference
    picture, or the current picture.
    
    This fixes decoding of FRExt_MMCO4_Sony_B.
    
    https://bugs.freedesktop.org/show_bug.cgi?id=64624
    https://bugzilla.gnome.org/show_bug.cgi?id=724518
    
    [squashed, edited to use GST_VAAPI_PICTURE_IS_COMPLETE() macro]
    Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>