Fixed in git master branch. Flush semantics are now implemented to support EOS events.

We add  printf(" %s, %s, %d\n", __FILE__, __FUNCTION__, __LINE__);  in libva/va/x11/va_x11.c , then play the stream by gst-vaapi and got display frame count:

Filename              Framecount     Display Framecount
CVPA1_TOSHIBA_B.264      90                78
CABREF3_Sand_D.264       50                35
HVLCFI0_Sony_B.264       19                7
CVWP1_TOSHIBA_E.264      90                9
CAFI1_SVA_C.264          33                31
CAPA1_TOSHIBA_B.264      90                20
MR2_TANDBERG_E.264       300               284

So we found that lost frames with these files.

(In reply to comment #2)
> We add  printf(" %s, %s, %d\n", __FILE__, __FUNCTION__, __LINE__);  in
> libva/va/x11/va_x11.c , then play the stream by gst-vaapi and got display
> frame count:

Instrumenting vaPutSurface() is not a good location to count the number of decoded frames. A more appropriate location to increment the counter is gstvaapidecode.c after a successful gst_video_decoder_finish_frame().

> Filename              Framecount     Display Framecount
> CVPA1_TOSHIBA_B.264      90                78
> CABREF3_Sand_D.264       50                35
> HVLCFI0_Sony_B.264       19                7
> CVWP1_TOSHIBA_E.264      90                9
> CAFI1_SVA_C.264          33                31
> CAPA1_TOSHIBA_B.264      90                20
> MR2_TANDBERG_E.264       300               284
> 
> So we found that lost frames with these files.

This is weird, please make sure you have built and installed from the latest git master sources. Thanks.

Hi, what gives the following command?
$ grep USE_LOCAL_CODEC_PARSERS config.status

S["USE_LOCAL_CODEC_PARSERS_JPEG_FALSE"]="#"
S["USE_LOCAL_CODEC_PARSERS_JPEG_TRUE"]=""
S["USE_LOCAL_CODEC_PARSERS_H264_FALSE"]="#"
S["USE_LOCAL_CODEC_PARSERS_H264_TRUE"]=""
S["USE_LOCAL_CODEC_PARSERS_MPEG2_FALSE"]="#"
S["USE_LOCAL_CODEC_PARSERS_MPEG2_TRUE"]=""

S["USE_LOCAL_CODEC_PARSERS_JPEG_FALSE"]="#"
S["USE_LOCAL_CODEC_PARSERS_JPEG_TRUE"]=""
S["USE_LOCAL_CODEC_PARSERS_H264_FALSE"]="#"
S["USE_LOCAL_CODEC_PARSERS_H264_TRUE"]=""
S["USE_LOCAL_CODEC_PARSERS_MPEG2_FALSE"]="#"
S["USE_LOCAL_CODEC_PARSERS_MPEG2_TRUE"]=""
(In reply to comment #4)
> Hi, what gives the following command?
$ grep USE_LOCAL_CODEC_PARSERS
> config.status

Hi,

(In reply to comment #2)
> We add  printf(" %s, %s, %d\n", __FILE__, __FUNCTION__, __LINE__);  in
> libva/va/x11/va_x11.c , then play the stream by gst-vaapi and got display
> frame count:
> 
> Filename              Framecount     Display Framecount
> CVPA1_TOSHIBA_B.264      90                78
> CABREF3_Sand_D.264       50                35
> HVLCFI0_Sony_B.264       19                7
> CVWP1_TOSHIBA_E.264      90                9
> CAFI1_SVA_C.264          33                31
> CAPA1_TOSHIBA_B.264      90                20
> MR2_TANDBERG_E.264       300               284
> 
> So we found that lost frames with these files.

Please re-run the tests from git master branch, per comment #1. Please also make sure to run the test under the following environment: GST_DEBUG=vaapi:3,vaapidecode:3

Created attachment 73490 [details]
GST_DEBUG=vaapi:3,vaapidecode:3 log

(In reply to comment #8)
> Created attachment 73490 [details]
> GST_DEBUG=vaapi:3,vaapidecode:3 log

What clip was that for? What were the results? Besides, it seems the log references codecparsers_h264:5 instead of vaapidecode:3.

Created attachment 73491 [details]
log for CVPA1_TOSHIBA_B.264 clip

(In reply to comment #10)
> Created attachment 73491 [details]
> log for CVPA1_TOSHIBA_B.264 clip

CVPA1_TOSHIBA_B.264 does not show any issue here.
$ cd ext/codecparsers
$ git log --oneline -n 1
?

How have you rebuilt gstreamer-vaapi?

(In reply to comment #11)
> (In reply to comment #10)
> Created attachment 73491 [details]
> log for
> CVPA1_TOSHIBA_B.264 clip

CVPA1_TOSHIBA_B.264 does not show any issue here.
> $ cd ext/codecparsers
$ git log --oneline -n 1
?
5d33da8 codecparsers: vc1: fix bitplanes decoding.

How have you rebuilt
> gstreamer-vaapi?

$ ./autogen.sh --enable-gtk-doc --enable-wayland=0
$ make 
$ make install

$ gst-launch filesrc location=/Videos/h264/conformance/Base_Ext_Main_profiles/CVPA1_TOSHIBA_B.264 ! h264parse ! vaapidecode ! vaapisink sync=false
$ gst-launch filesrc location=/Videos/h264/conformance/Base_Ext_Main_profiles/CVPA1_TOSHIBA_B.264 ! h264parse ! vaapidecode ! vaapisink sync=true
$ gst-launch filesrc location=/Videos/h264/conformance/Base_Ext_Main_profiles/CVPA1_TOSHIBA_B.264 ! h264parse ! vaapidecode ! vaapisink sync=false
$ gst-launch filesrc location=/Videos/h264/conformance/Base_Ext_Main_profiles/CVPA1_TOSHIBA_B.264 ! h264parse ! vaapidecode ! vaapisink sync=true

They all generate 90 frames. So, please counter-check your installation or be more precise in describing the problem.

What's the output of the following command?
$ GST_DEBUG=vaapi:3,vaapisink:5 gst-launch --gst-debug-no-color filesrc location=/Videos/h264/conformance/Base_Ext_Main_profiles/CVPA1_TOSHIBA_B.264 ! h264parse ! vaapidecode ! vaapisink sync=false

Created attachment 73548 [details]
log for CVPA1_TOSHIBA_B.264 GST_DEBUG=vaapi:3,vaapisink:5

Hi,

(In reply to comment #14)
> Created attachment 73548 [details]
> log for CVPA1_TOSHIBA_B.264 GST_DEBUG=vaapi:3,vaapisink:5

Your log file shows that 90 frames were displayed, out of the expected 90. So, I am still a little curious about the issue you see. :) Thanks.

So i thought this policy was also used on master, e.g.  video has 50 frames, it should decode 50 frames, and sink 51 frames. i found that sink 50 frames on master, it seemed it was different from QA branch, in QA branch, the first frame is sink twice.

For HCHP2_HHI_A.264 
QA branch test result:
file name Y SSIM extreme Y SSIM average	U SSIM extreme U SSIM average V SSIM extreme	V SSIM average frame-no dumped-frame lost-frames
HCHP2_HHI_A.264 1.000000 1.000000 1.000000 1.000000 1.000000 1.000000 250 251 0

master test results:
HCHP2_HHI_A.264 0.999154 0.999997 0.999925 1.000000 0.999987 1.000000 250 250 0

QA branch sink 1 more frame than master branch.

QA branch:
HPCV_BRCM_A.264 1.000000 1.000000 1.000000 1.000000 1.000000 1.000000 300 301 0
master branch:
HPCV_BRCM_A.264 0.971106 0.999904 0.994386 0.999981 0.993108 0.999977 300 300 0

on master branch, all the following files sink 1 more frame than QA branch.
SVA_CL1_E.264
SVA_NL1_B.264
CVWP1_TOSHIBA_E.264
MR5_TANDBERG_C.264
BANM_MW_D.264
HCHP2_HHI_A.264
MR2_TANDBERG_E.264
HPCV_BRCM_A.264
Freh1_B.264
freh5.264
SVA_NL2_E.264
CABA2_SVA_B.264
FREXT02_JVC_C.264
HCAMFF1_HHI.264
CAMA3_Sand_E.264
HCHP1_HHI_B.264
CANL1_SVA_B.264
CANL3_SVA_B.264
freh9.264
HCAFR3_HHI.264
CANL2_SVA_B.264
BA1_FT_C.264
FREXT01_JVC_D.264
CAWP1_TOSHIBA_E.264
LS_SVA_D.264
CANL4_SVA_B.264
CI_MW_D.264
SL1_SVA_B.264
CAPCM1_Sand_E.264
HCBP2_HHI_A.264
CAPCMNL1_Sand_E.264
CVWP2_TOSHIBA_E.264
Freh12_B.264
CABA3_SVA_B.264
HPCAQ2LQ_BRCM_B.264
SVA_BA2_D.264

Hi,

(In reply to comment #17)
> For HCHP2_HHI_A.264 
> QA branch test result:
> file name Y SSIM extreme Y SSIM average	U SSIM extreme U SSIM average V SSIM
> extreme	V SSIM average frame-no dumped-frame lost-frames
> HCHP2_HHI_A.264 1.000000 1.000000 1.000000 1.000000 1.000000 1.000000 250
> 251 0

Please tell me exactly how I can produce the same result. e.g. tool, command-line, etc.

> master test results:
> HCHP2_HHI_A.264 0.999154 0.999997 0.999925 1.000000 0.999987 1.000000 250
> 250 0
> 
> QA branch sink 1 more frame than master branch.

sink is not the determinant factor if the framerate remains below 60 Hz.

(In reply to comment #18)
> QA branch:
> HPCV_BRCM_A.264 1.000000 1.000000 1.000000 1.000000 1.000000 1.000000 300
> 301 0
> master branch:
> HPCV_BRCM_A.264 0.971106 0.999904 0.994386 0.999981 0.993108 0.999977 300
> 300 0

Please open a separate report for this one too (same as comment #17). This has nothing to do with lost frame. Looks like an issue with weighting factors. Probably another use of un-initialized variable.

(In reply to comment #17)
> For HCHP2_HHI_A.264 
> QA branch test result:
> file name Y SSIM extreme Y SSIM average	U SSIM extreme U SSIM average V SSIM
> extreme	V SSIM average frame-no dumped-frame lost-frames
> HCHP2_HHI_A.264 1.000000 1.000000 1.000000 1.000000 1.000000 1.000000 250
> 251 0
> 
> master test results:
> HCHP2_HHI_A.264 0.999154 0.999997 0.999925 1.000000 0.999987 1.000000 250
> 250 0
> 
> QA branch sink 1 more frame than master branch.

This is a VA driver bug. The driver is programming the hardware with incorrect values because it is not checking that the specific feature (prediction weight tables) is actually valid. i.e. it assumes that the application always sends sensible values if even the AVC specs explicitly excludes this case.

The PVR driver (and others) has the correct derivation process in this specific case. Of course, I could workaround that at the application layer too, but this will only fix for gstreamer-vaapi and not other frameworks or applications.

The original issue (lost frames) isn't driver related, please file a new bug to track the low SSIM issue.

> 
> This is a VA driver bug. The driver is programming the hardware with
> incorrect values because it is not checking that the specific feature
> (prediction weight tables) is actually valid. i.e. it assumes that the
> application always sends sensible values if even the AVC specs explicitly
> excludes this case.
> 
> The PVR driver (and others) has the correct derivation process in this
> specific case. Of course, I could workaround that at the application layer
> too, but this will only fix for gstreamer-vaapi and not other frameworks or
> applications.

Do you mean the following workaround in gstreamer-vaapi ?

======
commit ddaaa4202029ddbc04f01302e366f10a35f87087
Author: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
Date:   Fri Feb 8 11:56:54 2013 +0100

    h264: set {luma,chroma}_log2_weight_denom to 0 if no pred_weight_table().
======

There is no difference to the decoded frame after applying the above patch.  Actually the HW will ignore  luma_log2_weight_denom/chroma_log2_wieght_denom if no pred_weight_table().  Yes it would be better to check  luma_log2_weight_denom/chroma_log2_wieght_denom in the GEN driver.

(In reply to comment #24)
> There is no difference to the decoded frame after applying the above patch. 
> Actually the HW will ignore  luma_log2_weight_denom/chroma_log2_wieght_denom
> if no pred_weight_table().  Yes it would be better to check 
> luma_log2_weight_denom/chroma_log2_wieght_denom in the GEN driver.

If there is no difference in the decoded frame then the problem is probably elsewhere in the driver. At least, there is now no difference in the generated commands/slice & pic param structs (at the gstreamer-vaapi level) between 0.4-branch and master for the mentioned clip.

Yes, the HW normally should not care. Are we sure about it?

Let's wait for QA for testing. I honestly did not see (visually) any difference before and after the patch. Just that the generated structs are now identical.

This bug report is becoming too large and spans other multiple issues. It would be a good thing to trim/sort this down.

we have filed the bug about low ssim low value.
pls. refer to #bug 60099 [SNB IVB HSW] SSIM value is low with some AVC stream

QA branch:
HPCV_BRCM_A.264 1.000000 1.000000 1.000000 1.000000 1.000000 1.000000 300 301 0
master branch:
HPCV_BRCM_A.264 0.971106 0.999904 0.994386 0.999981 0.993108 0.999977 300 300 0

restest on master branch ,we found that the first frame sink twice as QA branch, but it seems that master branch lost the last frame. we only dump 300 frames (referece is 300 frames), if the first frame sink twice, we can dump 301 frames.


Environment:
--------------
Platform: IVB
Libva master:
commit 9f4dedc4de014cc665c32dfbac1c017f9396b563
Intel-driver master:
commit 7803fcae62c868c26f24425df04aee0405342563
Gstreamer 0.1 branch:
commit 935db833e236959c973e6c8ef59586d30f81a26c
gst-plugins-base 0.1 branch:
commit 95c384136a1dfc22ad656c13b547c4077566d80b
gst-plugins-good 0.1 branch:
commit 967fc61715fa854dacaa200069437fe57d4d8390
gst-plugins-ugly 0.1 branch:
commit 981912f29e54f6cacefe62ee2a24c38a227d0565
gst-plugins-bad 0.1 branch:
commit d21edfecd149886c88a35a92803a0bcfabdc7630
Gst-vaapi master branch:
commit ddaaa4202029ddbc04f01302e366f10a35f87087

I checked the HPCV_BRCM_A.h264 against master, 0.4-branch and QA branch and found:

a) The last frame in decoding order is lost when using the master branch.

b) The decoded frames and the referenece YUV picutres  have the same data  if ignoring the lost frame.

c)  display order is used in the test script.  I guess  they don't know which frame is lost  in display order, so the lost frame will impact the result.

I just tried 0.4-branch and qa branch, both works well. no frame lost.
I didn't try master branch. not sure about this. Assign to Gwenole since he works on master branch now, or maybe it's not a bug. maybe QA's test script issue.
The frame nums of this file:
Output Pictures: 90,
field and frame count: 138
If the test nums are same in master branch, it should not be a bug.