Bug 57923

Created attachment 71045 [details]
Log on Linux with h264parse version compiled from gst-plugins-good-0.10.31

h264parse is in gst-plugins-bad. The GStreamer SDK has lots of bugfixes and changes from the latest 0.10 GIT included.

h264parse->split_packetized is always set when converting stream-format=avc to another stream-format. Could you check which caps are on both pads of h264parse on Linux and Windows/OSX? The OSX log unfortunately does not contain the caps due to limitations on Windows/OSX.

I'm sorry for a mistake, I meant gst-plugins-bad-0.10.23 source package (was busy debugging also osxvideosink quite simultenaously so things got a little bit messed up, sorry). Unfortunately I didn't managed to work with the program in Windows today to get the exact caps on h264parse input and output, hope to get them tomorrow. I also noticed in fresh logs I attached above that the differences in behaviour start in gst_h264_parse_negotiate() around line 350. My version was that somehow SDK version goes to 'then' branch of gst_caps_can_intersect() and skips gst_pad_fixate_caps(). I couldn't test my version because, as I said, compiled from sources it start working as is. Hope to experiment on it some more.

Well, both input and output caps looks so similar on SDK and compiled sources version. Here's an SDK one:

h264parse caps:
	sink:
		allowed: EMPTY
		actual: video/x-h264, codec_data=(buffer)0142c01fffe100166742c01fda014016e840000003004000000ca3c60ca801000468ce0bc8, width=(int)1280, height=(int)720, framerate=(fraction)25/1
	src:
		allowed: video/x-h264, stream-format=(string)avc, codec_data=(buffer)0142c01fffe100166742c01fda014016e840000003004000000ca3c60ca801000468ce0bc8, parsed=(boolean)true, width=(int)1280, height=(int)720, framerate=(fraction)25/1, alignment=(string)au
		actual: video/x-h264, stream-format=(string)avc, codec_data=(buffer)0142c01fffe100166742c01fda014016e840000003004000000ca3c60ca801000468ce0bc8, parsed=(boolean)true, width=(int)1280, height=(int)720, framerate=(fraction)25/1, alignment=(string)au

Here's sources one:

h264parse caps:
	sink:
		allowed: EMPTY
		actual: video/x-h264, codec_data=(buffer)0142c01fffe100166742c01fda014016e840000003004000000ca3c60ca801000468ce0bc8, width=(int)1280, height=(int)720, framerate=(fraction)25/1
	src:
		allowed: video/x-h264, stream-format=(string)avc, codec_data=(buffer)0142c01fffe100166742c01fda014016e840000003004000000ca3c60ca801000468ce0bc8, parsed=(boolean)true, width=(int)1280, height=(int)720, framerate=(fraction)25/1, alignment=(string)au
		actual: video/x-h264, stream-format=(string)avc, codec_data=(buffer)0142c01fffe100166742c01fda014016e840000003004000000ca3c60ca801000468ce0bc8, parsed=(boolean)true, width=(int)1280, height=(int)720, framerate=(fraction)25/1, alignment=(string)au

The only difference I noticed is that there's a decodebin in the pipeline and before its 'pad-added' signal caps on h264parse on sources version is these:

h264parse caps:
	sink:
		allowed: video/x-h264, width=(int)[ 16, 4096 ], height=(int)[ 16, 4096 ], framerate=(fraction)[ 0/1, 2147483647/1 ], stream-format=(string)avc, codec_data=(buffer)0142001fffe100166742c01fda014016e840000003004000000ca3c60ca801000468ce0bc8, parsed=(boolean)false; video/x-h264, stream-format=(string)avc, codec_data=(buffer)0142001fffe100166742c01fda014016e840000003004000000ca3c60ca801000468ce0bc8, parsed=(boolean)false
		actual: video/x-h264, width=(int)[ 16, 4096 ], height=(int)[ 16, 4096 ], framerate=(fraction)[ 0/1, 2147483647/1 ]; video/x-h264
	src:
		allowed: video/x-h264, parsed=(boolean)true, stream-format=(string){ avc, byte-stream }, alignment=(string){ au, nal }, width=(int)[ 16, 4096 ], height=(int)[ 16, 4096 ], framerate=(fraction)[ 0/1, 2147483647/1 ]; video/x-h264, parsed=(boolean)true, stream-format=(string){ avc, byte-stream }, alignment=(string){ au, nal }
		actual: video/x-h264, parsed=(boolean)true, stream-format=(string){ avc, byte-stream }, alignment=(string){ au, nal }

But on SDK version they are just the same as after this signal: 

h264parse caps:
	sink:
		allowed: EMPTY
		actual: video/x-h264, codec_data=(buffer)0142c01fffe100166742c01fda014016e840000003004000000ca3c60ca801000468ce0bc8, width=(int)1280, height=(int)720, framerate=(fraction)25/1
	src:
		allowed: video/x-h264, stream-format=(string)avc, codec_data=(buffer)0142c01fffe100166742c01fda014016e840000003004000000ca3c60ca801000468ce0bc8, parsed=(boolean)true, width=(int)1280, height=(int)720, framerate=(fraction)25/1, alignment=(string)au
		actual: video/x-h264, stream-format=(string)avc, codec_data=(buffer)0142c01fffe100166742c01fda014016e840000003004000000ca3c60ca801000468ce0bc8, parsed=(boolean)true, width=(int)1280, height=(int)720, framerate=(fraction)25/1, alignment=(string)au

Seems that it's really makes sense for processing the first frame.

These different caps before decodebin exposes the pad (i.e. during the autoplugging) happens because the SDK decodebin (and the one in latest 0.10 GIT) has a more clever logic to select the ideal caps after a parser/convert element (like h264parse).

But if in the end the caps are all the same, I don't see why there should be any difference in behaviour there. Also split_packetized should be FALSE here in all cases if I'm not mistaken, it's always passing through stream-format=avc/alignment=au.

Any ideas what the difference there could be? I can't reproduce this here unfortunately and this really shouldn't be a platform specific difference.

Thanks for description, Sebastian.

> Any ideas what the difference there could be? I can't reproduce this here
> unfortunately and this really shouldn't be a platform specific difference.

 Well, not yet, I just quickly compared 0.10-upstream version of h264parse (in gst/videoparsers) with gst-plugins-bad-0.10.23' one and noticed no big differences in affected places. So no idea yet, but I've got working compiled sources version on both problematic platforms (Windows and Mac) so I hope I will play with it tomorrow.

The news is that I've recompiled the plugin in Windows and the behaviour is just the same as on Mac: compiled version works flawlessly (only libgstvideoparsersbad.dll was replaced).

I should add that that's all x86_64 versions.

I seem to manage to sort out some things about this problem yesterday:

1. Exactly, as it's been assumed, upstream version leads to faulty video output (error in first keyframe) and 0.10.23 doesn't.

2. The key difference is not about all these things, it's somewhere near gst/videoparsers/gsth264parse.c:1625:
0.10.23 version:
    if (format == GST_H264_PARSE_FORMAT_NONE) {
      format = GST_H264_PARSE_FORMAT_AVC;
      align = GST_H264_PARSE_ALIGN_AU;
    }

upstream version:
    if (format == GST_H264_PARSE_FORMAT_NONE)
      format = GST_H264_PARSE_FORMAT_AVC;
    if (align == GST_H264_PARSE_ALIGN_NONE)
      align = GST_H264_PARSE_ALIGN_AU;

So, the trick is that the former one sets align to GST_H264_PARSE_ALIGN_AU _only_ if format is GST_H264_PARSE_FORMAT_NONE and the latter one sets it _even_ if format is something else. The crucial occasion is that the format for my case is GST_H264_PARSE_FORMAT_AVC, so with the former snip align remains initialized to GST_H264_PARSE_ALIGN_NONE, which, again truly by occasion, saves the game: the condition (format == h264parse->format && align == h264parse->align) in line around 1658 fails and split_packetized by wonder is set to TRUE.

3. Somewhere further split_packetized is checked (l. 1858) and if it's true an additional buffer with nalu data is pushed out-of-order and it really helps ffdec_h264 to properly parse the stream.

I'm sure, it's clearly bug that was fixed, but exclusively for my stream it spoils the situation. So, I've got two questions:

1. Given that upstream code is correctly processes AVC stream, what is the exact fault in my AVC stream? I'm actually not responsible for the stream, I receive it from server, so I need some detailed arguments to ask server guys about their stream. I should add here that dumping this stream to libavcodec routines straight provides correct video output (i.e. it gives no errors on the first keyframe). I'm sure about that as the previous version of my application worked exactly that way.

2. Wouldn't it be more wise to push such a buffer with nalu at least for compartibility reasons? If not, I think this moment worth being well-documented, I can help with it if we clear it up completely. There is some comments:

        /* pass-through: no looking for frames (and nal processing),
         * so need to parse to collect data here */
        /* NOTE: so if it is really configured to do so,
         * pre_push can/will still insert codec-data at intervals,
         * which is not really pure pass-through, but anyway ... */

but I didn't manage to understand what it means.