(Can't seem to edit my earlier comment, so I'll continue here)
It seems like there's something strange going on with this GL extension. Someone on the Kodi forum looked into this for me and found out that the problem was that it's only partially supported. I'm not sure how this can or should be fixed. 

If this is a hardware limitation, perhaps there should be some way to find out if at least the one-way part of the extension is supported (from userspace). 

If it's only partially implemented in the radeon driver, perhaps this needs to be completed. Or maybe the part that's implemented is good enough to enable the extension, I don't know. 

But it seemed like it was worth mentioning to the appropriate people. There is also mention of a 200ms (or 175ms to be exact) delay/latency for the video processing (which means the video is lagging 200ms behind the audio) in the Kodi thread. It's possible that it is not related to this, but I wanted to mention it anyway (in case it is related).

From Kodi POV we like to have a best practice to cope with such devices. For now we check for the extension and if found use the methods it defines.

As those cards don't expose this extentions we exit early. If there is something we can do to make kodi work reliable on that hardware without any crude hacks - like asking for extension and use it even if not there at all - we are happy to implement such methods.

The best would be to implement the missing feature by some Shader / whatever solution - transparent to use and enable the extension completely again.

You can't really enable a GL extension if there is not support for the entire thing.  Early versions of UVD did not support field based output.  The extension provides access to both the raw decoded video surfaces (field based) and the vdpau processed output surface (frame based).  I think by default kodi only uses the vdapu processed output which is the part that is supported.  Changing the kodi settings may require the full extension which will not work.

I guess there are 3 options:

1. Force the option but limit it to cases where you are only using the output surface via the interop extension.  This works today.
2. Add an extra step to the drivers using shaders to split the frame up into separate fields in vlVdpVideoSurfaceGallium (src/gallium/state_trackers/vdpau/surface.c) for chips that don't support field based decode.  This adds an extra copy and someone needs to write the code.  With that done, the extension can be enabled on all asics.
3.  Write a new simplified GL/VDPAU interop extension that only supports interop with output surfaces.  This extension could be enabled on all asics.

I understand that you can't enable a GL extension if there's no support for it in the hardware, but I think this specific GL extension doesn't require anything specific in the hardware. 

From what I understand, the GL_NV_vdpau_interop extension needs to be supported in the driver, not in the hardware. In this Phoronix article (http://www.phoronix.com/scan.php?page=news_item&px=MTQ2NjY) it is mentioned that NVIDIA implemented support for this in their binary driver (not their hardware). So doesn't this mean that supporting this extension fully can be done by improving/expanding the implementation in the driver?

As for the field and frame issue, I don't really know much about that. But I do know that there's UVD2 and UVD2.2, which is often also indicated as UVD2. Is it possible that the "Early versions of UVD" that "did not support field based output" could be the UVD2 ones and the newer ones that work correctly are UVD2.2? Because being able to make that distinction could make it easier to figure out if this extension will likely work or not. 

I assume that Kodi will not work with the UVD2, but will work with UVD2.2. If UVD2.2 guarantees that VDPAU can be used, then that might be a good solution too.

(In reply to Alex Deucher from comment #3)

> 
> I guess there are 3 options:
> 
> 1. Force the option but limit it to cases where you are only using the
> output surface via the interop extension.  This works today.
> 2. Add an extra step to the drivers using shaders to split the frame up into
> separate fields in vlVdpVideoSurfaceGallium
> (src/gallium/state_trackers/vdpau/surface.c) for chips that don't support
> field based decode.  This adds an extra copy and someone needs to write the
> code.  With that done, the extension can be enabled on all asics.
> 3.  Write a new simplified GL/VDPAU interop extension that only supports
> interop with output surfaces.  This extension could be enabled on all asics.

I think option 1. is good enough. This is done by setting the environment variable, right? We can force using the mixer if this variable is set.

Option 1 is a good workaround, but it needs to be done manually. Option 2 is of course the easiest for userspace programs since they would not have to change anything, but it might not be practical. I think for Kodi it would be preferred to have a way to programmatically find out if this partial interop functionality is available. This would be a fourth option.

For example, if someone can confirm that the interop functionality is not available in UVD1 and UVD2, but partial interop is available in UVD2.2 and full interop is available in UVD3 and higer, then perhaps the solution for programs like Kodi would be to check the extension as well as the UVD version. Or if someone can find out if there's some other way to distinguish between hardware without interop support and with partial interop support (since both of these will have the interop extension disabled).

(In reply to riaasm from comment #4)
> I understand that you can't enable a GL extension if there's no support for
> it in the hardware, but I think this specific GL extension doesn't require
> anything specific in the hardware. 

It does require certain hardware features, namely decoding to separate fields.  That's why we only enable it on UVD 2.2 and newer.  Older UVD hardware only supports decoding to a full frame.

> 
> From what I understand, the GL_NV_vdpau_interop extension needs to be
> supported in the driver, not in the hardware. In this Phoronix article
> (http://www.phoronix.com/scan.php?page=news_item&px=MTQ2NjY) it is mentioned
> that NVIDIA implemented support for this in their binary driver (not their
> hardware). So doesn't this mean that supporting this extension fully can be
> done by improving/expanding the implementation in the driver?

The hardware needs to support certain features to allow you to implement a software extension.  E.g., consider tessellation.  If your GPU does not support tessellation, it does not make sense to expose the GL extension since the implementation would be done in software which is slow.  In the case of this extension, we could implement support for older UVD versions, but it involves an extra copies.

> 
> As for the field and frame issue, I don't really know much about that. But I
> do know that there's UVD2 and UVD2.2, which is often also indicated as UVD2.
> Is it possible that the "Early versions of UVD" that "did not support field
> based output" could be the UVD2 ones and the newer ones that work correctly
> are UVD2.2? Because being able to make that distinction could make it easier
> to figure out if this extension will likely work or not. 
> 

We know which parts support decoding to fields.  UVD 2.2 and newer.

UVD 1.0-2.1 (R6xx, RS780, RS880, RV770, RV790) - don't support decoding to fields
UVD 2.2 and newer (RV710, RV730, RV740, evergreen, NI, SI, CI, VI) - support decoding to fields

> I assume that Kodi will not work with the UVD2, but will work with UVD2.2.
> If UVD2.2 guarantees that VDPAU can be used, then that might be a good
> solution too.

We only expose the extension on asics that support it (UVD 2.2 and newer).  Kodi already supports that by checking if the extension is enabled.  The issue is what to do about older UVD versions that do not support the entire extension.  All UVD versions support the part of the extension that kodi uses with it's current default settings.

(In reply to Rainer Hochecker from comment #5)
> I think option 1. is good enough. This is done by setting the environment
> variable, right? We can force using the mixer if this variable is set.

Correct.

(In reply to riaasm from comment #6)
> For example, if someone can confirm that the interop functionality is not
> available in UVD1 and UVD2, but partial interop is available in UVD2.2 and
> full interop is available in UVD3 and higer, then perhaps the solution for
> programs like Kodi would be to check the extension as well as the UVD
> version. Or if someone can find out if there's some other way to distinguish
> between hardware without interop support and with partial interop support
> (since both of these will have the interop extension disabled).

We already know this.  We only expose the extension on asics that support the full functionality.  There are only two cases.  Asics that support the extension fully and asics that support the partial functionality.  UVD 2.2 and newer support the full functionality.  All asics support the partial functionality.

So if I understand this correctly, all versions of UVD can perform frame-based interop. I assume that the UVD hardware was created to perform frame-based interop, but after the GL_NV_vdpau_interop extension was proposed/suggested/completed by Nvidia the UVD hardware was updated to also allow field-based interop. The version in which this update was done is UVD 2.2. 

This explains why the UVD hardware only partially supports the GL extension, since it wasn't aware of the extension at the time the UVD hardware was created. Is this correct?

If so, userspace apps that only need frame-based interop can just check if UVD is available since it should work on any UVD hardware. And userspace apps that need field-based interop can check for the GL extension. Would this be a reasonable and correct way to check for interop functionality

(In reply to riaasm from comment #10)
> If so, userspace apps that only need frame-based interop can just check if
> UVD is available since it should work on any UVD hardware. And userspace
> apps that need field-based interop can check for the GL extension. Would
> this be a reasonable and correct way to check for interop functionality

That's what Alex suggest with his option number 3. E.g. write a new extension that allows application to query if a subset of GL_NV_vdpau_interop is supported.

But since this only applies to rather old hardware nobody is probably going to do so.

Option number 2 is actually rather easily doable as well, cause we already have most shaders for this available. Maybe fifty lines of code after all.

But again, rather old hardware and so nobody of the devs interested in fixing this.

If you are interested in coding this I can easily give you the hints where to look at.

I think option 3 would be the most suitable solution, since this would advertise exactly what the hardware does. I'm not sure if this is practical though, since it seems to me like creating a new GL extension would be a lot of administrative work. But in all fairness, I have no idea what's involved in this.

I'm not sure it would be wise to emulate some hardware functionality in software (option 2), mostly because I don't see the need for "faking it". Just let the hardware say what it can and can not do, so the higher-level application can use it appropriately. Also, I have no idea how to do this anyway, so I can't really help with that.

I also wanted to mention that the interop specs (https://www.opengl.org/registry/specs/NV/vdpau_interop.txt) seem to indicate that the GL_NV_vdpau_interop extension is actually supposed to be field based, not frame-based. Maybe I'm not reading it correctly or maybe the frame-based solution was added to the implementation. Either way, if the GL extension is really supposed to be field-based, it might even be useful for newer hardware (as well as older hardware) to have a separate GL extension for the frame-based solution (maybe GL_NV_vdpau_interop_frame, with the current one being for field interop only). This way it would match more closely to the specification of each GL extension. For most newer hardware, this means that both GL extensions would be enabled and it would be clear that the hardware supports both field and frame interop.

If I was to decide, I would suggest:

>2. Add an extra step to the drivers using shaders to split the frame up into >separate fields in vlVdpVideoSurfaceGallium
>(src/gallium/state_trackers/vdpau/surface.c) for chips that don't support field >based decode.  This adds an extra copy and someone needs to write the code.  With >that done, the extension can be enabled on all asics.

Reason:
- No application needs to be changed and all methods would be supported by this hardware, too. 
- Our workarounds + extra code to handle the env would also easily be 50+ lines :-) and that per application (kodi, mpv, ...)

So @Christian happy for pointers on how to get it going.

(In reply to Peter Frühberger from comment #13)
> So @Christian happy for pointers on how to get it going.

Yeah, completely agree with you. Software workarounds are very common in OpenGL and nothing unusual.

Here are a few pointers into the code:
1. vlVdpVideoSurfaceGallium() in src/gallium/state_trackers/vdpau/surface.c. That one is called to "export" video surfaces to OpenGL and the right place to convert from frame to field.

2. vlVdpDecoderRender() in src/gallium/state_trackers/vdpau/decode.c. Here we reallocate the buffer backing the surface before the decoding if the frame/field flag doesn't match what the decoder expects.

3. vlVdpVideoMixerRender() in src/gallium/state_trackers/vdpau/mixer.c. Here we use a vl_compositor_render() to compose the mixed output picture.

Essentially you only need to test if the surface is field based and if not reallocate it and either use vl_compositor_render() or pipe->blit() to copy the frame into fields.

-- GitLab Migration Automatic Message --

This bug has been migrated to freedesktop.org's GitLab instance and has been closed from further activity.

You can subscribe and participate further through the new bug through this link to our GitLab instance: https://gitlab.freedesktop.org/mesa/mesa/issues/556.