Bug 106625

Summary: AMD HEVC support for for libomxil-bellagio / gst-plugins-omx
Product: Mesa Reporter: Luke McKee <hojuruku>
Component: Drivers/Gallium/radeonsiAssignee: Default DRI bug account <dri-devel>
Status: CLOSED FIXED QA Contact: Default DRI bug account <dri-devel>
Severity: enhancement    
Priority: medium CC: daniel
Version: git   
Hardware: x86-64 (AMD64)   
OS: All   
Whiteboard:
i915 platform: i915 features:

Description Luke McKee 2018-05-23 07:53:18 UTC
The gallium driver has a h265 decode module for libomxil-bellagio, but not for https://github.com/tizonia/tizonia-openmax-il implementation.

It's in /gallium/state_trackers/omx
https://github.com/freedesktop/mesa/blob/master/src/gallium/state_trackers/omx/bellagio/vid_dec_h265.c

tizonia includes EGL support so maybe it would be better to add the H265 encode/decode support to that because libomxil-beligago hasn't been updated upstream since 2012.


Now I've checked with the official AMD Radeon (amdgpu) drivers 18.10 and their bundled /etc/xdg/gstomx.conf doesn't have the hevc video decoder enabled by default (through the configuration file), though I haven't checked yet if it's enabled in the code to make this a proper "GPL" source code request.

The patches are probably already floating AMD, OMX HEVC support has found it's way into mesa mesa but to the libraries that use it.

Can one use omx HEVC decode in the mesa state tracker without AMD's patch to libomxil-bellago? Can you please release it?

[omxh264dec]
type-name=GstOMXH264Dec
core-name=/usr/lib64/libomxil-bellagio.so.0
component-name=OMX.mesa.video_decoder.avc
rank=258
in-port-index=0
out-port-index=1

[omxmpeg2dec]
type-name=GstOMXMPEG2VideoDec
core-name=/usr/lib64/libomxil-bellagio.so.0
component-name=OMX.mesa.video_decoder.mpeg2
rank=258
in-port-index=0
out-port-index=1

#[omxh265dec]
#type-name=GstOMXH265Dec
#core-name=/usr/lib64/libomxil-bellagio.so.0
#component-name=OMX.mesa.video_decoder.hevc
#in-port-index=0
#out-port-index=1
#rank=258

[omxh264enc]
type-name=GstOMXH264Enc
core-name=/usr/lib64/libomxil-bellagio.so.0
component-name=OMX.mesa.video_encoder.avc
in-port-index=0
out-port-index=1

The hevc decode support added a 6 months ago (1.14.0) to gst-plugins-omx is for a different openmax implementation:

https://github.com/GStreamer/gst-omx/commit/5751001ddc0ea099d6cdbe817a6882959991a3c9

"Add HEVC decoder for the zynqultrascaleplus platform.

I used the H264 decoder code as a template.

https://bugzilla.gnome.org/show_bug.cgi?id=785434"

I know you probably implemented it differently but these are some of the things missing from OMX_Video.h

typedef enum OMX_VIDEO_CODINGTYPE {
    OMX_VIDEO_CodingUnused,     /**< Value when coding is N/A */
    OMX_VIDEO_CodingAutoDetect, /**< Autodetection of coding type */
    OMX_VIDEO_CodingMPEG2,      /**< AKA: H.262 */
    OMX_VIDEO_CodingH263,       /**< H.263 */
    OMX_VIDEO_CodingMPEG4,      /**< MPEG-4 */
    OMX_VIDEO_CodingWMV,        /**< all versions of Windows Media Video */
    OMX_VIDEO_CodingRV,         /**< all versions of Real Video */
    OMX_VIDEO_CodingAVC,        /**< H.264/AVC */
    OMX_VIDEO_CodingMJPEG,      /**< Motion JPEG */
    OMX_VIDEO_CodingHEVC,
    OMX_VIDEO_CodingKhronosExtensions = 0x6F000000, /**< Reserved region for introducing Khronos Standard Extensions */
    OMX_VIDEO_CodingVendorStartUnused = 0x7F000000, /**< Reserved region for introducing Vendor Extensions */
    OMX_VIDEO_CodingMax = 0x7FFFFFFF
} OMX_VIDEO_CODINGTYPE;




gstomxh265utils.h:29:1: error: unknown type name ‘OMX_VIDEO_HEVCPROFILETYPE’; did you mean ‘OMX_VIDEO_AVCPROFILETYPE’?
 OMX_VIDEO_HEVCPROFILETYPE gst_omx_h265_utils_get_profile_from_str (const
 ^~~~~~~~~~~~~~~~~~~~~~~~~
 OMX_VIDEO_AVCPROFILETYPE
gstomxh265utils.h:31:1: error: unknown type name ‘OMX_VIDEO_HEVCLEVELTYPE’; did you mean ‘OMX_VIDEO_AVCLEVELTYPE’?
 OMX_VIDEO_HEVCLEVELTYPE gst_omx_h265_utils_get_level_from_str (const gchar *
 ^~~~~~~~~~~~~~~~~~~~~~~
 OMX_VIDEO_AVCLEVELTYPE
gstomxh265utils.h:34:57: error: unknown type name ‘OMX_VIDEO_HEVCPROFILETYPE’; did you mean ‘OMX_VIDEO_AVCPROFILETYPE’?
 const gchar * gst_omx_h265_utils_get_profile_from_enum (OMX_VIDEO_HEVCPROFILETYPE e);
                                                         ^~~~~~~~~~~~~~~~~~~~~~~~~
                                                         OMX_VIDEO_AVCPROFILETYPE
gstomxh265dec.c: In function ‘set_profile_and_level’:
gstomxh265dec.c:138:25: error: ‘OMX_VIDEO_HEVCProfileUnknown’ undeclared (first use in this function); did you mean ‘OMX_VIDEO_AVCProfileMain’?
   if (param.eProfile == OMX_VIDEO_HEVCProfileUnknown)
                         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~
                         OMX_VIDEO_AVCProfileMain
gstomxh265dec.c:138:25: note: each undeclared identifier is reported only once for each function it appears in
gstomxh265dec.c:148:23: error: ‘OMX_VIDEO_HEVCLevelUnknown’ undeclared (first use in this function); did you mean ‘OMX_VIDEO_HEVCProfileUnknown’?
   if (param.eLevel == OMX_VIDEO_HEVCLevelUnknown)
                       ^~~~~~~~~~~~~~~~~~~~~~~~~~
                       OMX_VIDEO_HEVCProfileUnknown
Comment 1 Daniel Stone 2018-05-23 09:51:07 UTC
Bellagio (and Mesa generally) are not licensed under the GPL, but under the MIT/BSD family of licenses. These licenses have no obligation at all for source release: everyone is free to form the code, ship binaries, and never provide source.

Additionally, the GPL's requirements on producing source are very specific: you are only entitled to receive the source to GPL-licensed code, if you have also received binaries. It is totally fine to fork a GPL project and never release code to anyone, if you don't distribute binaries. It is also totally fine to distribute the binaries to a limited audience, only give the source to them if they ask for it, and never give the source to anyone else. There is no kind of requirement for all source to always be universally available.

So for this to be enforceable under the GPL: AMD would need to have definitely written the code (it is not clear if this has even happened), and distributed binaries built from it (it seems clear this has not happened), and the code would need to be under (L)GPL (it isn't).

I'm afraid this isn't an actionable legal request on those grounds, but good luck with your quest for HEVC support.
Comment 2 Luke McKee 2018-05-23 11:58:55 UTC
I'm hoping AMD will get around to releasing their changes to LGPL licensed gstreamer-plugins-omx / libomxil-bellago

It's mesa/Gallium that has the BSD/MIT license and all the code for libomix-belagio h265 support is already in the MESA source referred to in bug description.

https://www.mesa3d.org/license.html

To say gstreamer-plugins-omx and libomixl-bellagio is not gpl like you said on IRC would be incorrect.

https://github.com/GStreamer/gst-omx/blob/master/COPYING

https://sourceforge.net/p/omxil/omxil/ci/master/tree/COPYING

Now if I was stickler for making this a GPL request I'd be looking at the bundled gst-plugins-omx binary in to see if it has h265 support (though not enabled by default). But it makes you wonder why they package it?

https://support.amd.com/en-us/kb-articles/Pages/Radeon-Software-for-Linux-Release-Notes.aspx

Seeing AMD is moving to tizonia in the last 3 months, that too lacks h265 support, so it has to be a custom interface to use h265.

So consider this a GPL request for the source for AMD's build of gstreamer-plugins-omx for the binaries the release. Finger's crossed the source shows their patches :)

https://www1.qt.io/qt-licensing-terms/

"Complete corresponding source code of the library used with the application or the device built using LGPL, including all modifications to the library, should be delivered with the application (or alternatively provide a written offer with instructions on how to get the source code). It should be noted that the complete corresponding source code has to be delivered even if the library has not been modified at all."

Sorry Daniel but you are just plain wrong.
Comment 3 Christian König 2018-05-23 12:10:25 UTC
The H265 support in the OpenMAX state tracker is a non-standard feature you can enable by forcing applications to use the specified component for video decoding.

OpenMAX just takes a binary stream as input and a picture stream as output, so that works for any video codec as long as the application can handle it.

I can ask if we could release the test application we use for development, but as far as I know we never created any code for GStreamer, libomxil-bellagio or tizonia.
Comment 4 Luke McKee 2018-05-23 12:20:21 UTC
Thanks Christian.

I tried to use this to prod AMD developers as it was the only way in, bypassing legal.

Releasing that test application would be great then it could given to the devs of gst-plugins-openmax to get this functionality used by people who run Mesa outside of AMD's HQ'.

Also can you please provide the tarball used to build gst-plugins-openmax that you bundle with your released driver, or just confirm it's vanilla source / patch free, as you are not bundling debian / ubuntu's version for some reason.
Comment 5 Luke McKee 2018-05-23 12:22:26 UTC
Re-opened pending response from Christian König to release some kind of h265 openmax bellagio test-app, or a portion of it there-of as an example to document the API for use by other opensource devs.
Comment 6 Daniel Stone 2018-05-23 12:27:53 UTC
(In reply to Luke McKee from comment #2)
> I'm hoping AMD will get around to releasing their changes to LGPL licensed
> gstreamer-plugins-omx / libomxil-bellago
> 
> It's mesa/Gallium that has the BSD/MIT license and all the code for
> libomix-belagio h265 support is already in the MESA source referred to in
> bug description.
> 
> https://www.mesa3d.org/license.html

And from the code you linked here:
https://github.com/freedesktop/mesa/blob/master/src/gallium/state_trackers/omx/bellagio/vid_dec_h265.c

> To say gstreamer-plugins-omx and libomixl-bellagio is not gpl like you said
> on IRC would be incorrect.
> 
> https://github.com/GStreamer/gst-omx/blob/master/COPYING
> 
> https://sourceforge.net/p/omxil/omxil/ci/master/tree/COPYING

I never said that on IRC, and as far as I can tell no-one else did either. But it's correct anyway - gst-omx is LGPL and _not_ GPL. Both have equivalent terms to allow those in possession of a binary to demand corresponding source, but they are very different licenses.

> Now if I was stickler for making this a GPL request I'd be looking at the
> bundled gst-plugins-omx binary in to see if it has h265 support (though not
> enabled by default). But it makes you wonder why they package it?
> 
> https://support.amd.com/en-us/kb-articles/Pages/Radeon-Software-for-Linux-
> Release-Notes.aspx

Right, I can see now that you're talking indeed about gst-omx. I was just confused by the talk about and links to Bellagio (relevant to Mesa) at the start. This is about gst-omx which we don't develop, and isn't released by the Mesa project at all. If Mesa or freedesktop.org were releasing those binaries then this would be the right forum, but it's not: AMD is doing the distribution of, again, someone else's code.

It's important for us to be very clear; filing bugs on Mesa alleging that there have been GPL violations is pretty bad for both Mesa and freedesktop.org, and super important to have rectified and clarified immediately.

> Seeing AMD is moving to tizonia in the last 3 months, that too lacks h265
> support, so it has to be a custom interface to use h265.
> 
> So consider this a GPL request for the source for AMD's build of
> gstreamer-plugins-omx for the binaries the release. Finger's crossed the
> source shows their patches :)
> 
> [... LGPL explainer snipped ...]
>  
> Sorry Daniel but you are just plain wrong.

Yes, the AMD distribution of gst-omx does seem - on the face of it - like it would require an accompanying source offer.

But, on the other hand:
  - distribution of Bellagio is not relevant, as it is licensed under permissive terms
  - any development AMD may or may not be doing on Tizonia is totally irrelevant, as they do not appear to be distributing it currently; copyright/licensing terms only apply on distribution
  - the driver is distributed by amd.com, and not freedesktop.org
  - the work you're referring to isn't really related to the Mesa project at all

Please take this up with AMD. Again I appreciate your zeal, but please do not throw claims of GPL violations around so lightly. I hope you understand why I've had to figure out exactly what it is you're alleging and requesting here, and be very precise about it.
Comment 7 Christian König 2018-05-23 14:11:00 UTC
Asking around Leo actually patches gstomx to work with that because the HEVC support wasn't upstream yet.

The code was always public here https://cgit.freedesktop.org/~leoliu/gstomx, but I don't think that this is production quality.

On the other hand according to Leo the upstream HEVC support in gstomx should now work as well with the OpenMAX state tracker as well (but was never tested).

The internal tool can't be released cause that is all proprietary hardware and software validation code (unit tests framework).

No idea what gstomx version is packed with the pro driver, but since we don't have any internal repository I think it is just a certain upstream version.
Comment 8 Luke McKee 2018-05-23 14:43:21 UTC
Dan,

> And from the code you linked here:
https://github.com/freedesktop/mesa/blob/master/src/gallium/state_trackers/omx/bellagio/vid_dec_h265.c

Yep, to show a feature that was ALREADY OPENSOURCED BY MESA that couldn't be used without additional source which was either not upstreamed or not in a well known location. Why would I be linking to the source code for something in the original POST and then asking for the source code to be opensourced via a LGPL request????? WT#? You are not making sense. Please quit flogging the dead horse here.

The request was a LGPL source code request to AMD (yes not freedesktop) for their bundling LGPL licensed libraries libopenmax-bellagilo (not modified in AMD's case like other openmax implementations were for HEVC suport) OR the LGPL licensed gst-plugins-omx. 

Yes I am guilty as charged, I used the freedestop.org bug tracker as a shortcut to get in touch with some AMD devs by only putting the request in the AMD Gallium section, and it was mission accomplished.

However this request did relates to a feature in MESA gallium state tracker that can't easily be implemented due to no documentation or info on what libraries can use said feature, hence why it's also relevant to freedesktop.org / mesa support.


---

Christian,

Thanks for your help. The reason I was playing with openmax because of encoding problems with FFMPEG that are still not resolved, and because ffmpeg vaapi until recently didn't support EGL / play nice with mpv.

Thanks for linking to the AMD fork of libgst-openmax, I'll see how it works, if not so good, I think I'll try tizonia-openmax-il as it supports EGL and hope you'll implement gstreamer h265-encoding/decoding down the track there too. The openmax interface to the encoder previously was more stable than the ffmpeg/vaapi one.

Cheers,

Luke
Comment 9 Christian König 2018-05-24 11:09:20 UTC
Asking around further I confirmed we build exactly https://cgit.freedesktop.org/~leoliu/gstomx/commit/?id=b0a03765a183d6d4633d7e71f46797f91a3c0951 into the pro-package.

This commit id is also noted in the changelog included in the package. Closing this bug as that code was always available public.

Use of freedesktop.org services, including Bugzilla, is subject to our Code of Conduct. How we collect and use information is described in our Privacy Policy.