Created attachment 125615 [details]
only printk's added to openCV, fglrx is closed source so can't do anything about that

At the moment I'm trying to get the openCV library to work with openCL (and there T-API).

At first i would like to state that I'm don't have much knowledge about the opencl specs, so please indicate when my
interpretations are correct.

A setup with the proprietary AMD fgrlx driver works fine with openCV, but it's a PITA to setup and update.
So i am trying to get it to work with Mesa/Clover, but i seem to have a few (related) issues:

The way OpenCV uses clCreateBuffer is as follows ( https://github.com/opencv/opencv/blob/master/modules/core/src/ocl.cpp#L4455 )

	if (u->origdata == cv::alignPtr(u->origdata, 4)) // There are OpenCL runtime issues for less aligned data
	{
	    handle = clCreateBuffer(ctx_handle, CL_MEM_USE_HOST_PTR|createFlags,
	                            u->size, u->origdata, &retval);
	}
	if((!handle || retval < 0) && !(accessFlags & ACCESS_FAST))
	{
	    handle = clCreateBuffer(ctx_handle, CL_MEM_COPY_HOST_PTR|CL_MEM_READ_WRITE|createFlags,
	                               u->size, u->origdata, &retval);
	    tempUMatFlags |= UMatData::TEMP_COPIED_UMAT;
	}

So it first tries with the flag CL_MEM_USE_HOST_PTR and only if that fails will use CL_MEM_COPY_HOST_PTR.

What i have observed and am wondering about is:
1) Removing the CL_MEM_USE_HOST_PTR flag from the first invocation in openCV makes things work with current upstream Mesa/Clover.
   Instead of carrying on it will now throw a "CL_INVALID_HOST_PTR" exception and openCV will now try with "CL_MEM_COPY_HOST_PTR|CL_MEM_READ_WRITE|createFlags" flags. This succeeds and all is well.
   (so it's related to this part of the code and this specific flag)
2) With the CL_MEM_USE_HOST_PTR flag still in place, openCV bails out with the following error:


3) I'm using a discrete graphics card (radeon HD7950) and not a APU, nor a card with hybrid memory.
   So I'm wondering if in that situation, shouldn't Mesa/Clover throw an exception when the CL_MEM_USE_HOST_PTR flag is specified in
   combination with a device which doesn't support "Host unified memory", or has this nothing to do with "Host unified memory" ?

   Here i see a few things i can't place, the openCV code dumps some stats about the openCL device:
   A) These values differ between the AMD fglrx driver and using Mesa/Clover:
      AMD fglrx:
        Current OpenCL device:
            Type = dGPU
            Name = Tahiti
            Version = OpenCL 1.2 AMD-APP (1702.3)
            Compute units = 28
            Max work group size = 256
            Local memory size = 32 kB
            Max memory allocation size = 2 GB 49 MB
            Double support = Yes
            Host unified memory = No
            Has AMD Blas = No
            Has AMD Fft = No
            Preferred vector width char = 4
            Preferred vector width short = 2
            Preferred vector width int = 1
            Preferred vector width long = 1
            Preferred vector width float = 1
            Preferred vector width double = 1


      Mesa/Clover:
        Current OpenCL device:
            Type = iGPU
            Name = AMD TAHITI (DRM 2.43.0 / 4.6.0-1-amd64, LLVM 3.8.1)
            Version = OpenCL 1.1 Mesa 12.1.0-devel (git-3e5db9a)
            Compute units = 28
            Max work group size = 256
            Local memory size = 32 kB
            Max memory allocation size = 3 GB
            Double support = Yes
            Host unified memory = Yes
            Has AMD Blas = No
            Has AMD Fft = No
            Preferred vector width char = 16
            Preferred vector width short = 8
            Preferred vector width int = 4
            Preferred vector width long = 2
            Preferred vector width float = 4
            Preferred vector width double = 2

       A1) Where do OpenCV and Mesa/Clover derive these values from ?
       A2) Is this an issue with the Linux kernel radeon firmware?

   B) I have added a lot of printk's to both openCV and Mesa/Clover, i have attached logs for all four scenario's
      (note that the opencv and clover printk output is not in sync, clover printk's seems to arrive later at stderr)

      From these i noticed some things and have some questions:

      B1) In the run with AMD fglrx driver, openCV doesn't hit the printk which indicates it is throwing an exception.
          So that would overthrow the whole "CL_MEM_COPY_HOST_PTR" in combination with "Host unified memory=NO" should throw an exception theory.
      B2) Since it works fine with the AMD fglrx driver and AMD APP SDK, the pattern that openCV uses seems legit and according to the
          openCL specs (1.1/1.2), but is that correct ?
      B3) Is it something completely different which I'm overlooking ?


Both openCV and Mesa/Clover are build from recent pulls from the upstream git repo's against llvm 3.8.1 on a machine with Debian
sid, running a recent kernel. The test is test with is the "opencv_perf_objdetect" binary, which is one of the performance tests
build by the openCV library.

Attached files:
    - amd-fglrx-opencv-vanilla.txt     WORKS (only printk's added to openCV, fglrx is closed source so can't do anything about that)
    - mesa-vanilla-opencv-vanilla.txt  FAILS (only printk's added, no functional changes to both projects)
    - mesa vanilla-opencv-patched.txt  WORKS (run with the patch "opencv-patch.txt" applied (and additional debug printk patches))
    - mesa-patched-opencv-vanilla.txt  WORKS (run with the patch "mesa-patch.txt" applied (and additional debug printk patches))

    - mesa-patch.diff    (patch against mesa-git-vanilla used to throw an exception when encountering CL_MEM_USE_HOST_PTR flag)
    - opencv-patch.diff  (patch against opencv-git-vanilla used to disable the use of the CL_MEM_USE_HOST_PTR flag by openCV)


Hopefully someone more knowledgeable about openCL could help me, it would be great if OpenCV can work out of the box with out of the box Mesa/Clover. I you need more info, or have any (debug) patches you would like me to run, please don't hesitate.

--
Sander