Use Y value 0x42, Cb Value 0x65, Cr Value 0x5E
The resulted RGB value should be 0x04, 0x60, 0x04, which is according to https://en.wikipedia.org/wiki/YCbCr
But Shader CSC result is 0x05, 0x60, 0x05.
In some situation, it has a bit difference with what is precise
Seems the matrix it is using has some problem
Do you have a test program that shows the issue?
Yes I have.
Also piglit test has YUV2RGB test
You can extend this with the problematic YUV value:
Author: Johnson Lin <firstname.lastname@example.org>
Date: Thu May 4 14:37:52 2017 +0800
nir/lower_tex: Fix minor error in YUV color conversion matrix
The matrix used for YCbCr to RGB is listed in:
There was an error in converting the offsets from integers to unorm
values: 0.0625=16/256 should be 16.0/255,and 0.5=128.0/256 should be
128.0/255. With this fix, the CSC result is bit aligned with wikipedia's
conversion result and FFMPeg's result.
Reviewed-by: Eric Anholt <email@example.com>
Reviewed-by: Kristian H. Kristensen <firstname.lastname@example.org>