Bug 86388 - Page not rendered - text is wrong colour
Summary: Page not rendered - text is wrong colour
Status: RESOLVED FIXED
Alias: None
Product: poppler
Classification: Unclassified
Component: general (show other bugs)
Version: unspecified
Hardware: x86-64 (AMD64) Linux (All)
: medium normal
Assignee: poppler-bugs
QA Contact:
URL:
Whiteboard:
Keywords:
: 75027 (view as bug list)
Depends on:
Blocks:
 
Reported: 2014-11-17 12:32 UTC by David Hedley
Modified: 2014-11-28 10:14 UTC (History)
1 user (show)

See Also:
i915 platform:
i915 features:


Attachments
Incorrectly rendered page (21.84 KB, application/pdf)
2014-11-17 12:32 UTC, David Hedley
Details
Acrobat rendering (11.43 KB, image/png)
2014-11-17 14:01 UTC, David Hedley
Details
poppler-0.28.1 output (8.29 KB, image/png)
2014-11-17 22:19 UTC, David Hedley
Details
rendered in my machine (14.69 KB, image/png)
2014-11-17 22:40 UTC, Jose Aliste
Details
Use correct LAB byte array for lcms input (8.04 KB, patch)
2014-11-22 10:52 UTC, Thomas Freitag
Details | Splinter Review
Use correct LAB byte array for lcms input (rebased) (8.04 KB, patch)
2014-11-24 13:53 UTC, Thomas Freitag
Details | Splinter Review
Output after applying the patch (14.67 KB, image/png)
2014-11-24 16:50 UTC, Thomas Freitag
Details
print no instead of auto if lcms not found (634 bytes, patch)
2014-11-24 19:26 UTC, Adrian Johnson
Details | Splinter Review

Description David Hedley 2014-11-17 12:32:13 UTC
Created attachment 109615 [details]
Incorrectly rendered page

The attached page was generated from ghostscript-9.14.
I am using poppler-0.28.1.

The text and image on the page are not rendered correctly with either pdftoppm or pdftocairo

If you use pdftocairo -svg, you can see the text is in the, it's just drawn as white (on white!) rather than the correct colour.
Comment 1 Albert Astals Cid 2014-11-17 12:48:43 UTC
Do you have any pdf renderer that shows you the page as you expect it to show? To me all the renderers i have show it the same way so i'd say the pdf is just broken.
Comment 2 David Hedley 2014-11-17 12:56:48 UTC
It displays fine in Acrobat.

I understand that this may still mean the PDF is broken. I am just a bit surprised as it was generated with ghostscript.

I've just tried it with the Foxit reader and it displays identically to poppler.

Do you have an indication of what is broken in the PDF? I will submit a bug report to ghostscript.
Comment 3 David Hedley 2014-11-17 13:42:34 UTC
It also displays fine in ghostscript (9.02)
Comment 4 Albert Astals Cid 2014-11-17 13:52:38 UTC
Which version of acrobat?
Comment 5 Albert Astals Cid 2014-11-17 13:54:12 UTC
You'll have to attach what is "fine" then.
Comment 6 David Hedley 2014-11-17 13:59:27 UTC
It displays fine in:

Acrobat 9.5.2, 10.1.10 and 11.0.09
Ghostscript 9.02

FWIW, Preflight "Report PDF syntax issues" in Acrobat report no errors in the file.
Comment 7 David Hedley 2014-11-17 14:01:57 UTC
Created attachment 109621 [details]
Acrobat rendering

How Acrobat renders the PDF
Comment 8 David Hedley 2014-11-17 14:02:32 UTC
(Ghostscript renders it the same as Acrobat)
Comment 9 Albert Astals Cid 2014-11-17 21:19:42 UTC
Can you attach what you get? Because i actually get that. Also are you self compiling? or using a distro package?
Comment 10 David Hedley 2014-11-17 22:19:31 UTC
Created attachment 109646 [details]
poppler-0.28.1 output

This is output I get from poppler.
I get identical output to this with Foxit Reader and MuPDF.
I built poppler myself from source.
Comment 11 Albert Astals Cid 2014-11-17 22:21:05 UTC
Can you pelase attach the summary at the end of the configure/cmake step?
Comment 12 David Hedley 2014-11-17 22:23:56 UTC
Building poppler with support for:
  font configuration: fontconfig
  splash output:      yes
  cairo output:       yes
  qt4 wrapper:        no
  qt5 wrapper:        no
  glib wrapper:       yes
    introspection:    no
  cpp wrapper:        yes
  use gtk-doc:        no
  use libjpeg:        yes
  use libpng:         yes
  use libtiff:        yes
  use zlib:           no
  use libcurl:        no
  use libopenjpeg:    yes
  use cms:            auto
  command line utils: yes

There doesn't appear to be an easy way of getting the versions of all the dependencies, but if there are specific packages you'd like to know the versions of then please let me know.

Many thanks for looking at this
Comment 13 David Hedley 2014-11-17 22:27:38 UTC
System info:

Linux fedora 3.15.7-200.fc20.x86_64 #1 SMP Mon Jul 28 18:50:26 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
Comment 14 Albert Astals Cid 2014-11-17 22:33:40 UTC
looks similar to mine, i don't really have time to look into why doesn't work for you, all i can say is it works for me in ubuntu development branch (vivid)
Comment 15 David Hedley 2014-11-17 22:37:08 UTC
Are you 32 bit?
Comment 16 Jose Aliste 2014-11-17 22:40:37 UTC
Created attachment 109647 [details]
rendered in my machine

I get this rendering (either with pdftoppm or pdftocairo)
Comment 17 Albert Astals Cid 2014-11-17 22:59:15 UTC
No
Comment 18 David Hedley 2014-11-18 08:39:51 UTC
OK so that's 3 different results in 3 different environments.

Any pointers as to where to look in the code?
Color space handling? Pattern/Shading handling?
Comment 19 Thomas Freitag 2014-11-18 11:18:06 UTC
It seems to be the colorspace handling and there the lcms integration:
The missing or red painted elements should all be painted in the separation colorspace "pantone warm gray 7 c". "pantone warm gray 7 c" itself is an icc based colorspace with lab values input.
So we start i.e. with a color value of 0xff. This is transformed by the separation colorspace to the lab values (58.8, 3, 6), which still seems to be okay, but the transformation used on the icc based colorspace transforms it to rgb value (255,0,0) which is pure red.
Can't really figure out why this happens.
Comment 20 Thomas Freitag 2014-11-21 12:39:44 UTC
The problem is that poppler expects that all gfx color values are doubles in the range from 0 to 1. That's almost true, but NOT for LAB colorspaces.
So all the helper functions like dblToCol, colToByte, byteToDbl will not work for gfx color values in LAB colorspaces. The best way here would probably be to use the double values itself for the lcms transformation how the following small test program shows:

#include "lcms2.h"
typedef unsigned int GfxColorComp;

#define gfxColorComp1 0x10000

static inline GfxColorComp dblToCol(double x) {
  return (GfxColorComp)(x * gfxColorComp1);
}

static inline unsigned char colToByte(GfxColorComp x) {
  // 255 * x + 0.5  =  256 * x - x + 0x8000
  return (unsigned char)(((x << 8) - x + 0x8000) >> 16);
}

static inline double byteToDbl(unsigned char x) {
  return (double)x / (double)255.0;
}

int main(void)
{
  cmsHPROFILE hInProfile, hOutProfile;
  cmsHTRANSFORM hTransform, hTransform1;
  unsigned char in[3];
  unsigned char out[3];
  double col[3];
  int i;

  hInProfile = cmsOpenProfileFromFile("gray.icc", "r");
  hOutProfile = cmsCreate_sRGBProfile();
  hTransform = cmsCreateTransform(hInProfile, COLORSPACE_SH(PT_Lab) |CHANNELS_SH(3) | BYTES_SH(1), 
    hOutProfile, COLORSPACE_SH(PT_RGB) |CHANNELS_SH(3) | BYTES_SH(1),
    INTENT_PERCEPTUAL, 0);
  hTransform1 = cmsCreateTransform(hInProfile, COLORSPACE_SH(PT_Lab) |CHANNELS_SH(3) | BYTES_SH(0),
    hOutProfile, COLORSPACE_SH(PT_RGB) |CHANNELS_SH(3) | BYTES_SH(1),
    INTENT_PERCEPTUAL, 0);
  cmsCloseProfile(hInProfile);
  cmsCloseProfile(hOutProfile);

  col[0] = 58.8235; col[1] = 3; col[2] = 6;
  for (i = 0;i < 3;i++) {
    in[i] = colToByte(dblToCol(col[i]));
  }
  fprintf(stderr, "Poppler: %x %x %x ->", in[0], in[1], in[2]); 
  cmsDoTransform(hTransform, in, out, 1);
  fprintf(stderr, "%x %x %x\n", out[0], out[1], out[2]); 
  fprintf(stderr, "Use double: %f %f %f ->", col[0], col[1], col[2]); 
  cmsDoTransform(hTransform1, col, out, 1);
  fprintf(stderr, "%x %x %x\n", out[0], out[1], out[2]); 

  fprintf(stderr, "Convert byte back to double: %f %f %f\n", byteToDbl(in[0]), byteToDbl(in[1]), byteToDbl(in[2]));
}

Output of this program:

Poppler: 98 fd fa ->ff 0 0
Use double: 58.823500 3.000000 6.000000 ->96 8b 83
Convert byte back to double: 0.596078 0.992157 0.980392

(gray.icc is the ICC profile I extract from the PDF)
Comment 21 Thomas Freitag 2014-11-22 10:50:28 UTC
The sample program in comment 20 only works with lcms 1. lcms 2 seems to skip support for double in.
Comment 22 Thomas Freitag 2014-11-22 10:52:37 UTC
Created attachment 109843 [details] [review]
Use correct LAB byte array for lcms input

This patch convert LAB input values to the correct byte array needed by lcms.
Comment 23 David Hedley 2014-11-24 09:59:38 UTC
This patch does not apply cleanly for me. What source/branch is it against?

Thanks
Comment 24 Thomas Freitag 2014-11-24 13:53:52 UTC
Created attachment 109941 [details] [review]
Use correct LAB byte array for lcms input (rebased)

Ups. Normally all my patches are against git master. But this time I oversee that Albert just committed another of my patches, and I forgot a 'git pull'.

So here a rebased patch.
Comment 25 David Hedley 2014-11-24 16:28:08 UTC
The patch now applied cleanly thanks, however it makes no different to my output.

Given I appear to be getting different results to other people, is there a dependency on any external library or code that I can check?
Comment 26 Thomas Freitag 2014-11-24 16:43:45 UTC
(In reply to David Hedley from comment #25)
> The patch now applied cleanly thanks, however it makes no different to my
> output.
> 
> Given I appear to be getting different results to other people, is there a
> dependency on any external library or code that I can check?

Are You sure that You have lcms2 installed and poppler compiled against it? (lcms1 should also work, but I haven't tested it). Without lcms the PDF can't be rendered correctly by poppler, because the PDF doesn't specify an alternate colorspace for the ICC based colorspace. In this case, NO ICC support and NO alternate colorspace, the PDF spec says: 

If this entry is omitted and the conforming reader does not understand
the ICC profile data, the colour space that shall be used is DeviceGray,
DeviceRGB, or DeviceCMYK, depending on whether the value of N is 1, 3, or
4, respectively.

N is here 3, so DeviceRGB will be taken, but that's definitely wrong because the input values are LAB values. But this is not detectable when there is no ICC support, we get this information from the profile!!!
Comment 27 Thomas Freitag 2014-11-24 16:50:25 UTC
Created attachment 109951 [details]
Output after applying the patch

Without the patch and WITH lcms2 I get the output of comment 16, after applying the patch I got the attached output (similar to comment 7).

And this is the configure output:

Building poppler with support for:
  font configuration: fontconfig
  splash output:      yes
  cairo output:       yes
  qt4 wrapper:        yes
  qt5 wrapper:        no
  glib wrapper:       yes
    introspection:    no
  cpp wrapper:        yes
  use gtk-doc:        no
  use libjpeg:        yes
  use libpng:         yes
  use libtiff:        yes
  use zlib:           no
  use libcurl:        no
  use libopenjpeg:    yes
  use cms:            yes               <--- !!!!
      with lcms2                        <--- !!!!
  command line utils: yes
Comment 28 David Hedley 2014-11-24 17:07:10 UTC
Aha!

I had:
use cms:            auto

Although lcms2 was installed (lcms2-2.6-1.fc20.x86_64), lcms2-devel was not.
With it now installed, I get:
  use cms:            yes
      with lcms2

And the output is correct!

Excellent job thanks
Comment 29 Adrian Johnson 2014-11-24 19:26:20 UTC
Created attachment 109960 [details] [review]
print no instead of auto if lcms not found
Comment 30 Albert Astals Cid 2014-11-28 10:07:44 UTC
*** Bug 75027 has been marked as a duplicate of this bug. ***
Comment 31 Albert Astals Cid 2014-11-28 10:14:03 UTC
I've commited Thomas' and Adrian's patches, thanks all \o/


Use of freedesktop.org services, including Bugzilla, is subject to our Code of Conduct. How we collect and use information is described in our Privacy Policy.