Bug 26243 - PDF renders incorrectly
Summary: PDF renders incorrectly
Status: RESOLVED WORKSFORME
Alias: None
Product: poppler
Classification: Unclassified
Component: general (show other bugs)
Version: unspecified
Hardware: x86 (IA32) Linux (All)
: medium normal
Assignee: poppler-bugs
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2010-01-26 05:14 UTC by Harry Roberts
Modified: 2010-02-09 15:14 UTC (History)
1 user (show)

See Also:
i915 platform:
i915 features:


Attachments
Problem PDF (811.61 KB, application/pdf)
2010-01-26 05:14 UTC, Harry Roberts
Details
poppler glib demo rendering the document correctly (516.76 KB, image/png)
2010-01-26 12:49 UTC, Carlos Garcia Campos
Details
First step to solve it (136.68 KB, image/jpeg)
2010-01-28 11:46 UTC, Thomas Freitag
Details

Description Harry Roberts 2010-01-26 05:14:42 UTC
Created attachment 32823 [details]
Problem PDF

I've been trying to rasterize this document, however in Poppler I've ended up with rather strange output.

XPDF renders it .. nearly correctly, I'm trying to figure out if it's a bug regression or if it's something new from Poppler.

And as always Photoshop seems to render flawlessly.

In both cases i'm using the Splash backend and the pdftopnm application

Poppler (from custom app) output: http://midnight-labs.org/cal/wd164/page_0076_large.png
Poppler pdftoppm output: http://midnight-labs.org/cal/wd164/page_0076_large.png-1.png
XPDF 3.02 pdftoppm output: http://midnight-labs.org/cal/wd164/-000001.jpg
Photoshop CS3 'reference' output: http://i.imgur.com/iu8MW.jpg (ignore the colour space differences)

I'm using poppler from git, last revision in my clone is: 
commit 6c61a457e9a8ec10945bc1e0700c8e4d121faa58
Author: Albert Astals Cid <aacid@kde.org>
Date:   Wed Jan 20 21:59:05 2010 +0000

with pdftoppm I was using `-aa yes -aaVector no -freetype yes`, though there's little difference with different options.

with the custom application I'm rendering at 2-3x original size without vector antialiasing but *with* freetype antialiasing, this gets rid of the weird issue with small white lines around boxes and after downsampling it looks nearly as good as the vector-antialiased version.

Many regards,
 - Harry Roberts
Comment 1 Carlos Garcia Campos 2010-01-26 05:21:48 UTC
It works for me with cairo backend, I can reproduce it with splash. 
Comment 2 Harry Roberts 2010-01-26 12:29:21 UTC
I've just replaced the SplashOutputDev based code in my app with Cairo support and I'm getting the same results with both.

See: http://i.imgur.com/oe51z.jpg

This leads me to think it's a bug in poppler PDF drawing code rather than the graphics output devices... 
Comment 3 Carlos Garcia Campos 2010-01-26 12:49:21 UTC
Created attachment 32826 [details]
poppler glib demo rendering the document correctly

As I said it works for me with the cairo backend. It doesn't with splash.
Comment 4 Harry Roberts 2010-01-26 13:13:26 UTC
hmm, which revision of poppler are you using?

Also, weirdly, the numbers in the blue triangles are missing in that one you posted :(

(In reply to comment #3)
> Created an attachment (id=32826) [details]
> poppler glib demo rendering the document correctly
> 
> As I said it works for me with the cairo backend. It doesn't with splash. 
> 

Comment 5 Albert Astals Cid 2010-01-26 13:51:21 UTC
fails for me in master with both splash and cairo backends too
Comment 6 Albert Astals Cid 2010-01-26 14:21:23 UTC
For me the problem starts at 104f9286ceb5fcb5f4795bca7633029142d5f6a4

Thomas could you have a look? That commit is the one in which we added "Support colorizing text in pattern colorspace"
Comment 7 Carlos Garcia Campos 2010-01-27 00:46:42 UTC
(In reply to comment #4)
> hmm, which revision of poppler are you using?
>

git master, and cairo from git master too. 

> Also, weirdly, the numbers in the blue triangles are missing in that one you
> posted :(

right, there are still some other issues. 

> (In reply to comment #3)
> > Created an attachment (id=32826) [details] [details]
> > poppler glib demo rendering the document correctly
> > 
> > As I said it works for me with the cairo backend. It doesn't with splash. 
> > 
> 

Comment 8 Thomas Freitag 2010-01-27 02:06:21 UTC
(In reply to comment #6)
> For me the problem starts at 104f9286ceb5fcb5f4795bca7633029142d5f6a4
> 
> Thomas could you have a look? That commit is the one in which we added "Support
> colorizing text in pattern colorspace"
> 
I just took a quick look at it, but I don't think, that is comes from the pattern colorspace, even if it runs several times in the added routines: If I disable support of pattern colorspace for text and masks in SplashOutputDev (just comment out fillMaskCSPattern and supportTextCSPattern in SplashOutputDev.h, then it runs the default way), the output becomes even more worth.
Do You have any other hint? Must be some other code that that results in that output. I'll take a deeper look in it, but I fear that will take a little bit!
Comment 9 Albert Astals Cid 2010-01-27 15:35:15 UTC
For me it's all fault of this patch http://cgit.freedesktop.org/poppler/poppler/commit/?id=104f9286ceb5fcb5f4795bca7633029142d5f6a4 Without it works and with it does not.

The problem is detecting what part of the patch is the problematic one :D
Comment 10 Thomas Freitag 2010-01-28 11:46:44 UTC
Created attachment 32879 [details]
First step to solve it
Comment 11 Thomas Freitag 2010-01-28 11:55:15 UTC
(In reply to comment #9)
> For me it's all fault of this patch
> http://cgit.freedesktop.org/poppler/poppler/commit/?id=104f9286ceb5fcb5f4795bca7633029142d5f6a4
> Without it works and with it does not.
> 
> The problem is detecting what part of the patch is the problematic one :D
> 

Yes, You're true. The easiest way to solve it is to call

state->setRender(0);

instead of

state->setRender(savedRender);

in endTextObject (SplashOutputDev.cc and PSOutputDev.cc).
But it seems, as if there is still a small problem left, see marked area in "First step to solve it" (but this problem also exists in "poppler glib demo rendering the document correctly". But to take a deeper look into it, I need some time, and even then, I first must go to the actual poppler release (I'm still testing with poppler 0.10.2 :-( ).
So decide by yourself, if You make the smaller changes (which are quick & dirty, I don't know anything about side effects!) for the moment
Comment 12 Albert Astals Cid 2010-01-28 13:56:30 UTC
I'll regtest that change and see if it gives us improvements or not, and please take a look, no matter how much time you need, it is very appreciated.
Comment 13 Harry Roberts 2010-01-31 22:12:46 UTC
Cool, I've done some testing and this fixes the main issue. I've also done regressions tests on a few other PDFs and not noticed any issues, though I don't want to bet on it subtly breaking another edge cases.

I just wish I could've been more help, but I'm still getting my head round the XPDF/Poppler rendering pipeline (and how PDFs work in general).

Since I started working with magazine PDFs I've seen so many weirdly broken ones (yes, Adobe Illustrator... I'm looking at you) that it's been a constant nightmare trying to get things to look right... so far Poppler is doing a stellar job :D

(In reply to comment #11)
> Yes, You're true. The easiest way to solve it is to call
> 
> state->setRender(0);
> 
> instead of
> 
> state->setRender(savedRender);
> 
> in endTextObject (SplashOutputDev.cc and PSOutputDev.cc).
Comment 14 Albert Astals Cid 2010-02-03 15:52:16 UTC
@Thomas: Unfortunately your patch breaks page 8 (a T in the vertical title is not rendered anymore) of this pdf http://people.freedesktop.org/~aacid/bug155022.pdf

Do you think you'll be able to have a patch before end of March? If not any hint so we can try to fix it ourselves?
Comment 15 Thomas Freitag 2010-02-07 07:25:38 UTC
(In reply to comment #14)
> @Thomas: Unfortunately your patch breaks page 8 (a T in the vertical title is
> not rendered anymore) of this pdf
> http://people.freedesktop.org/~aacid/bug155022.pdf
> 
> Do you think you'll be able to have a patch before end of March? If not any
> hint so we can try to fix it ourselves?
> 

This regression doesn't come from a pattern colorspace, this PDF code is handled wrong:
q
578.159970 476.880040 12.719971 273.359950 re
W
n
/Cs1 cs
1 1 1 sc
q
0 -0.240000 0.240000 0 580.080020 750.240050 cm
BT
42 0 0 42 0 0 Tm
/F1#2e0 1 Tf
(TRANSP) Tj
42 0 0 42 169.558590 0 Tm
(ARENT SPEAKER CABLES) Tj
ET
Q
Q

Cs1 is a ICC based colorspace. I looked at it, and either the clip rectangle is calculated wrong or the starting textposition. I would expect that it is the clip rectangle, because it seems as if the position of the displayed text is okay, but I'm not an expert in this part of SplashOutputDev.
And finally PSOutputDev with my path handles this page correctly.
Are there any other experts in this? I fear it will take a long time for me to find and correct this error!
Comment 16 Thomas Freitag 2010-02-07 07:31:32 UTC
I had a deeper look into that and examined where my code works wrong: It is in Gfx.cc, opSetFillColorSpace. because during an BT-ET bracket the pattern colorspace chnaged serveral times. The correct code should therefore be:

	if (drawText) {
		if (colorSpace->getMode() == csPattern) {
                        // start new statements
			if (out->supportTextCSPattern(state) && textHaveCSPattern) {
				GBool needFill = out->deviceHasTextClip(state);
				out->endTextObject(state);
				if (needFill)
					doPatternFill(gTrue);
				out->restoreState(state);
			}
                        // end new statements
			colorSpaceText = NULL;
			textHaveCSPattern = gTrue;
			out->beginTextObject(state);
		} else if (textHaveCSPattern) {
			GBool needFill = out->deviceHasTextClip(state);
			out->endTextObject(state);
			if (needFill)
				doPatternFill(gTrue);
			out->beginTextObject(state);
			out->updateRender(state);
			out->updateTextMat(state);
			out->updateTextPos(state);
			textHaveCSPattern = gFalse;
		}
	}

Unforunately this still not solve the regression in the cloud, therefore I'll have anothher look at it next weekend
Comment 17 Albert Astals Cid 2010-02-08 15:33:31 UTC
You rock, i'll pass this through the regrssion testing asap.
Comment 18 Thomas Freitag 2010-02-09 00:24:16 UTC
(In reply to comment #17)
> You rock, i'll pass this through the regrssion testing asap.
> 

Please keep in mind that page 8 from comment #14 still not works correctly with the patch. This is not a pattern colorspace issue!
Comment 19 Albert Astals Cid 2010-02-09 00:44:27 UTC
I think you didn't understand my problem with that page, the problem was that the
state->setRender(0);
call made the T in page 8 disappear (a regression), not that the images paint with "weird" colors (always has happened)

This new patch is OK for me if fixes this file and doesn't cause other regressions, i'm not asking you to fix the weird coloring on page 8 of file in comment #14, though of course if you do you'll be my hero ;-)
Comment 20 Albert Astals Cid 2010-02-09 13:45:45 UTC
I've commited the patch that fixes the rendering of the main area of the file, the bubble thing is minor, but i'll leave the bug open since it also should be ideally fixed.
Comment 21 Harry Roberts 2010-02-09 14:04:35 UTC
(In reply to comment #20)
> I've commited the patch that fixes the rendering of the main area of the file,
> the bubble thing is minor, but i'll leave the bug open since it also should be
> ideally fixed.
> 

Personally I'd just close the bug, my main problem was the masking stuff (which was fixed with setRender(0)). You guys are awesome, I stared at code for hours and couldn't think of a fix.

I have a handful of PDFs that are causing problems that I need to get around to posting as bugs, but generally the bugs are "white lines" around objects (as can be seen around the cloud in the PDF I attached).

That problem is a weird one though, I've used perhaps 4-5 different libraries/programs to do PDF rasterizing and they all come up with the white lines, except Photoshop. The only difference I can think of is that perhaps Photoshop is rounding out to the next nearest point/subpoint; I'm honestly stumped at how print shops can convert these PDFs into the magazines you see on store shelves considering how much weird stuff I've seen.
Comment 22 Albert Astals Cid 2010-02-09 15:14:40 UTC
Ok, the reporter says our solution works for him, closing the bug, of course work in improvements is still more than welcome. Thanks to everyone involved, specially Thomas.


Use of freedesktop.org services, including Bugzilla, is subject to our Code of Conduct. How we collect and use information is described in our Privacy Policy.