Created attachment 40578 [details] pdf that shows the problem The non-dictionary form of the imagemask operator has a parameter called "polarity". According to the postscript language reference manual, Decode [1 0] -> polarity true = paint 1 bits, ignore 0 bits, and Decode [0 1] -> polarity false = ignore 1 bits, paint 0 bits. PSOutputDev.cc calls the "polarity" parameter "invert". With level 2 output, it doesn't matter as long as the mapping back and forth between the Decode dictionary entry and the poppler "invert" variable is consistent. With level 1 output, PSOutputDev.cc writes the "invert" variable as the "polarity" argument to imagemask (wrapped by pdfImM1), and as a result, the polarity of imagemasks is flipped with -level1sep. The patch below fixes the problem. --- poppler-0.15.2/poppler/PSOutputDev.cc- 2010-11-10 23:31:51.000000000 +0100 +++ poppler-0.15.2/poppler/PSOutputDev.cc 2010-11-26 03:52:13.871729108 +0100 @@ -4544,7 +4562,7 @@ // image/imagemask command if ((inType3Char || preload) && !colorMap) { writePSFmt("{0:d} {1:d} {2:s} [{3:d} 0 0 {4:d} 0 {5:d}] pdfImM1a\n", - width, height, invert ? "true" : "false", + width, height, invert ? "false" : "true", width, -height, height); } else if (colorMap) { writePSFmt("{0:d} {1:d} 8 [{2:d} 0 0 {3:d} 0 {4:d}] pdfIm1\n", @@ -4552,7 +4570,7 @@ width, -height, height); } else { writePSFmt("{0:d} {1:d} {2:s} [{3:d} 0 0 {4:d} 0 {5:d}] pdfImM1\n", - width, height, invert ? "true" : "false", + width, height, invert ? "false" : "true", width, -height, height); }
Created attachment 40579 [details] [review] proposed patch
Created attachment 40586 [details] screen capture of correct rendering
Created attachment 40587 [details] screen capture of incorrect rendering with white bar at lower left with -level1sep
This problem is also in xpdf 3.02, and the confusion with polarity and invert could have been a factor in the imagemask issue of bug 22216 https://bugs.freedesktop.org/show_bug.cgi?id=22216 although 22216 is a different issue. This new patch does not correct the -level1sep rendering of the pdf in the 22216 report.
Created attachment 40589 [details] Pdf that breaks with the patch I don't think your patch is correct it breaks lots of pdf i have lying around, attaching one.
You are right. My new PDF draws through a pattern similar to bug 22216. In PSOutputDev::drawImageMask(), state->getFillColorSpace()->getMode() == csPattern, which is currently not handled for level1 and level1sep. My goal is to create an EPS that I can separate using a prologue that redefines color operators. I do not need a strictly level1 eps. I just need something that works with my separation prologue or that I can enhance my separation prologue to support. When I use -level2sep with the PDF attached to this report and try to separate the eps, gs gets "/typecheck in --image--" on the second image, which I think is the image with the pattern that shows as a white bar with -level1sep. Since the attached PDF displays correctly by flipping the polarity argument to imagemask, the pattern is probably trivial, either all set or all clear. As a partial fix, would it be possible to scan the data to check for special cases where inverting the imagemask polarity will work?
No idea really, i know almost nothing about PostScript
I looked at the output more closely. The original pdf has a black shadow around the letters, while the ps produced with my patch does not show the shadow. It is hard to see because the shadow is small and the background is very dark. That means that this file is not even an example of a special case where inverting the polarity will help. Maybe for now, the best that I can do is to write a note in the trailer if PSOutputDev::drawImageMask() finds state->getFillColorSpace()->getMode() == csPattern when producing level1sep output. Then applications that read the eps can know that it is incorrect and show a warning.
Created attachment 40621 [details] [review] add a comment to the trailer when pdftops creates incorrect output This patch writes the comment "%%XHasColorPattern" to the postscript trailer when pdftops runs with -level1 or -level1sep and finds an image where state->getFillColorSpace()->getMode() == csPattern. The generated postscript will not get postscript errors printing, but it will not produce the same image as the PDF. The level1 output code in PSOutputDev.cc does not support this case, and the generated postscript will often display a white or black rectangle over part of the image. This patch allows applications that depend on level1 output from pdftops to detect postscript files that have not been translated correctly.
Created attachment 40622 [details] [review] patch to add tiff support to pdftoppm This patch adds tiff support to pdftoppm using libtiff. The patch updates configure.ac and two Makefile.am files, so you need to run poppler's autogen.sh after applying it. For my purposes, I can get a usable level1 eps by running pdftoppm with tiff output at a high resolution and then sending the tiff to tiff2ps (a tool distributed with tiff-v3.4beta037).
(In reply to comment #9) > Created an attachment (id=40621) [details] > add a comment to the trailer when pdftops creates incorrect output > > This patch writes the comment "%%XHasColorPattern" to the postscript trailer That's not a standard DSC comment. I would recommend using a non DSC style comment ie single % followed by space. eg "% HasColorPattern" to avoid causing problems for software expecting DSC compliant output from poppler.
Created attachment 40639 [details] [review] revised color comment patch >I would recommend using a non DSC style comment ie single % followed by space. eg "% HasColorPattern" I revised the patch as you suggested.
Hi, i'm not commiting any of the two patches since from my point of view this are new features and not bugfixes and trunk is currently frozen for new features until we release 0.16.0 late december. About the tiff patch, it would be cool if you opened a new bug since it has nothing to do with this. Also it has two features in there, singlepage and tiff support which i'd prefer as separate patches/bugs. Also CMake build system support is missing for it. Finally i'd appreciate if you went the same way we did with jpeg and png support and implemented a class that inherits from ImgWriter
>About the tiff patch, it would be cool if you opened a new bug >it has two features in there, singlepage and tiff support which i'd prefer as separate patches/bugs I'll make new bug entries for them in a few days. >Also CMake build system support is missing for it. Is that just adding a #cmakedefine line to config.h.cmake? >Finally i'd appreciate if you went the same way we did with jpeg and png support and implemented a class that inherits from ImgWriter OK.
Created attachment 40745 [details] Correct rendering from gv + gs-9.00
Created attachment 40747 [details] Incorrect rendering from evince 2.32.0 + popper/cairo 0.14.5 I was looking at this file in more detail, and gv+gs shows the letters with a black faded shadow, while evince+poppler does not show the shadow. With poppler-0.15.2, both pdftops (when not using -level1) and pdftoppm show the black shadow correctly, so the problem might be limited to the older evince and poppler used by Fedora 14. I added a patch for the -singlefile option as bug 32025 https://bugs.freedesktop.org/show_bug.cgi?id=32025 and I added a patch for tiff support in pdftoppm as bug 32027 https://bugs.freedesktop.org/show_bug.cgi?id=32027
Created attachment 41132 [details] [review] revised color comment patch This revised patch writes the HasColorPattern trailer comment in two additional cases where a PDF has a mask that is not handled by pdftops -level1sep.
Sincerely i fail to see the point of this patch. So instead of doing it right we are adding a random comment noone knows about saying "Hey, we tried but failed". Why?
>So instead of doing it right we are adding a random comment noone knows about saying "Hey, we tried but failed". Why? I have an application that depends on converting PDF to separable Level 1 EPS, and the EPS must print exactly the same way as the PDF. poppler's pdftops is the best way that I have found to do this. pdftops runs quickly and produces EPS that works well with my application. The problem is that I have no control over the incoming PDFs, and about 1 in 100 uses a color image mask in a way that pdftops produces an incorrect EPS. I need some way to detect that the EPS is unusable. I don't care if pdftops writes a mark in the file (like the comment produced by the patch), or if pdftops returns an error status to the shell, or if pdftops deletes the output file. I just need to know that pdftops wrote an unusable file so that my application can inform the user or can revert to another less efficient method like rasterizing the PDF to a very high resolution tiff and then converting the tiff to an EPS. Of course, I would rather do it right, but I have not figured out how. I submitted a patch to add support for Level 1 output from black and white image masks https://bugs.freedesktop.org/show_bug.cgi?id=32365 but I have not yet figured out how to generate Level 1 output from color image masks. William
I am regtesting 32365 at the moment, it takes a while. About this one, it seems a hotfix for your application which i don't think it's right to add to mainline poppler. If we can not aim at the correct fix (maybe PS can simply not do that?) the second correct fix as far as i understand would be the PreScanOutputDev detecting that situation and maybe depending on a command line output * a) returning an error or * b) rasterizing the output What do you think?
>If we can not aim at the correct fix (maybe PS can simply not do that?) I am pretty sure that PS can do that because one of the people using my application has a high-end third-party commercial application that can open the PDFs that pdftops doesn't convert correctly and save them as Level 1 separable EPS files that work with my application. The problem is that it is a manual process (while pdftops is scriptable) and not everyone using my application has another way to convert the PDF into a Level 1 EPS. Also, the people using my application currently have to review the printed output manually to look for PDFs that pdftops did not convert correctly. The differences are sometimes very subtle, but it can be expensive if the user does not notice an incorrect image. >detecting that situation and maybe depending on a command line output >* a) returning an error >or >* b) rasterizing the output Either of those is fine with me. Rasterizing the output (based on a resolution passed on the command line) would be better if it is possible. If pdftops returns an error code, then I would have to revert to rasterizing the entire PDF at a very high resolution to ensure that small fonts are readable, but if pdftops could know how to rasterize only what is necessary, the resulting EPS would be smaller and would produce a cleaner image. William
Any chance i can convince you to try to implement the PreScanOutputDev solution?
>Any chance i can convince you to try to implement the PreScanOutputDev solution? Do I just need to find the locations in my patch in PSOutputDev where it sets "hasColorPattern" and make the corresponding place in PreScanOutputDev set "transparency"?
I would just add another bool member like "level1PSBug" or something like that to PreScanOutputDev. And then PSOutputDev could read that variable and a variable passed from pdftops to decide if rasterize or simply return error.
>I would just add another bool member like "level1PSBug" I should have time to do that in a week or two. Should it be a new entry or be attached to this one? The patch https://bugs.freedesktop.org/show_bug.cgi?id=32365 fixes the cases for black and white masks, which affects the cases that PreScanOutputDev would have to check for. It would be easier if I could start with a version of poppler with that patch applied. Have you had a chance to review it?
Just attach it here.
Created attachment 42353 [details] [review] patch to fix level1 output in pdftops This patch adds a check in PreScanOutputDev to set a new flag called level1PSBug for -level1 and -level1sep if it finds an image with state->getFillColorSpace()->getMode() == csPattern. PSOutputDev now rasterizes if either transparency or level1PSBug is set. This corrects pdftops -level1 and -level1sep output both for the PDF attached to this bug and for the PDF attached to bug 22216 https://bugs.freedesktop.org/show_bug.cgi?id=22216 The affected files end up larger due to the rasterization, but it is better than being small and wrong. William
Any reason you need the level to be a member of the class instead of a local in the only function that seems to use it?
I saved globalParams->getPSLevel() in a variable because I had originally thought that I would need to test it in more places. Also, GlobalParams::getPSLevel() might need to lock the mutex that protects the global parameters before reading the value of psLevel. I thought that it was better to call globalParams->getPSLevel() just once than to do it one or twice for every image. I think that is why PSOutputDev has its own local copy of level. If you want, I can redo the patch without the local copy of level in PreScanOutputDev or I can change the PreScanOutputDev constructor to take the level as an argument so it never has to call globalParams->getPSLevel(). William
Makes sense, i'll run a regtest to make sure we don't rasterize something that doesn't need to be rasterized.
Found a pdf file that is rasterized because of this patch and the non rasterized version seems fine, this is bad since it means the file is bigger and has a general worse quality. It's too big so i'll mail it to you privately.
Created attachment 42502 [details] [review] patch to fix bug-poppler27482.pdf for levels 2 and 3 >Found a pdf file that is rasterized because of this patch and the non rasterized version seems fine When I process this file with pdftops for level2 or level3, it looks bad to me when I view the generated postscript with gv+gs and compare it with the bug-poppler27482.pdf viewed with gv+gs. I uncompressed the PDF with a short C program called pdfdecode. Here is a section from the end. 136.628387 69.802902 m 164.446289 56.828781 l 148.904312 23.505157 l 121.086411 36.479286 l h W n q 136.628204 69.803017 m 164.446152 56.828873 l 148.90419 23.505249 l 121.086235 36.479393 l h The corresponding section of the level2sep output is 136.6284 69.8029 m 164.4463 56.8288 l 148.9043 23.5052 l 121.0864 36.4793 l 136.6284 69.8029 l h W q 136.6282 69.803 m 164.4462 56.8289 l 148.9042 23.5052 l 121.0862 36.4794 l 136.6282 69.803 l h I think that dropping the number of decimal places from 6 to 4 is enough to break the way that the PDF draws the little rectangles that form the black shading behind the big letters at the top. I think that the person who made the PDF rasterized the shadow. This patch changes a bunch of ".4g" formats to ".6g" in PSOutput.cc, and the level2 results look much better. I changed the formats with positions to 6 decimal places but left the formats with colors at 4 decimal places. This PDF has a color image mask. It must be rasterized for level1 output or else the image will be wrong. I think that since the level2 image is wrong, both the level1 and level2 images end up looking similar, and that is why it seems like the level1 image does not need to be rasterized, but I think that they are both wrong. I will post gv+gs screen captures.
Created attachment 42504 [details] level1sep output of bug-poppler27482.pdf before my rasterization patch
Created attachment 42505 [details] level1sep output of bug-poppler27482.pdf after my rasterization patch
Created attachment 42506 [details] level2sep output of bug-poppler27482.pdf before my precision patch
Created attachment 42507 [details] level2sep output of bug-poppler27482.pdf after my precision patch
Created attachment 42525 [details] level1sep output of bug-poppler27482.pdf by pdftops-0.14.5 This image shows what happens using -level1sep without rasterization. I used pdftops-0.14.5 provided by Fedora 14 in /usr/bin. The white text in the green bar at the top of the image is unreadable. The black shadow behind the letters uses a color mask. Since pdftops does not implement color masks, the black shadow shows as a white box and merges with the letters.
Right, i didn't appreciate the overlapping curve with the text. Iḿ doing something else today but tomorrow i'll continue regtesting this patch and the one for the increased precision for level2 and 3
>i'll continue regtesting Thanks! I will have another patch in a few days to make the rasterization resolution settable on the command line. PSOutputDev.cc sets splashDPI to 300, but some of my files are printed on a press at 1024dpi and have text that looks bad when rasterized at 300dpi. Rasterizing at 1024dpi will make huge files, so I don't want to hard-code it in PSOutputDev.cc, but it would be useful on the pdftops command line. It would also give people the option to rasterize at a resolution lower than 300 if they wanted.
Commited the increased precision patch, still working on the regtest for the other one.
Thanks!
Hmmm, sorry to come back to bug-poppler27482.pdf again, but if i run it with level1 using poppler 0.16.2 + your precision patch it looks ok (but then your patch decides to rasterize it) So as i was saying maybe the condition to decide to rasterize is too agressive?
Created attachment 42787 [details] pdftops 0.16.2+precisionpatch -level1sep bug-poppler27482-precision.pdf The attached image shows the result of pdftops 0.16.2 with the precision patch with -level1sep on bug-poppler27482-precision.pdf. I circled the white text on the green background at the top. pdftops 0.16.2 does not have the rasterization patch, and the output looks wrong to me. Compare it with the output from a version with the patch. The letters are not clear. They are supposed to have a black shadow, but without the rasterization patch, the rectangles that hold the bitmaps for the black shadow become entirely white. I think that the rasterization is necessary for level 1 output. Does something different happen for you? Can you attach a screen capture of what the generated -level1sep postscript looks like in a postscript viewer? I could have made a mistake with the different versions and patches.
I get exactly that output with -level1sep, but i was speaking about -level1 (which does not have that problem as far as i can see). Can you try -level1 and see if you get different results?
Created attachment 42845 [details] Three images of bug-poppler27482-precision.pdf I think that the rasterization is necessary. Without the rasterization, the mask becomes a solid black block. In some cases, the image looks reasonable with the mask solid white or black, but in general it does not work. I have attached close-ups of three versions of the image of bug-poppler27482 using gv + gs 9.01. The first is a view of the ps generated by pdftops-0.16.2 + the precision patch with -level1 bug-poppler27482.pdf The second is a view of the ps generated by pdftops-0.16.2 + the precision patch + the level1bug rasterization patch with -level1 on bug-poppler27482.pdf. The third is a view of the original pdf. I circled a part of the shadow at the upper right of the "S" in the word "Samstag" (at the upper left of the full image). In the first image, without the level1bug patch, the shadow is solid black. In the second image, with the level1bug patch, and in the third image, the shadow is shaded. The same difference between solid black and gray scaled shading happens in other places, for example, at the bottoms of the letters. The solid black without the level1bug patch is wrong. It is barely visible in this image, but it ruins the image in other files, like the faces at the bottom of the testimagemask.pdf file attached to https://bugs.freedesktop.org/show_bug.cgi?id=22216 . In this file, three of the faces are gray scaled through a mask, and without the level1bug patch, those faces become black like the background. The second image in the triplet shows why the level1bug patch should be followed with another patch that makes the resolution settable. The rasterization is currently hard-coded to 300 dpi which is good enough for pictures but a little low for text. I am still planning on doing it.
Ok, i've commited the patch. Also i'm closing this bug. Could you please open a new one for future reference saying there is an issue cnverting to level1/level1sep and that we have a workaround for it that should be removed once the fix is developed? Thanks.
>Ok, i've commited the patch. Thanks! If I have another issue, I'll open a new bug. William
Use of freedesktop.org services, including Bugzilla, is subject to our Code of Conduct. How we collect and use information is described in our Privacy Policy.