Bug 58966 - "Syntax Error (202): Command token too long"
Summary: "Syntax Error (202): Command token too long"
Status: RESOLVED FIXED
Alias: None
Product: poppler
Classification: Unclassified
Component: general (show other bugs)
Version: unspecified
Hardware: x86-64 (AMD64) Linux (All)
: medium normal
Assignee: poppler-bugs
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2013-01-03 03:00 UTC by Robert Muil
Modified: 2013-01-04 10:59 UTC (History)
1 user (show)

See Also:
i915 platform:
i915 features:


Attachments
An example file demonstrating the problem. (17.72 KB, text/plain)
2013-01-03 03:00 UTC, Robert Muil
Details
fix bug in linearization code when first object is a stream (1.08 KB, patch)
2013-01-03 05:19 UTC, Adrian Johnson
Details | Splinter Review
make parser return error object if stream found when streams are disabled (1.82 KB, patch)
2013-01-03 05:58 UTC, Adrian Johnson
Details | Splinter Review
72423: make parser return error object if stream found when streams are disabled (1.82 KB, patch)
2013-01-03 06:01 UTC, Adrian Johnson
Details | Splinter Review

Note You need to log in before you can comment on or make changes to this bug.
Description Robert Muil 2013-01-03 03:00:53 UTC
Created attachment 72413 [details]
An example file demonstrating the problem.

Some PDFs seem to be broken in a way that generates the error message "Syntax Error (202): Command token too long". This is emitted, assumedly from the poppler library, to stderr, and it is not obvious how to trace this to where in the PDF file the error is occurring.

This has already been documented here: http://sarovar.org/tracker/index.php?func=detail&aid=4347&group_id=106&atid=493 and here: http://tex.stackexchange.com/questions/42348/pdflatex-command-token-too-long-error

I've attached a file that generates the error. To recreate, simply run:

`pdftotex stimulus_category_influence_on_ratings_videos_emoman.pdf`

BTW, the PDF file contains no text, by design.

Rob.
Comment 1 Adrian Johnson 2013-01-03 05:17:37 UTC
(In reply to comment #0)
> I've attached a file that generates the error. To recreate, simply run:
> 
> `pdftotex stimulus_category_influence_on_ratings_videos_emoman.pdf`

I assume you mean 'pdftotext'. I can reproduce the error with pdftotext, pdftoppm, and pdftocairo. Ghostscript and acroread open the file without errors or warnings.

I had a look at where the error is coming from and it turns out it is in the check for linearized files in Linearization::Linearization. This function checks if the first object in the file after the header is a Linearization dictionary. The problem is this function has disabled the 'allow streams' flag. As a result when opening a PDF file where the first object is a stream results in the error due to trying to parse the stream as if it were not a stream.
Comment 2 Adrian Johnson 2013-01-03 05:19:29 UTC
Created attachment 72420 [details] [review]
fix bug in linearization code when first object is a stream

With this patch the error is no longer displayed.
Comment 3 Robert Muil 2013-01-03 05:23:06 UTC
I did indeed mean pdftotext, sorry.

That was amazingly fast! Chapeau!

(In reply to comment #1)
> (In reply to comment #0)
> > I've attached a file that generates the error. To recreate, simply run:
> > 
> > `pdftotex stimulus_category_influence_on_ratings_videos_emoman.pdf`
> 
> I assume you mean 'pdftotext'. I can reproduce the error with pdftotext,
> pdftoppm, and pdftocairo. Ghostscript and acroread open the file without
> errors or warnings.
> 
> I had a look at where the error is coming from and it turns out it is in the
> check for linearized files in Linearization::Linearization. This function
> checks if the first object in the file after the header is a Linearization
> dictionary. The problem is this function has disabled the 'allow streams'
> flag. As a result when opening a PDF file where the first object is a stream
> results in the error due to trying to parse the stream as if it were not a
> stream.
Comment 4 Adrian Johnson 2013-01-03 05:58:56 UTC
Created attachment 72423 [details] [review]
make parser return error object if stream found when streams are disabled

The previous patch was incorrect. I change the wrong boolean parameter which disabled linearization.

This patch fixes the parser to ensure it does not try reading into a stream when allowStreams is false.
Comment 5 Adrian Johnson 2013-01-03 06:01:45 UTC
Created attachment 72424 [details] [review]
72423: make parser return error object if stream found when streams are disabled

remove obsolete patch
Comment 6 Albert Astals Cid 2013-01-04 10:09:06 UTC
Adrian, the patch works for me, question is we want it in 0.22 or only in master? looks like it should not cause any harm but otoh the warning is merely a cosmetic thing, but the fact that is a cosmetic thing is only known to us, so i'm undecided :D
Comment 7 Adrian Johnson 2013-01-04 10:30:06 UTC
(In reply to comment #6)
> Adrian, the patch works for me, question is we want it in 0.22 or only in
> master?

It would be good to get it in 0.22.
Comment 8 Albert Astals Cid 2013-01-04 10:41:59 UTC
ok, let's do it in 0.22 then (commit to 0.22 and merge to master plz)
Comment 9 Adrian Johnson 2013-01-04 10:59:57 UTC
Pushed


Use of freedesktop.org services, including Bugzilla, is subject to our Code of Conduct. How we collect and use information is described in our Privacy Policy.