Bug 68438

I tried pdftocairo from 0.20.4, got the same crash. But it works if I disable the usage of libopenjpeg2 (in either version!). openjpeg bug?

I tried it now with the latest release of family 1 (1.5.1) of openjpeg. (With openjpeg 2.0.0 I wasn't able to compile, seems as if the interface completely changed.)
Good news: it didn't crash anymore. Bad news: I got a lot of errors and warnings, and at least one of the image wasn't correctly rendered. But the reason for this can't be the PDF: if I compile without openjpeg usage, I get no errors and the output is okay.

I've no idea how we should continue with such cases...

FWIW, here the console output:

Syntax Warning: Mismatch between font type and embedded font file
Syntax Warning: SOT marker inconsistency in tile 0: tile-part index greater (5) than number of tile-parts (5)<0a>
Syntax Warning: SOT marker inconsistency in tile 1: tile-part index greater (5) than number of tile-parts (5)<0a>
Syntax Warning: SOT marker inconsistency in tile 2: tile-part index greater (5) than number of tile-parts (5)<0a>
Syntax Warning: SOT marker inconsistency in tile 3: tile-part index greater (5) than number of tile-parts (5)<0a>
Syntax Warning: SOT marker inconsistency in tile 4: tile-part index greater (5) than number of tile-parts (5)<0a>
Syntax Warning: SOT marker inconsistency in tile 5: tile-part index greater (5) than number of tile-parts (5)<0a>
Syntax Warning: SOT marker inconsistency in tile 6: tile-part index greater (5) than number of tile-parts (5)<0a>
Syntax Warning: SOT marker inconsistency in tile 7: tile-part index greater (5) than number of tile-parts (5)<0a>
Syntax Warning: SOT marker inconsistency in tile 0: tile-part index greater (5) than number of tile-parts (5)<0a>
Syntax Warning: SOT marker inconsistency in tile 1: tile-part index greater (5) than number of tile-parts (5)<0a>
Syntax Warning: SOT marker inconsistency in tile 2: tile-part index greater (5) than number of tile-parts (5)<0a>
Syntax Warning: SOT marker inconsistency in tile 3: tile-part index greater (5) than number of tile-parts (5)<0a>
Syntax Warning: SOT marker inconsistency in tile 4: tile-part index greater (5) than number of tile-parts (5)<0a>
Syntax Warning: SOT marker inconsistency in tile 5: tile-part index greater (5) than number of tile-parts (5)<0a>
Syntax Warning: SOT marker inconsistency in tile 6: tile-part index greater (5) than number of tile-parts (5)<0a>
Syntax Warning: SOT marker inconsistency in tile 7: tile-part index greater (5) than number of tile-parts (5)<0a>
Syntax Warning: SOT marker inconsistency in tile 0: tile-part index greater (5) than number of tile-parts (5)<0a>
Syntax Warning: SOT marker inconsistency in tile 1: tile-part index greater (5) than number of tile-parts (5)<0a>
Syntax Warning: SOT marker inconsistency in tile 2: tile-part index greater (5) than number of tile-parts (5)<0a>
Syntax Warning: SOT marker inconsistency in tile 3: tile-part index greater (5) than number of tile-parts (5)<0a>
Syntax Warning: SOT marker inconsistency in tile 4: tile-part index greater (5) than number of tile-parts (5)<0a>
Syntax Warning: SOT marker inconsistency in tile 5: tile-part index greater (5) than number of tile-parts (5)<0a>
Syntax Warning: SOT marker inconsistency in tile 6: tile-part index greater (5) than number of tile-parts (5)<0a>
Syntax Warning: SOT marker inconsistency in tile 7: tile-part index greater (5) than number of tile-parts (5)<0a>
Syntax Warning: SOT marker inconsistency in tile 0: tile-part index greater (5) than number of tile-parts (5)<0a>
Syntax Warning: SOT marker inconsistency in tile 1: tile-part index greater (5) than number of tile-parts (5)<0a>
Syntax Warning: SOT marker inconsistency in tile 2: tile-part index greater (5) than number of tile-parts (5)<0a>
Syntax Warning: SOT marker inconsistency in tile 3: tile-part index greater (5) than number of tile-parts (5)<0a>
Syntax Warning: SOT marker inconsistency in tile 4: tile-part index greater (5) than number of tile-parts (5)<0a>
Syntax Warning: SOT marker inconsistency in tile 5: tile-part index greater (5) than number of tile-parts (5)<0a>
Syntax Warning: SOT marker inconsistency in tile 6: tile-part index greater (5) than number of tile-parts (5)<0a>
Syntax Warning: SOT marker inconsistency in tile 7: tile-part index greater (5) than number of tile-parts (5)<0a>
Syntax Warning: SOT marker inconsistency in tile 0: tile-part index greater (5) than number of tile-parts (5)<0a>
Syntax Warning: SOT marker inconsistency in tile 1: tile-part index greater (5) than number of tile-parts (5)<0a>
Syntax Warning: SOT marker inconsistency in tile 2: tile-part index greater (5) than number of tile-parts (5)<0a>
Syntax Warning: SOT marker inconsistency in tile 3: tile-part index greater (5) than number of tile-parts (5)<0a>
Syntax Warning: SOT marker inconsistency in tile 4: tile-part index greater (5) than number of tile-parts (5)<0a>
Syntax Warning: SOT marker inconsistency in tile 5: tile-part index greater (5) than number of tile-parts (5)<0a>
Syntax Warning: SOT marker inconsistency in tile 6: tile-part index greater (5) than number of tile-parts (5)<0a>
Syntax Warning: SOT marker inconsistency in tile 7: tile-part index greater (5) than number of tile-parts (5)<0a>
Syntax Warning: SOT marker inconsistency in tile 0: tile-part index greater (5) than number of tile-parts (5)<0a>
Syntax Warning: SOT marker inconsistency in tile 1: tile-part index greater (5) than number of tile-parts (5)<0a>
Syntax Warning: SOT marker inconsistency in tile 2: tile-part index greater (5) than number of tile-parts (5)<0a>
Syntax Warning: SOT marker inconsistency in tile 3: tile-part index greater (5) than number of tile-parts (5)<0a>
Syntax Warning: SOT marker inconsistency in tile 4: tile-part index greater (5) than number of tile-parts (5)<0a>
Syntax Warning: SOT marker inconsistency in tile 5: tile-part index greater (5) than number of tile-parts (5)<0a>
Syntax Warning: SOT marker inconsistency in tile 6: tile-part index greater (5) than number of tile-parts (5)<0a>
Syntax Warning: SOT marker inconsistency in tile 7: tile-part index greater (5) than number of tile-parts (5)<0a>
Syntax Error: Error with COLR Box<0a>
Syntax Error: Failed to decode jp2 structure<0a>
Syntax Warning: Did no succeed opening JPX Stream as JP2, trying as J2K.
Syntax Error: 00000000: expected a marker instead of 0<0a>
Syntax Warning: Did no succeed opening JPX Stream as J2K, trying as JPT.
Syntax Error: Forbidden value encounter in message header !!<0a>
Syntax Error: [JPT-stream] : Expecting Main header first [class_Id 0] !<0a>
Syntax Error: Did no succeed opening JPX Stream.
Syntax Error: Error with COLR Box<0a>
Syntax Error: Failed to decode jp2 structure<0a>
Syntax Warning: Did no succeed opening JPX Stream as JP2, trying as J2K.
Syntax Error: 00000000: expected a marker instead of 0<0a>
Syntax Warning: Did no succeed opening JPX Stream as J2K, trying as JPT.
Syntax Error: Forbidden value encounter in message header !!<0a>
Syntax Error: [JPT-stream] : Expecting Main header first [class_Id 0] !<0a>
Syntax Error: Did no succeed opening JPX Stream.
Syntax Warning: SOT marker inconsistency in tile 0: tile-part index greater (5) than number of tile-parts (5)<0a>
Syntax Warning: SOT marker inconsistency in tile 1: tile-part index greater (5) than number of tile-parts (5)<0a>
Syntax Warning: SOT marker inconsistency in tile 2: tile-part index greater (5) than number of tile-parts (5)<0a>
Syntax Warning: SOT marker inconsistency in tile 3: tile-part index greater (5) than number of tile-parts (5)<0a>
Syntax Warning: SOT marker inconsistency in tile 4: tile-part index greater (5) than number of tile-parts (5)<0a>
Syntax Warning: SOT marker inconsistency in tile 5: tile-part index greater (5) than number of tile-parts (5)<0a>
Syntax Warning: SOT marker inconsistency in tile 6: tile-part index greater (5) than number of tile-parts (5)<0a>
Syntax Warning: SOT marker inconsistency in tile 7: tile-part index greater (5) than number of tile-parts (5)<0a>
Syntax Warning: SOT marker inconsistency in tile 0: tile-part index greater (5) than number of tile-parts (5)<0a>
Syntax Warning: SOT marker inconsistency in tile 1: tile-part index greater (5) than number of tile-parts (5)<0a>
Syntax Warning: SOT marker inconsistency in tile 2: tile-part index greater (5) than number of tile-parts (5)<0a>
Syntax Warning: SOT marker inconsistency in tile 3: tile-part index greater (5) than number of tile-parts (5)<0a>
Syntax Warning: SOT marker inconsistency in tile 4: tile-part index greater (5) than number of tile-parts (5)<0a>
Syntax Warning: SOT marker inconsistency in tile 5: tile-part index greater (5) than number of tile-parts (5)<0a>
Syntax Warning: SOT marker inconsistency in tile 6: tile-part index greater (5) than number of tile-parts (5)<0a>
Syntax Warning: SOT marker inconsistency in tile 7: tile-part index greater (5) than number of tile-parts (5)<0a>
Syntax Error (32518662): insufficient arguments for Marked Content

Yeah openjpeg is a bit of a hit and miss with releases, we have http://code.google.com/p/openjpeg/issues/detail?id=211 and a few more issues reported https://bugs.freedesktop.org/show_bug.cgi?id=58906 sadly they seem to be a bit slow doing releases.

Can you check if 1.5.0 release works? And if you have time if the "head of 1.5 (no clue in which branch that's stored)" works?

TBH I wouldn't say this is "our" bug (other than not complaining if we find you're using a "bad" openjpeg release or using openjpeg instead of something else to decode JPEG2000 (which tbh i think openjpeg is the best there is around))

(In reply to comment #3)
> Yeah openjpeg is a bit of a hit and miss with releases, we have
> http://code.google.com/p/openjpeg/issues/detail?id=211 and a few more issues
> reported https://bugs.freedesktop.org/show_bug.cgi?id=58906 sadly they seem
> to be a bit slow doing releases.

Thanks for the hints. I'll have a look if the patches in openjpeg they pointed to are already part of 1.5.1 or at least the 1.5 branch, and also try to rebase Adrians patch to have a look at the PDF with openjpeg 2.0.

> 
> Can you check if 1.5.0 release works? And if you have time if the "head of
> 1.5 (no clue in which branch that's stored)" works?

Hmmh, I'd already a look at it with 1.5.1, as I already mentioned in comment 2. So what do You expect with going back to 1.5.0?

> 
> TBH I wouldn't say this is "our" bug (other than not complaining if we find
> you're using a "bad" openjpeg release or using openjpeg instead of something
> else to decode JPEG2000 (which tbh i think openjpeg is the best there is
> around))

After I made the ahead mentioned tests, I'll report it here. If You don't care then wait until that, and then close it. Let us see if I open a bug report at openjpeg.

I finished now my test series:

1. actual ubuntu openjpeg library (1.3.x):
crashes in opj_create_decompress(CODEC_J2K)
2. actual openjpeg 1.5.1, available over http://code.google.com/p/openjpeg/downloads/list
doesn't display 2 of the JPEG2000 images, gives warnings for each other, but doesn't crash
3. actual openjpeg 1.5 branch, available over  svn checkout http://openjpeg.googlecode.com/svn/branches/openjpeg-1.5 (1.5.2 beta)
doesn't display 2 of the JPEG2000 images, gives warnings for each other, but doesn't crash
4. actual openjpeg 2.0.0, available over http://code.google.com/p/openjpeg/downloads/list, together with slight modified patch from Adrian of bug 58906 (changes just to get it compiled with actual master)
countless error messages, non of the JPEG2000 images are shown, but no crash
5. actual  openjpeg master,  available svn checkout http://openjpeg.googlecode.com/svn/trunk/ (2.0.1 beta?), together with slight modified patch from Adrian
stops with assert: xpdfmerge/openjpeg-read-only/src/lib/openjp2/cio.c:555: opj_stream_get_number_byte_left: Assertion `p_stream->m_user_data_length >= (OPJ_UINT64)p_stream->m_byte_offset' failed.

I'm not deep enough in JPEG2000 to be really sure, if it's our integration of openjpeg, if it's an openjpeg bug or if it's a problem of Adobe Indesign CS4 which created the PDF.
But since altona says in the comments it is amongst others a JPEG2000 test case, the behaviour changes with nearly every version of openjpeg, Adobe Acrobat Pro opens it without any problems or erreor messages and last but not least, poppler can render it internally (with openjpeg) without any problems, I would guess it is an openjpeg bug or openjpeg is benevolent enough.

I for myself decide now to disable openjpeg and continue working with the internal code.

FWIW: The two images, where openjpeg 1.3.x crashes and with openjpeg 1.5.x cannot be decompressed, are JPEG2000 images in LAB colorspace, so it seems, that openjpeg is not able to handle extended JPEG2000 images.

FWIW, mupdf can render the PDF (beside the two LAB images) with openpdf 2.x, BUT:
1. it calls opj_stream_set_user_data_length with the stream length to avoid the assert in 2.0.1 beta(?)
2. it modifies the code in j2k.c and uses this modified code.
My conclusion: openjpeg 2.x can't be used PDF JPXStream, at least not without any further changes!

Well, let's report the bugs to openjpeg then, that's our only possibility. Unless you want to take care of improving the shitty and feature-less code we have insource :D

FYI, I was now able to modify Adrian's openjpeg 2.0.0 integration in bug 58906 so that also this PDF works with openjpeg 2.0.0 (and also with master version of openjpeg available over svn) WITHOUT any modification of openjpeg code. The only images which still cannot be rendered are the LAB images.

So I'll upload a modified patch to bug 58906 next week (I'll start to a short private trip tomorrow morning) and then also report the bug to openjpeg.

The problem with LAB images is now reported to openjpeg, http://code.google.com/p/openjpeg/issues/detail?id=234