This vulnerability has been reported privately. Albert Astals Cid already fixed it in the following commit:
https://cgit.freedesktop.org/poppler/poppler/commit/?id=a3a98a6d83dfbf49f565f5aa2d7c07153a7f62fc

The CVE-2017-9775 has been assigned to this vulnerability.

The vulnerability is due of an integer overflow in the file JBIG2Stream.cc:895 at the function JBIG2Bitmap::combine:
894  for (yy = y0; yy < y1; ++yy) {
895    if (unlikely(y + yy) >= h)
896      continue;
897
898    // one byte per line -- need to mask both left and right side
899    if (oneByte) {
900      if (x >= 0) {
901        destPtr = data + (y + yy) * line + (x >> 3);
902        srcPtr = bitmap->data + yy * bitmap->line;
903        dest = *destPtr;
904        src1 = *srcPtr;

The user controls the variables y, line and x. During the addition y + yy an integer overflow could happens if the user uses a very high values for y , for example 0x7ffffff.

The code is checking for overflows with the code at the beggining of the loop:
895    if (unlikely(y + yy) >= h)
896      continue;

But when the integer overflow happens this check is bypassed and it's possible to overflow the data variable in line 901. The variable data is in the heap.

The vulnerability leads to a heap overflow vulnerability. The different samples corrupt the heap in different ways. Some of then finish in a double free, other in an error calling malloc and free.

The file JBIG2Stream.cc_903_smallest_possible.pdf contains the minimum data to trigger the bug.

This bug could be used by an attacker to control the heap.

This vulnerability has been found by Offensive Research at Salesforce.com:
Alberto Garcia (@algillera), Francisco Oca (@francisco_oca) & Suleman Ali (@Salbei_)