Bug 23581

Summary:

Stack overflow when rendering with a Picture that has an alpha_map cycle

Product:

xorg

Reporter:

Aaron Plattner <aplattner>

Component:

Server/General

Assignee:

Xorg Project Team <xorg-team>

Status:

RESOLVED FIXED

QA Contact:

Xorg Project Team <xorg-team>

Severity:

normal

Priority:

medium

Version:

git

Hardware:

Other

OS:

All

Whiteboard:

i915 platform:

i915 features:

Attachments:

Description	Flags
Test case	none
Do not recurse looking for an alpha map	none

Description Aaron Plattner 2009-08-28 12:58:48 UTC

Compile the attached program with "gcc -o render-alphamap-cycle render-alphamap-cycle.c -lX11 -lXrender" and then run it.  X crashes due to a stack overflow:

Program received signal SIGSEGV, Segmentation fault.
0xfffffd7fff257b5d in cleanfree () from /usr/lib/amd64/libc.so.1
(gdb) bt 20
#0  0xfffffd7fff257b5d in cleanfree () from /usr/lib/amd64/libc.so.1
#1  0xfffffd7fff256de1 in _malloc_unlocked () from /usr/lib/amd64/libc.so.1
#2  0xfffffd7fff256d06 in malloc () from /usr/lib/amd64/libc.so.1
#3  0xfffffd7ffe9402d4 in allocate_image () from /usr/lib/amd64/libpixman-1.so.0
#4  0xfffffd7ffe94081c in pixman_image_create_bits () from /usr/lib/amd64/libpixman-1.so.0
#5  0xfffffd7ffa421e3e in wfb_image_from_pict () from /usr/X11/lib/modules/amd64//libwfb.so
#6  0xfffffd7ffa42208a in wfb_image_from_pict () from /usr/X11/lib/modules/amd64//libwfb.so
#7  0xfffffd7ffa42208a in wfb_image_from_pict () from /usr/X11/lib/modules/amd64//libwfb.so
#8  0xfffffd7ffa42208a in wfb_image_from_pict () from /usr/X11/lib/modules/amd64//libwfb.so
#9  0xfffffd7ffa42208a in wfb_image_from_pict () from /usr/X11/lib/modules/amd64//libwfb.so
#10 0xfffffd7ffa42208a in wfb_image_from_pict () from /usr/X11/lib/modules/amd64//libwfb.so
#11 0xfffffd7ffa42208a in wfb_image_from_pict () from /usr/X11/lib/modules/amd64//libwfb.so
#12 0xfffffd7ffa42208a in wfb_image_from_pict () from /usr/X11/lib/modules/amd64//libwfb.so
#13 0xfffffd7ffa42208a in wfb_image_from_pict () from /usr/X11/lib/modules/amd64//libwfb.so
#14 0xfffffd7ffa42208a in wfb_image_from_pict () from /usr/X11/lib/modules/amd64//libwfb.so
#15 0xfffffd7ffa42208a in wfb_image_from_pict () from /usr/X11/lib/modules/amd64//libwfb.so
#16 0xfffffd7ffa42208a in wfb_image_from_pict () from /usr/X11/lib/modules/amd64//libwfb.so
#17 0xfffffd7ffa42208a in wfb_image_from_pict () from /usr/X11/lib/modules/amd64//libwfb.so
#18 0xfffffd7ffa42208a in wfb_image_from_pict () from /usr/X11/lib/modules/amd64//libwfb.so
#19 0xfffffd7ffa42208a in wfb_image_from_pict () from /usr/X11/lib/modules/amd64//libwfb.so
(More stack frames follow...)

Idea courtesy of Mark Vojkovich.

Comment 1 Aaron Plattner 2009-08-28 12:59:18 UTC

Created attachment 28987 [details]
Test case

Comment 2 Søren Sandmann Pedersen 2009-08-28 13:35:21 UTC

My immediate reaction is that it should be a BadMatch if you try to set a picture as its own alpha map, and also if you try to set an alpha map that itself has an alpha map already. Those two rules together seem like they would prevent cycles.

There is also the case where a picture and its alpha map point to the same underlying drawable, but I don't think that will cause other issues than unpredictable painting, at least in the software implementation.

Comment 3 Keith Packard 2010-06-05 19:52:45 UTC

Created attachment 36080 [details] [review]
Do not recurse looking for an alpha map

This patch makes the alpha map lookup non-recursive, stopping after the first alpha map.

Comment 4 Adam Jackson 2010-08-10 10:20:58 UTC

Fixed in 1.9.

Use of freedesktop.org services, including Bugzilla, is subject to our Code of Conduct. How we collect and use information is described in our Privacy Policy.