Bug 17589

Summary: dsjpeg Huffman table parser validation error.
Product: swfdec Reporter: M Joonas Pihlaja <jpihlaja>
Component: libraryAssignee: swfdec ml <swfdec>
Status: RESOLVED FIXED QA Contact: swfdec ml <swfdec>
Severity: critical    
Priority: medium    
Version: unspecified   
Hardware: All   
OS: All   
Whiteboard:
i915 platform: i915 features:
Attachments: trigger a buffer overflow in the DHT marker handler.
don't overflow huffmantable entries array
missing free for error_message

Description M Joonas Pihlaja 2008-09-15 09:00:06 UTC
Created attachment 18885 [details]
trigger a buffer overflow in the DHT marker handler.

dsjpeg can be tricked into overflowing its internal Huffman table arrays.  Valgrind says of the attached test case:

==31295== Copyright (C) 2004-2006, and GNU GPL'd, by OpenWorks LLP.
==31295== Using valgrind-3.2.1-Debian, a dynamic binary instrumentation framework.
==31295== Copyright (C) 2000-2006, and GNU GPL'd, by Julian Seward et al.
==31295== For more details, rerun with: -v
==31295==
==31295== Invalid write of size 1
==31295==    at 0x40382C: huffman_table_add (jpeg_huffman.c:48)
==31295==    by 0x401B59: huffman_table_init_jpeg (jpeg.c:273)
==31295==    by 0x402B8C: jpeg_decoder_define_huffman_tables (jpeg.c:751)
==31295==    by 0x4028D9: jpeg_decoder_decode (jpeg.c:672)
==31295==    by 0x403C24: jpeg_decode_argb (jpeg_rgb_decoder.c:58)
==31295==    by 0x400DB0: main (load.c:46)
==31295==  Address 0x537B434 is 12 bytes after a block of size 43,984 alloc'd
==31295==    at 0x4A1B858: malloc (vg_replace_malloc.c:149)
==31295==    by 0x40245F: jpeg_decoder_new (jpeg.c:535)
==31295==    by 0x403C07: jpeg_decode_argb (jpeg_rgb_decoder.c:55)
==31295==    by 0x400DB0: main (load.c:46)
==31295==
==31295== Invalid write of size 4
==31295==    at 0x403844: huffman_table_add (jpeg_huffman.c:49)
==31295==    by 0x401B59: huffman_table_init_jpeg (jpeg.c:273)
[snip]

When run without valgrind this test case causes glibc to abort on x86-64:

*** glibc detected *** free(): invalid pointer: 0x0000000000512f40 ***
error: decoder error: bad huffsize[] arrayAborted

On x86-32 the test causes dsjpeg to error out with a message "bad huffsize[] array" seemingly intact, but note that the bug isn't 64 bit specific.
Comment 1 Riccardo Magliocchetti 2008-09-16 00:41:56 UTC
Adjusted the severity. I think the fastest we switch to ijg jpeg the better.
Comment 2 Riccardo Magliocchetti 2008-09-21 09:20:35 UTC
Created attachment 19067 [details] [review]
don't overflow huffmantable entries array

This fix the issue for me, thanks Joonas for sharing the fun ;)
Comment 3 Riccardo Magliocchetti 2008-09-21 10:28:57 UTC
Created attachment 19069 [details] [review]
missing free for error_message

Noticed while testing the patch, the buffer used for reporting the error message was never freed.
Comment 4 Benjamin Otte 2008-09-24 09:39:48 UTC
fixed in git master (and soon) 0.8

Use of freedesktop.org services, including Bugzilla, is subject to our Code of Conduct. How we collect and use information is described in our Privacy Policy.