[Pkg-tigervnc-devel] Bug#768369: Acknowledgement ([libjpeg62-turbo] [DOS] Stack smashing)

Fri Nov 7 17:35:10 UTC 2014

On Fri, Nov 7, 2014 at 6:26 PM, roucaries bastien
<roucaries.bastien+debian at gmail.com> wrote:
> On Fri, Nov 7, 2014 at 4:57 PM, DRC <dcommander at users.sourceforge.net> wrote:
>> Happy to fix it, but I need to be able to reproduce it first, using only
>> libjpeg-turbo.  Currently I cannot.  I tried running
>
> Here a backtrace, do you want to get some argument of the call function ?
> #0  0x00007ffff7067107 in __GI_raise (sig=sig at entry=6) at
> ../nptl/sysdeps/unix/sysv/linux/raise.c:56
> #1  0x00007ffff70684e8 in __GI_abort () at abort.c:89
> #2  0x00007ffff70a5044 in __libc_message (do_abort=do_abort at entry=2,
> fmt=fmt at entry=0x7ffff719568b "*** %s ***: %s terminated\n") at
> ../sysdeps/posix/libc_fatal.c:175
> #3  0x00007ffff7128137 in __GI___fortify_fail
> (msg=msg at entry=0x7ffff7195673 "stack smashing detected") at
> fortify_fail.c:31
> #4  0x00007ffff7128100 in __stack_chk_fail () at stack_chk_fail.c:28
> #5  0x00007ffff39d7553 in encode_mcu_huff (cinfo=0x7fffffff42e0,
> MCU_data=0x63a450) at jchuff.c:641
> #6  0x00007ffff39ca717 in compress_output (cinfo=0x7fffffff42e0,
> input_buf=<optimized out>) at jccoefct.c:381
> #7  0x00007ffff39ca006 in jpeg_finish_compress (cinfo=0x7fffffff42e0)
> at jcapimin.c:183
> #8  0x00007ffff3c222d0 in WriteJPEGImage (image_info=0x2c0c,
> image=0x2c0c) at ../../coders/jpeg.c:2810
> #9  0x00007ffff79aa1bc in WriteImage (image_info=0x60e530,
> image=0x626070) at ../../magick/constitute.c:1114
> #10 0x00007ffff79aa87a in WriteImages (image_info=<optimized out>,
> images=<optimized out>, filename=<optimized out>, exception=0x604e10)
> at ../../magick/constitute.c:1327
> #11 0x00007ffff763bc81 in ConvertImageCommand (image_info=0x4, argc=5,
> argv=0x604810, metadata=0xffffffffffffffff, exception=0x0) at
> ../../wand/convert.c:3215
> #12 0x00007ffff76a5ee7 in MagickCommandGenesis
> (image_info=image_info at entry=0x604f90, command=0x400810
> <ConvertImageCommand at plt>, argc=argc at entry=5,
> argv=argv at entry=0x7fffffffe118,
>     metadata=metadata at entry=0x0, exception=exception at entry=0x604e10)
> at ../../wand/mogrify.c:168
> #13 0x0000000000400887 in ConvertMain (argv=0x7fffffffe118, argc=5) at
> ../../utilities/convert.c:81
> #14 main (argc=5, argv=0x7fffffffe118) at ../../utilities/convert.c:92

Here more information note that valgrind does not detect anything.
Maybe a compiler bug?
(gdb) down
#4  0x00007ffff7128100 in __stack_chk_fail () at stack_chk_fail.c:28
28      stack_chk_fail.c: Aucun fichier ou dossier de ce type.
(gdb) up
#5  0x00007ffff39d7553 in encode_mcu_huff (cinfo=0x7fffffff42e0,
MCU_data=0x63a450) at jchuff.c:641
641     jchuff.c: Aucun fichier ou dossier de ce type.
(gdb) display *cinfo
6: *cinfo = {err = 0x7fffffff4150, mem = 0x621550, progress = 0x0,
client_data = 0x7fffffff4200, is_decompressor = 0, global_state = 101,
dest = 0x63a040, image_width = 1944,
  image_height = 2592, input_components = 3, in_color_space = JCS_RGB,
input_gamma = 1, data_precision = 8, num_components = 3,
jpeg_color_space = JCS_YCbCr, comp_info = 0x61c450,
  quant_tbl_ptrs = {0x61c810, 0x61c8a0, 0x0, 0x0}, dc_huff_tbl_ptrs =
{0x61c930, 0x61cb70, 0x0, 0x0}, ac_huff_tbl_ptrs = {0x61ca50,
0x61cc90, 0x0, 0x0},
  arith_dc_L = '\000' <repeats 15 times>, arith_dc_U = '\001' <repeats
16 times>, arith_ac_K = '\005' <repeats 16 times>, num_scans = 1,
scan_info = 0x0, raw_data_in = 0, arith_code = 0,
  optimize_coding = 1, CCIR601_sampling = 0, smoothing_factor = 0,
dct_method = JDCT_FLOAT, restart_interval = 0, restart_in_rows = 0,
write_JFIF_header = 1, JFIF_major_version = 1 '\001',
  JFIF_minor_version = 1 '\001', density_unit = 1 '\001', X_density =
72, Y_density = 72, write_Adobe_marker = 0, next_scanline = 2592,
progressive_mode = 0, max_h_samp_factor = 2,
  max_v_samp_factor = 2, total_iMCU_rows = 162, comps_in_scan = 3,
cur_comp_info = {0x61c450, 0x61c4b0, 0x61c510, 0x0}, MCUs_per_row =
122, MCU_rows_in_scan = 162, blocks_in_MCU = 6,
  MCU_membership = {0, 0, 0, 0, 1, 2, 0, 0, 0, 0}, Ss = 0, Se = 63, Ah
= 0, Al = 0, master = 0x63a080, main = 0x63a6d0, prep = 0x63a140, coef
= 0x63a430, marker = 0x63a840,
  cconvert = 0x63a0b0, downsample = 0x63a0d0, fdct = 0x63a1e0, entropy
= 0x63a370, script_space = 0x0, script_space_size = 0}
(gdb) display *MCU_data
7: *MCU_data = (JBLOCKROW) 0x7ffff3014130
(gdb) down
down           down-silently
(gdb) down
down           down-silently
(gdb) down
#4  0x00007ffff7128100 in __stack_chk_fail () at stack_chk_fail.c:28
28      stack_chk_fail.c: Aucun fichier ou dossier de ce type.
(gdb) up
#5  0x00007ffff39d7553 in encode_mcu_huff (cinfo=0x7fffffff42e0,
MCU_data=0x63a450) at jchuff.c:641
641     jchuff.c: Aucun fichier ou dossier de ce type.
(gdb) up
#6  0x00007ffff39ca717 in compress_output (cinfo=0x7fffffff42e0,
input_buf=<optimized out>) at jccoefct.c:381
381     jccoefct.c: Aucun fichier ou dossier de ce type.
(gdb) display *cinfo
8: *cinfo = {err = 0x7fffffff4150, mem = 0x621550, progress = 0x0,
client_data = 0x7fffffff4200, is_decompressor = 0, global_state = 101,
dest = 0x63a040, image_width = 1944,
  image_height = 2592, input_components = 3, in_color_space = JCS_RGB,
input_gamma = 1, data_precision = 8, num_components = 3,
jpeg_color_space = JCS_YCbCr, comp_info = 0x61c450,
  quant_tbl_ptrs = {0x61c810, 0x61c8a0, 0x0, 0x0}, dc_huff_tbl_ptrs =
{0x61c930, 0x61cb70, 0x0, 0x0}, ac_huff_tbl_ptrs = {0x61ca50,
0x61cc90, 0x0, 0x0},
  arith_dc_L = '\000' <repeats 15 times>, arith_dc_U = '\001' <repeats
16 times>, arith_ac_K = '\005' <repeats 16 times>, num_scans = 1,
scan_info = 0x0, raw_data_in = 0, arith_code = 0,
  optimize_coding = 1, CCIR601_sampling = 0, smoothing_factor = 0,
dct_method = JDCT_FLOAT, restart_interval = 0, restart_in_rows = 0,
write_JFIF_header = 1, JFIF_major_version = 1 '\001',
  JFIF_minor_version = 1 '\001', density_unit = 1 '\001', X_density =
72, Y_density = 72, write_Adobe_marker = 0, next_scanline = 2592,
progressive_mode = 0, max_h_samp_factor = 2,
  max_v_samp_factor = 2, total_iMCU_rows = 162, comps_in_scan = 3,
cur_comp_info = {0x61c450, 0x61c4b0, 0x61c510, 0x0}, MCUs_per_row =
122, MCU_rows_in_scan = 162, blocks_in_MCU = 6,
  MCU_membership = {0, 0, 0, 0, 1, 2, 0, 0, 0, 0}, Ss = 0, Se = 63, Ah
= 0, Al = 0, master = 0x63a080, main = 0x63a6d0, prep = 0x63a140, coef
= 0x63a430, marker = 0x63a840,
  cconvert = 0x63a0b0, downsample = 0x63a0d0, fdct = 0x63a1e0, entropy
= 0x63a370, script_space = 0x0, script_space_size = 0}
(gdb) display *input_buf
9: *input_buf = <error: value has been optimized out>
(gdb) down
#5  0x00007ffff39d7553 in encode_mcu_huff (cinfo=0x7fffffff42e0,
MCU_data=0x63a450) at jchuff.c:641
641     jchuff.c: Aucun fichier ou dossier de ce type.
(gdb) up
#6  0x00007ffff39ca717 in compress_output (cinfo=0x7fffffff42e0,
input_buf=<optimized out>) at jccoefct.c:381
381     jccoefct.c: Aucun fichier ou dossier de ce type.
(gdb) up
#7  0x00007ffff39ca006 in jpeg_finish_compress (cinfo=0x7fffffff42e0)
at jcapimin.c:183
183     jcapimin.c: Aucun fichier ou dossier de ce type.
(gdb) display *input_buf
No symbol "input_buf" in current context.
(gdb) display *cinfo
10: *cinfo = {err = 0x7fffffff4150, mem = 0x621550, progress = 0x0,
client_data = 0x7fffffff4200, is_decompressor = 0, global_state = 101,
dest = 0x63a040, image_width = 1944,
  image_height = 2592, input_components = 3, in_color_space = JCS_RGB,
input_gamma = 1, data_precision = 8, num_components = 3,
jpeg_color_space = JCS_YCbCr, comp_info = 0x61c450,
  quant_tbl_ptrs = {0x61c810, 0x61c8a0, 0x0, 0x0}, dc_huff_tbl_ptrs =
{0x61c930, 0x61cb70, 0x0, 0x0}, ac_huff_tbl_ptrs = {0x61ca50,
0x61cc90, 0x0, 0x0},
  arith_dc_L = '\000' <repeats 15 times>, arith_dc_U = '\001' <repeats
16 times>, arith_ac_K = '\005' <repeats 16 times>, num_scans = 1,
scan_info = 0x0, raw_data_in = 0, arith_code = 0,
  optimize_coding = 1, CCIR601_sampling = 0, smoothing_factor = 0,
dct_method = JDCT_FLOAT, restart_interval = 0, restart_in_rows = 0,
write_JFIF_header = 1, JFIF_major_version = 1 '\001',
  JFIF_minor_version = 1 '\001', density_unit = 1 '\001', X_density =
72, Y_density = 72, write_Adobe_marker = 0, next_scanline = 2592,
progressive_mode = 0, max_h_samp_factor = 2,
  max_v_samp_factor = 2, total_iMCU_rows = 162, comps_in_scan = 3,
cur_comp_info = {0x61c450, 0x61c4b0, 0x61c510, 0x0}, MCUs_per_row =
122, MCU_rows_in_scan = 162, blocks_in_MCU = 6,
  MCU_membership = {0, 0, 0, 0, 1, 2, 0, 0, 0, 0}, Ss = 0, Se = 63, Ah
= 0, Al = 0, master = 0x63a080, main = 0x63a6d0, prep = 0x63a140, coef
= 0x63a430, marker = 0x63a840,
  cconvert = 0x63a0b0, downsample = 0x63a0d0, fdct = 0x63a1e0, entropy
= 0x63a370, script_space = 0x0, script_space_size = 0}
(gdb) up
#8  0x00007ffff3c222d0 in WriteJPEGImage (image_info=0x2c0c,
image=0x2c0c) at ../../coders/jpeg.c:2810
2810    ../../coders/jpeg.c: Aucun fichier ou dossier de ce type.
(gdb)

>>   jpegtran -optimize -rotate 270 003632r270.jpg >out.jpg
>>
>> and
>>
>>   jpegtran -progressive -optimize -rotate 270 003632r270.jpg >out.jpg
>>
>> with valgrind, and no issues were detected.
>>
>> I also tried the convert command line listed above, and with my (admittedly
>> older) version of ImageMagick, no issues were detected. This leads me to
>> suspect an issue with ImageMagick, not libjpeg-turbo. Furthermore, Mozilla
>> bangs on the -optimize switch a tremendous amount, since that switch is
>> enabled by default in their mozjpeg encoder (mozjpeg is focused on getting
>> the absolute best compression ratio possible-- at the expense of like a 50x
>> drop in performance-- so they enable progressive & optimize by default, as
>> well as include other extensions like jpgcrush and trellis coding that
>> aren't in libjpeg-turbo.)  Furthermore, there is nothing about the optimized
>> (multi-pass) Huffman coding feature that is different between libjpeg-turbo
>> and libjpeg, so if this is genuinely a bug in libjpeg-turbo, it is likely to
>> exist in libjpeg as well.  Our optimizations affect only single-pass Huffman
>> coding.
>>