Commit graph

31659 commits

Author SHA1 Message Date
Michael Niedermayer
a3d331f2d8 Merge remote-tracking branch 'qatar/release/0.7' into release/0.8
* qatar/release/0.7: (96 commits)
  intfloat_readwrite: fix signed addition overflows
  smacker: validate channels and sample format.
  smacker: check buffer size before reading output size
  smacker: validate number of channels
  sipr: fix get_bits(0) calls
  motion_est: make MotionExtContext.map_generation unsigned
  4xm: prevent NULL dereference with invalid huffman table
  4xmdemux: prevent use of uninitialized memory
  4xm: clear FF_INPUT_BUFFER_PADDING_SIZE bytes in temporary buffers
  ptx: check for out of bound reads
  tiffdec: fix out of bound reads/writes
  eacmv: check for out of bound reads
  eacmv: fix potential pointer arithmetic overflows
  adpcm: fix out of bound reads due to integer overflow
  anm: prevent infinite loop
  avsdemux: check for out of bound writes
  avs: check for out of bound reads
  avsdemux: check for corrupted data
  mxfdec: Fix some buffer overreads caused by the misuse of AVPacket related functions.
  vaapi: Fix VC-1 decoding (reconstruct bitstream TTFRM correctly).
  ...

Conflicts:
	libavcodec/adpcm.c
	libavcodec/bink.c
	libavcodec/h264.c
	libavcodec/h264.h
	libavcodec/h264_cabac.c
	libavcodec/h264_cavlc.c
	libavcodec/motion_est_template.c
	libavcodec/mpegvideo.c
	libavcodec/nellymoserdec.c
	libavcodec/ptx.c
	libavcodec/svq3.c
	libavcodec/vaapi_vc1.c
	libavcodec/xan.c
	libavfilter/vf_scale.c
	libavformat/4xm.c
	libavformat/flvdec.c
	libavformat/mpeg.c
	tests/ref/fate/motionpixels

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2012-03-19 05:14:44 +01:00
Mans Rullgard
73ad066939 intfloat_readwrite: fix signed addition overflows
These additions might overflow the signed range for large
input values.  Converting to unsigned before the addition
rather than after avoids such undefined behaviour.  The
result under normal two's complement wraparound remains
unchanged.

Signed-off-by: Mans Rullgard <mans@mansr.com>
(cherry picked from commit 88d1e2b2b0)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:48 +01:00
Justin Ruggles
1cc0b08635 smacker: validate channels and sample format.
(cherry picked from commit ff1f89de2d)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:46 +01:00
Justin Ruggles
b3d7fffee3 smacker: check buffer size before reading output size
(cherry picked from commit cf044f8bff)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:43 +01:00
Justin Ruggles
ef7a4df458 smacker: validate number of channels
(cherry picked from commit e190e453bd)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:41 +01:00
Mans Rullgard
3b7a1ba90e sipr: fix get_bits(0) calls
Zero-length get_bits() is undefined, must check before calling.

Signed-off-by: Mans Rullgard <mans@mansr.com>
(cherry picked from commit c79d2a20ba)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:41 +01:00
Mans Rullgard
da73a2005a motion_est: make MotionExtContext.map_generation unsigned
The way this value is used, it should be an unsigned type.
While the numerical value has no meaning, unsigned wraparound
is relied upon.

Signed-off-by: Mans Rullgard <mans@mansr.com>
(cherry picked from commit cb668476ab)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:41 +01:00
Laurent Aimar
6b011631e9 4xm: prevent NULL dereference with invalid huffman table
Signed-off-by: Janne Grunau <janne-libav@jannau.net>
(cherry picked from commit 1b1182ce97)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:41 +01:00
Laurent Aimar
5ab326d7db 4xmdemux: prevent use of uninitialized memory
Signed-off-by: Janne Grunau <janne-libav@jannau.net>
(cherry picked from commit 79964745b3)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:40 +01:00
Laurent Aimar
39fd8d0083 4xm: clear FF_INPUT_BUFFER_PADDING_SIZE bytes in temporary buffers
Signed-off-by: Janne Grunau <janne-libav@jannau.net>
(cherry picked from commit 8d518a9c4f)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:40 +01:00
Laurent Aimar
b3bdefb01b ptx: check for out of bound reads
Signed-off-by: Janne Grunau <janne-libav@jannau.net>
(cherry picked from commit dc64f203a6)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:40 +01:00
Laurent Aimar
4eb51d96dd tiffdec: fix out of bound reads/writes
Signed-off-by: Janne Grunau <janne-libav@jannau.net>
(cherry picked from commit 04a845caa7)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:40 +01:00
Laurent Aimar
d75c80e942 eacmv: check for out of bound reads
Signed-off-by: Janne Grunau <janne-libav@jannau.net>
(cherry picked from commit 46cb2f6a29)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:40 +01:00
Laurent Aimar
34d6f22a57 eacmv: fix potential pointer arithmetic overflows
Signed-off-by: Janne Grunau <janne-libav@jannau.net>
(cherry picked from commit 8df8a87e3f)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:40 +01:00
Laurent Aimar
518c72474d adpcm: fix out of bound reads due to integer overflow
Signed-off-by: Janne Grunau <janne-libav@jannau.net>
(cherry picked from commit c7f89064e2)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:40 +01:00
Laurent Aimar
39fed2e95b anm: prevent infinite loop
Signed-off-by: Janne Grunau <janne-libav@jannau.net>
(cherry picked from commit 2475f1a83c)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:40 +01:00
Laurent Aimar
7fa13e12e6 avsdemux: check for out of bound writes
Signed-off-by: Janne Grunau <janne-libav@jannau.net>
(cherry picked from commit 6de33611c9)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:40 +01:00
Laurent Aimar
ab201f6f1b avs: check for out of bound reads
Signed-off-by: Janne Grunau <janne-libav@jannau.net>
(cherry picked from commit de049a95f4)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:40 +01:00
Laurent Aimar
b696d61518 avsdemux: check for corrupted data
Signed-off-by: Janne Grunau <janne-libav@jannau.net>
(cherry picked from commit 76c6971a64)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:40 +01:00
Alex Converse
a23bcc923d mxfdec: Fix some buffer overreads caused by the misuse of AVPacket related functions.
(cherry picked from commit 0c46e958d1)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:36 +01:00
Gwenole Beauchesne
559261ce49 vaapi: Fix VC-1 decoding (reconstruct bitstream TTFRM correctly).
Signed-off-by: Diego Biurrun <diego@biurrun.de>
(cherry picked from commit 53efb758c0)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:36 +01:00
Mans Rullgard
f9d17e6f54 4xm: fix signed overflow
Signed-off-by: Mans Rullgard <mans@mansr.com>
(cherry picked from commit 84dda40762)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:36 +01:00
Mans Rullgard
0b1ac7bf4f wmavoice: fix a signed overflow
Signed-off-by: Mans Rullgard <mans@mansr.com>
(cherry picked from commit ba3f07d061)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:36 +01:00
Mans Rullgard
af0a56e6ef mpegvideo_enc: fix a signed overflow
Signed-off-by: Mans Rullgard <mans@mansr.com>
(cherry picked from commit 05795f35be)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:36 +01:00
Mans Rullgard
5e3ba60e6f crc: fix signed overflow
This fixes a signed overflow from i << 24 when i == 255 by
making i unsigned.  The result of the shift is already
assigned to an variable of unsigned type.

Signed-off-by: Mans Rullgard <mans@mansr.com>
(cherry picked from commit 8b19ae0761)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:35 +01:00
Mans Rullgard
48f9a80072 mpeg12enc: use sign_extend() function
Signed-off-by: Mans Rullgard <mans@mansr.com>
(cherry picked from commit 2f329db90e)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:35 +01:00
Mans Rullgard
2c99aa48d7 lavf: fix signed overflow in avformat_find_stream_info()
On the first iteration through this code, last_dts is always
INT64_MIN (AV_NOPTS_VALUE) and the subtraction overflows in
an invalid manner.  Although the result is only used if the
input values are valid, performing the subtraction is still
not allowed in a strict environment.

Signed-off-by: Mans Rullgard <mans@mansr.com>
(cherry picked from commit a31e9f68a4)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:35 +01:00
Mans Rullgard
fdc669fcbb vp8: fix signed overflows
In addition to avoiding undefined behaviour, an unsigned type
makes more sense for packing multiple 8-bit values.

Signed-off-by: Mans Rullgard <mans@mansr.com>
(cherry picked from commit bb59156606)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:35 +01:00
Mans Rullgard
fe3314a413 motion_est: fix some signed overflows
Signed-off-by: Mans Rullgard <mans@mansr.com>
(cherry picked from commit e708afd3c0)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:35 +01:00
Mans Rullgard
58afe6061a dca: fix signed overflow in shift
Signed-off-by: Mans Rullgard <mans@mansr.com>
(cherry picked from commit 559c244d42)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:35 +01:00
Mans Rullgard
8c2ae575ad aacdec: fix undefined shifts
Since nnz can be zero, this is needed to avoid a shift by 32.

Signed-off-by: Mans Rullgard <mans@mansr.com>
(cherry picked from commit d12294304a)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:35 +01:00
Laurent Aimar
9c78fe9360 bink: Check for various out of bound writes
Signed-off-by: Janne Grunau <janne-libav@jannau.net>
(cherry picked from commit a00676e48e)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:35 +01:00
Laurent Aimar
c98d7882d8 bink: Check for out of bound writes when building tree
Signed-off-by: Janne Grunau <janne-libav@jannau.net>
(cherry picked from commit 24adf7832b)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:35 +01:00
Mans Rullgard
e52e85ac3a put_bits: fix invalid shift by 32 in flush_put_bits()
If flush_put_bits() is called when the 32-bit buffer is empty,
e.g. after writing a multiple of 32 bits, and invalid shift by
32 is performed.  Since flush_put_bits() is called infrequently,
this additional check should have negligible performance impact.

Signed-off-by: Mans Rullgard <mans@mansr.com>
(cherry picked from commit ac6eab1496)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:35 +01:00
Alex Converse
4faa00b256 mpegps: Use av_get_packet() instead of poorly emulating it.
(cherry picked from commit 98ef887a75)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:31 +01:00
Janne Grunau
90d7146511 motionpixels: decode only the 111 complete frames for fate
Signed-off-by: Janne Grunau <janne-libav@jannau.net>
(cherry picked from commit c2f2dfb3dd)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:31 +01:00
Laurent Aimar
59050c0629 mpc8: Check out of bound bands limit
Signed-off-by: Janne Grunau <janne-libav@jannau.net>
(cherry picked from commit 9bd854b1ff)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:31 +01:00
Laurent Aimar
be2404b06d xan: Prevent NULL dereference with missing palette
Signed-off-by: Janne Grunau <janne-libav@jannau.net>
(cherry picked from commit 7d17a794f0)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:31 +01:00
Laurent Aimar
49007b494e xan: Check for out of bound reads in xan_huffman_decode()
Signed-off-by: Janne Grunau <janne-libav@jannau.net>
(cherry picked from commit 3db3fdf4c6)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:31 +01:00
Laurent Aimar
0277c82de2 xan: Fixed out of bound accesses in xan_unpack()
Signed-off-by: Janne Grunau <janne-libav@jannau.net>
(cherry picked from commit 3e0757c2a8)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:31 +01:00
Laurent Aimar
5fa8e43b54 motionpixels: Prevent calling init_vlc() with invalid parameters
Signed-off-by: Janne Grunau <janne-libav@jannau.net>
(cherry picked from commit 1cd0a55163)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:31 +01:00
Laurent Aimar
737bea21b6 shorten: Fix out of bound writes in fix_bitshift()
The data pointers s->decoded[*] already take into account s->nwrap.

Signed-off-by: Janne Grunau <janne-libav@jannau.net>
(cherry picked from commit 5f05cf4ea9)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:31 +01:00
Laurent Aimar
aa9e308580 dsicinav: Check for out of bounds writes
Signed-off-by: Janne Grunau <janne-libav@jannau.net>
(cherry picked from commit 1720603287)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:31 +01:00
Laurent Aimar
d57d039e04 tiertexseqv: Check for out of bound reads
Signed-off-by: Janne Grunau <janne-libav@jannau.net>
(cherry picked from commit 64263dd526)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:31 +01:00
Laurent Aimar
97a1ab4bce quickdraw: Check for out of bound reads
Signed-off-by: Janne Grunau <janne-libav@jannau.net>
(cherry picked from commit 4fd56f842c)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:31 +01:00
Laurent Aimar
914b9b0b2b dsicinav: Check for out of bounds reads
Signed-off-by: Janne Grunau <janne-libav@jannau.net>
(cherry picked from commit e3ca9b93d9)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:31 +01:00
Laurent Aimar
39de0e008d motionpixels: Fix the size of workspace buffers
Some buffers must be mod 4 in width and/or height.

Signed-off-by: Janne Grunau <janne-libav@jannau.net>
(cherry picked from commit 210c80331e)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:31 +01:00
Laurent Aimar
f2f2a00d39 motionpixels: Clear FF_INPUT_BUFFER_PADDING_SIZE bytes at the end of the temporary buffer
Signed-off-by: Janne Grunau <janne-libav@jannau.net>
(cherry picked from commit d337dd3a90)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:31 +01:00
Laurent Aimar
905d0633a6 wmavoice: Check for corrupted extra data
Signed-off-by: Janne Grunau <janne-libav@jannau.net>
(cherry picked from commit d99427cb8b)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:31 +01:00
Laurent Aimar
95605595b5 wmavoice: Check for out of bound writes
Signed-off-by: Janne Grunau <janne-libav@jannau.net>
(cherry picked from commit 1c1449b548)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2012-03-18 17:50:31 +01:00