Commit graph

15211 commits

Author SHA1 Message Date
Ronald S. Bultje
8f881885c2 xxan: don't read before start of buffer in av_memcpy_backptr().
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit f1279e286b)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-14 20:47:19 +01:00
Ronald S. Bultje
26521d87ba dsicinvideo: validate buffer offset before copying pixels.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit c95fefa042)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-14 20:47:19 +01:00
Ronald S. Bultje
e1a4143793 cook: error out on quant_index values outside [-63, 63] range.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 97e48b2f54)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-14 20:47:19 +01:00
Ronald S. Bultje
b9482a6efd cook: extend channel uncoupling tables so the full bit range is covered.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 37cc8600d0)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-14 20:47:19 +01:00
Ronald S. Bultje
88c3cc019c cook: expand dither_tab[], and make sure indexes into it don't overflow.
Fixes overflows in accessing dither_tab[].

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 442c3a8cb1)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-13 23:40:29 +01:00
Ronald S. Bultje
9980e4df3b huffyuv: add padding to classic (v1) huffman tables.
We slightly overread the input buffer, so we require
padding at the end of the buffer, as is documented in the
get_bits API. Without padding, we'll read uninitialized
data or beyond the end of the .rodata, which may crash.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 4ffe5e2aa5)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-13 23:36:39 +01:00
Ronald S. Bultje
d4f2786cda avs: fix infinite loop on end-of-stream.
The codec would keep returning the last decoded frame if the stream
contains B-frames, since it wouldn't clear that frame from the list of
frames to be returned to the user.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 83f15a1228)

Conflicts:

	libavcodec/cavsdec.c

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-13 23:36:02 +01:00
Alex Converse
2744fdbd9e tiffdec: Prevent illegal memory access caused by recycled pointers.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit fd0be63049)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-13 23:30:55 +01:00
Ronald S. Bultje
1fcc2c6091 wma: fix off-by-one in array bounds check.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit b4bccf3e4e)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-13 23:30:39 +01:00
Ronald S. Bultje
74871ac70a dv: check buffer size before reading profile.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit e97efecec8)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-13 23:30:21 +01:00
Ronald S. Bultje
9cb7f6e54a raw: move buffer size check up.
This way, it protects against overreads for 4bpp/2bpp content also.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit cc5dd632ce)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-13 23:23:58 +01:00
Ronald S. Bultje
ed6aaf579d dca: prevent accessing static arrays with invalid indexes.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit e6ffd997cb)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-13 23:22:32 +01:00
Ronald S. Bultje
e1b4614ab4 lpcm: fix sample size calculation for 20bit LCPM.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit f1320dc3be)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-13 23:12:00 +01:00
Ronald S. Bultje
12247a13e0 Don't use ff_cropTbl[] for IDCT.
Results of IDCT can by far outreach the range of ff_cropTbl[], leading
to overreads and potentially crashes.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit c23acbaed4)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-08 22:07:55 +01:00
Ronald S. Bultje
9def2f200e error_resilience: initialize s->block_index[].
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 6193ff6854)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-08 22:07:55 +01:00
Ronald S. Bultje
7b676935ee svq3: protect against negative quantizers.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 11b940a1a8)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-08 22:07:55 +01:00
Justin Ruggles
4a15240a27 mov: set channel layout for AC-3 streams based on the 'dac3' atom info
fixes Bug 225
(cherry picked from commit 3798205a77)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-08 22:07:54 +01:00
Janne Grunau
a47b96bdd3 rv34: handle size changes during frame multithreading
Factors all context dynamic memory handling to its own functions.
Fixes bug 220.
(cherry picked from commit 2bd730010d)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-08 22:07:54 +01:00
Alex Converse
48ac765efe rv10/20: Fix slice overflow with checked bitstream reader.
(cherry picked from commit 9243ec4a50)
2012-03-06 15:31:23 -08:00
Michael Niedermayer
522645e38f h263dec: Disallow width/height changing with frame threads.
Fixes CVE-2011-3937

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 71db86d53b)

Conflicts:

	libavcodec/h263dec.c

Signed-off-by: Alex Converse <alex.converse@gmail.com>
2012-03-06 15:28:01 -08:00
Alex Converse
e891ee4bf6 adpcm: Clip step_index values read from the bitstream at the beginning of each frame.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
(cherry picked from commit bbeb29133b)
2012-03-06 15:28:01 -08:00
Alex Converse
ef673211e7 tiff: Make the TIFF_LONG and TIFF_SHORT types unsigned.
TIFF v6.0 (unimplemented) adds signed equivalents.
(cherry picked from commit e32548d133)
2012-03-06 15:28:01 -08:00
Alex Converse
eaeaeb265f dpcm: ignore extra unpaired bytes in stereo streams.
Fixes: CVE-2011-3951

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
(cherry picked from commit ce7aee9b73)
2012-03-06 15:28:01 -08:00
Alex Converse
db315c796d svq3: Prevent illegal reads while parsing extradata.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
(cherry picked from commit 9e1db721c4)
2012-03-06 15:28:01 -08:00
Michael Niedermayer
e3743869e9 ac3dec: Move center and surround mix level tables to the parser.
That way all mix levels as exported by avpriv_ac3_parse_header()
will have the same meaning.

Previously the 3-bit center mix level for E-AC-3 was used to index in a
4-entry table, leading to out-of-array reads.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Justin Ruggles <justin.ruggles@gmail.com>
Signed-off-by: Alex Converse <alex.converse@gmail.com>
(cherry picked from commit e6d9fa66f1)
2012-03-06 15:28:01 -08:00
Mans Rullgard
627f4621f5 ac3: Do not read past the end of ff_ac3_band_start_tab.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Alex Converse <alex.converse@gmail.com>
(cherry picked from commit 034b03e7a0)
2012-03-06 15:28:01 -08:00
Dale Curtis
feed0c6b6a mpegaudiodec: Prevent premature clipping of mp3 input buffer.
Instead of clipping extrasize based on EXTRABYTES, clip based on the
amount of buffer actually left. Without this fix, there are warbles
and other distortions in the test case below.

http://kevincennis.com/mix/assets/sounds/1901_voxfx.mp3
(cherry picked from commit b716542691)

Signed-off-by: Alex Converse <alex.converse@gmail.com>
2012-03-06 15:28:00 -08:00
Alex Converse
d0e53ecff7 mp3dec: Fix a heap-buffer-overflow
In some cases, what is left to read from ptr is smaller than EXTRABYTES.

Based on a patch by Thierry Foucu <tfoucu@gmail.com>.

Signed-off-by: Alex Converse <alex.converse@gmail.com>
(cherry picked from commit f372ce119b)
2012-03-06 15:28:00 -08:00
Alex Converse
1ca84aa162 mpeg12: Pad framerate tab to 16 entries.
There are many places where we read an unchecked 4-bit index into it.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
(cherry picked from commit dfa37fe8a3)
2012-03-06 15:28:00 -08:00
Michael Niedermayer
d5f2382d03 kgv1dec: Increase offsets array size so it is large enough.
Fixes CVE-2011-3945

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 807a045ab7)

Signed-off-by: Alex Converse <alex.converse@gmail.com>
(cherry picked from commit a02e8df973)
2012-03-06 15:28:00 -08:00
Alex Converse
416849f2e0 kmvc: Check palsize.
Fixes: CVE-2011-3952

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Based on fix by Michael Niedermayer
(cherry picked from commit 386741f887)
2012-03-06 15:28:00 -08:00
Martin Storsjö
ca7e97bdcf g722: Fix the QMF scaling
This fixes clipping if the encoder input used the full 16 bit
input range (samples with a magnitude below 16383 worked fine).
The filtered subband samples should be 15 bit maximum, while
the code earlier produced them scaled to 16 bit.

This makes the decoder output have double the magnitude
compared to before.

The spec reference samples doesn't test the QMF at all, which
was why this part slipped past initially.

(cherry picked from commit b087ce2bee)

Signed-off-by: Martin Storsjö <martin@martin.st>
2012-03-06 15:45:30 +02:00
Justin Ruggles
4ae138cb12 ac3dsp: do not use pshufb in ac3_extract_exponents_ssse3()
We need to do unsigned saturation in order to cover the corner case when the
absolute coefficient value is 16777215 (the maximum value).

Fixes Bug #216
(cherry picked from commit d483bb58c3)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-06 13:55:35 +01:00
Fabian Greffrath
003f7e3dd0 Fix format string vulnerability detected by -Wformat-security.
Signed-off-by: Diego Biurrun <diego@biurrun.de>
(cherry picked from commit c9dbac36ad)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-05 18:01:37 +01:00
Ronald S. Bultje
85eb76a23f h264: fix mmxext chroma deblock to use correct TC values.
(cherry picked from commit b0c4f04338)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-05 18:00:43 +01:00
Ronald S. Bultje
5186984ee9 h264: change underread for 10bit QPEL to overread.
This prevents us from reading before the start of the buffer, and thus
prevents crashes resulting from this behaviour. Fixes bug 237.
(cherry picked from commit 291c9b6285)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-05 18:00:31 +01:00
Ronald S. Bultje
b5331b979b cscd: use negative error values to indicate decode_init() failures.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 8a9faf33f2)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-05 14:48:35 +01:00
Vitor Sessak
11f3173e1b amrnbdec: check frame size before decoding.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Ronald S. Bultje <rsbultje@gmail.com>
(cherry picked from commit 882abda5a2)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-05 14:48:35 +01:00
Ronald S. Bultje
cd17195d1c h264: prevent overreads in intra PCM decoding.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit d1604b3de9)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-05 14:48:35 +01:00
Justin Ruggles
1128b10247 wmaenc: fix m/s stereo encoding for the first frame
We need to set ms_stereo in encode_init() in order to avoid incorrectly
encoding the first frame as non-m/s while flagging it as m/s. Fixes an
uncomfortable pop in the left channel at the start of playback.

CC:libav-stable@libav.org
(cherry picked from commit 51ddf35c90)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-04 21:26:29 +01:00
Justin Ruggles
6a073aa7a7 wmaenc: limit allowed sample rate to 48kHz
ff_wma_init() allows up to 50kHz, but this generates an exponent band
size table that requires 65 bands. The code assumes 25 bands in many
places, and using sample rates higher than 48kHz will lead to buffer
overwrites.

CC:libav-stable@libav.org
(cherry picked from commit 1ec075cfec)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-04 21:26:29 +01:00
Justin Ruggles
073891e875 wmaenc: limit block_align to MAX_CODED_SUPERFRAME_SIZE
This is near the theoretical limit for wma frame size and is the most that
our decoder can handle. Allowing higher bit rates will just end up padding
each frame with empty bytes.

Fixes invalid writes for avconv when using very high bit rates.

CC:libav-stable@libav.org
(cherry picked from commit c2b8dea182)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-04 21:26:29 +01:00
Justin Ruggles
2e341bc99a wmaenc: require a large enough output buffer to prevent overwrites
The maximum theoretical frame size is around 17000 bytes. Although in
practice it will generally be much smaller, we require a larger buffer
just to be safe.

CC: libav-stable@libav.org
(cherry picked from commit dfc4fdedf8)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-04 21:26:29 +01:00
Ronald S. Bultje
78d4f8cc56 amrwb: remove duplicate arguments from extrapolate_isf().
Prevents warnings because the dst and src overlap (are the same) in the
memcpy() inside the function.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 9d87374ec0)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-04 21:26:28 +01:00
Ronald S. Bultje
de2656ec25 amrwb: error out early if mode is invalid.
Prevents using the invalid mode as an index in a static array, which
would generate invalid reads.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 154b8bb800)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-04 21:26:28 +01:00
Ronald S. Bultje
b863979c0f wma: fix invalid buffer size assumptions causing random overreads.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 349b7977e4)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-04 12:26:06 +01:00
Alex Converse
fecd7468fc wmadec: Verify bitstream size makes sense before calling init_get_bits.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
(cherry picked from commit 48f1e5212c)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-04 12:26:06 +01:00
Alex Converse
19da1a39e8 rv10/20: Fix a buffer overread caused by losing track of the remaining buffer size.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 2f6528537f)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-04 12:26:06 +01:00
Ronald S. Bultje
7e88df99e1 lcl: return negative error codes on decode_init() errors.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit bd17a40a7e)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-04 12:26:06 +01:00
Ronald S. Bultje
750f5baf30 h264: error out on invalid bitdepth.
Fixes invalid reads while initializing the dequant tables, which uses
the bit depth to determine the QP table size.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 0ce4fe482c)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-04 12:26:06 +01:00