Commit graph

17115 commits

Author SHA1 Message Date
Michael Niedermayer
1ab4578c88 lavc: Fix assignments in if() when calling ff_af_queue_add
Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit 1d7ffd06e4)

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-05-08 19:05:21 +02:00
Anton Khirnov
d8745de6ae indeo3: fix off by one in MV validity check
CC:libav-stable@libav.org
(cherry picked from commit 95220be1fa)

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-05-03 19:21:45 +02:00
Luca Barbato
46fd6e4f2e aac: check the maximum number of channels
Broken bitstreams could report a larger than specified number of
channels and cause outbound writes.

CC:libav-stable@libav.org
(cherry picked from commit a943a132f3)

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-04-28 00:43:43 +02:00
Anton Khirnov
a3410b5a1f svq1dec: clip motion vectors to the frame size.
Fixes invalid reads for corrupted files.

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC:libav-stable@libav.org
(cherry picked from commit ecff5acb5a)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-04-18 22:06:09 +02:00
Anton Khirnov
43039f9386 svq1dec: check that the reference frame has the same dimensions as the current one
They can be different if the last keyframe failed to decode correctly.
Fixes possible invalid reads in such a case.

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC:libav-stable@libav.org
(cherry picked from commit b1bb8fb860)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-04-18 22:05:55 +02:00
Anton Khirnov
d0c4d61c8b qdm2: check that the FFT size is a power of 2
Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC:libav-stable@libav.org
(cherry picked from commit 34f87a5853)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-04-18 22:05:20 +02:00
Anton Khirnov
95db1624ef indeo3: switch parsing the header to bytestream2
Also add an additional sanity check to the alt_quant table.
Fixes invalid reads with corrupted files.

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC:libav-stable@libav.org
(cherry picked from commit 66531d634e)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-04-18 22:05:09 +02:00
Anton Khirnov
b0b33ce148 indeo3: check motion vectors.
Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC:libav-stable@libav.org
(cherry picked from commit a0a872d073)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-04-18 22:04:53 +02:00
Anton Khirnov
fa4192e31f rv10: check that extradata is large enough
Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC:libav-stable@libav.org

(cherry picked from commit 01d376f598)

Conflicts:

	libavcodec/rv10.c
2013-04-18 22:03:32 +02:00
Anton Khirnov
4c412580fd indeo3: fix data size check
The data offsets are relative to the bistream header, which is 16 bytes
after the start of the data.
Fixes invalid reads with corrupted files.

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC:libav-stable@libav.org
(cherry picked from commit 34e6af9e20)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-04-18 22:01:24 +02:00
Anton Khirnov
62f9253781 dfa: check for invalid access in decode_wdlt().
This can happen when the number of skipped lines is not consistent with
the number of coded lines.

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 3623589edc)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-04-06 11:47:56 +02:00
Anton Khirnov
ba31b72f46 bmv: check for len being valid in bmv_decode_frame().
It can be 0 or -1 for invalid files, which may result in invalid memory
access.

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit b88f902125)

Conflicts:

	libavcodec/bmv.c
2013-04-06 11:47:01 +02:00
Matt Wolenetz
fc6825ebb6 vp8: Fix pthread_cond and pthread_mutex leaks
CC: libav-stable@libav.org

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
(cherry picked from commit 1d6e618939)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-03-31 10:38:22 +02:00
Xi Wang
0b0e87bb54 atrac3: avoid oversized shifting in decode_bytes()
When `off' is 0, `0x537F6103 << 32' in the following expression invokes
undefined behavior, the result of which is not necessarily 0.

    (0x537F6103 >> (off * 8)) | (0x537F6103 << (32 - (off * 8)))

Avoid oversized shifting.

CC: libav-stable@libav.org

Signed-off-by: Xi Wang <xi.wang@gmail.com>
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>

(cherry picked from commit eba1ff3130)
2013-03-15 13:20:55 +01:00
Luca Barbato
88089eecfd shorten: use the unsigned type where needed
get_uint returns an unsigned value, use an unsigned to store
blocksize to make sure the comparison logic is correct and report
correctly the error for the channel count not supported.

CC: libav-stable@libav.org

(cherry picked from commit 5cf7c72757)

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-03-12 13:37:10 +01:00
Luca Barbato
0daf1428e8 shorten: report meaningful errors
(cherry picked from commit 4c364eb2b8)

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-03-12 13:37:10 +01:00
Luca Barbato
97cc2f286f shorten: K&R formatting cosmetics
(cherry picked from commit a2ad554def)

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-03-12 13:37:10 +01:00
Michael Niedermayer
21d568be17 shorten: set invalid channels count to 0
Prevent the loop shorten_decode_close from writing and freeing out of
the array boundary.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>

CC: libav-stable@libav.org

(cherry picked from commit c10da30d84)
2013-03-12 13:36:50 +01:00
Anton Khirnov
0cb3cab343 eamad: allocate a dummy reference frame when the real one is missing
Fixes invalid reads when the first frame is not an I-frame.

CC:libav-stable@libav.org
(cherry picked from commit 7b89cd20d8)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2013-03-09 19:05:42 +01:00
Justin Ruggles
b77d9cbbd5 libmp3lame: use the correct remaining buffer size when flushing
CC:libav-stable@libav.org
(cherry picked from commit e984f47873)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-03-09 18:10:16 +01:00
Justin Ruggles
905f5c8a1e png: use av_mallocz_array() for the zlib zalloc function
Fixes valgrind uninitialized memory errors when decoding png.

CC:libav-stable@libav.org
(cherry picked from commit 486f0b0cfc)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-03-09 18:10:16 +01:00
Anton Khirnov
20373a66ec wmaprodec: require block_align to be set.
Avoids an infinite loop in the calling programs with decoder not
consuming any input and not returning output.

CC:libav-stable@libav.org
(cherry picked from commit cacad1c058)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-03-09 18:10:16 +01:00
Anton Khirnov
d48da91373 ffv1: fix calculating slice dimensions for version 2
It got broken in 0f13cd3187.

CC:libav-stable@libav.org
(cherry picked from commit d243896987)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-03-09 18:10:16 +01:00
Anton Khirnov
62a657de16 xxan: fix invalid memory access in xan_decode_frame_type0()
The loop a few lines below the xan_unpack() call accesses up to
dec_size * 2 bytes into y_buffer, so dec_size must be limited to
buffer_size / 2.

CC:libav-stable@libav.org
(cherry picked from commit 8a49d2bcbe)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-03-09 18:10:16 +01:00
Anton Khirnov
c1f479e8df wmadec: require block_align to be set.
Avoids an infinite loop in the calling programs with decoder not
consuming any input and not returning output.

CC:libav-stable@libav.org
(cherry picked from commit ea1136baaf)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-03-09 18:01:09 +01:00
Anton Khirnov
74880e78d8 ivi_common: do not call MC for intra frames when dc_transform is unset
CC:libav-stable@libav.org
(cherry picked from commit 3ba40ebb6c)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-03-09 18:01:09 +01:00
Anton Khirnov
747fbe0c21 roqvideodec: fix a potential infinite loop in roqvideo_decode_frame().
When there is just 1 byte remanining in the buffer, nothing will be read
and the loop will continue forever. Check that there are at least 8
bytes, which are always read at the beginning.

CC:libav-stable@libav.org
(cherry picked from commit 3e2f200237)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-03-09 18:01:09 +01:00
Alexander Kojevnikov
d3b40af01f mp3dec: Fix VBR bit rate parsing
When parsing the Xing/Info tag, don't set the bit rate if it's an Info tag.

When parsing the stream, don't override the bit rate if it's already set,
otherwise calculate the mean bit rate from parsed frames. This way, the bit
rate will be set correctly both for CBR and VBR streams.

CC:libav-stable@libav.org

Signed-off-by: Anton Khirnov <anton@khirnov.net>
(cherry picked from commit eae0879d96)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-03-09 18:01:09 +01:00
Anton Khirnov
60dd8b5733 wmaprodec: return an error, not 0, when the input is too small.
Returning 0 may result in an infinite loop in valid calling programs. A
decoder should never return 0 without producing any output.

CC:libav-stable@libav.org
(cherry picked from commit 4c0080b7e7)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-03-09 18:01:09 +01:00
Anton Khirnov
77cf052e39 vmdaudio: fix invalid reads when packet size is not a multiple of chunk size
CC:libav-stable@libav.org
(cherry picked from commit f86d66bcfa)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-03-09 18:01:09 +01:00
Luca Barbato
146eac0a0c h264: check for luma and chroma bit dept being equal
The decoder assumes a single bit depth for all the planes
while the specification allows different bit depths for luma
and chroma.

Avoid the possible problems described in CVE-2013-2277

CC: libav-stable@libav.org
(cherry picked from commit 4987faee78)

Conflicts:

	libavcodec/h264.c
2013-03-09 18:01:09 +01:00
Anton Khirnov
704952fee5 h264: set ref_count to 0 for intra slices.
CC:libav-stable@libav.org
(cherry picked from commit 437211ae73)

Fixes deadlocks waiting for non-existing references with some fuzzed files.

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-03-02 11:20:59 +01:00
Anton Khirnov
b6f5a1ca58 h264: on reference overflow, reset the reference count to 0, not 1.
Since decode_slice_header() returns before the reference lists are
constructed, there are zero valid references.

CC:libav-stable@libav.org
(cherry picked from commit 668e16a0dd)

Conflicts:

	libavcodec/h264.c
2013-03-02 11:20:59 +01:00
Luca Barbato
c6c4dc6935 vorbisdec: Accept 0 amplitude_bits
The specification does not prevent an encoder to write the amplitude 0
as 0 amplitude_bits.

Our get_bits() implementation might not support a zero sized read
properly, thus the additional branch.
(cherry picked from commit 23bd9ef4b2)

Conflicts:

	libavcodec/vorbisdec.c
2013-02-26 20:21:01 +01:00
Michael Niedermayer
494ddd377a vorbisdec: Error on bark_map_size equal to 0.
The value is used to calculate output LSP curve and a division by zero
and out of array accesses would occur.

CVE-2013-0894

CC: libav-stable@libav.org

Reported-by: Dale Curtis <dalecurtis@chromium.org>
Found-by: inferno@chromium.org
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
(cherry picked from commit 11dcecfcca)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-02-26 20:21:01 +01:00
Luca Barbato
37e99e384e vorbisdec: Add missing checks
Rate and order must not be 0 even if the specification does not say that
explicitly.
(cherry picked from commit 5b47c19bfd)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-02-26 20:21:01 +01:00
Justin Ruggles
73d6f4651e ac3dec: validate channel output mode against channel count
Damaged frames can lead to a mismatch, which can cause a segfault
due to using an incorrect channel mapping.

CC:libav-stable@libav.org
(cherry picked from commit d7c450436f)

Conflicts:

	libavcodec/ac3dec.c
2013-02-26 20:21:01 +01:00
Anton Khirnov
77493bfd97 cavs: initialize various context tables to 0
Avoids crashes with corrupted files.

CC:libav-stable@libav.org
(cherry picked from commit 4f3b058c84)

Conflicts:

	libavcodec/cavs.c

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-02-23 14:46:54 +01:00
Anton Khirnov
bb3f1cad17 4xm: check the return value of read_huffman_tables().
CC:libav-stable@libav.org
(cherry picked from commit 8097fc9a2d)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-02-23 14:41:24 +01:00
Anton Khirnov
a6403a3b69 qtrle: add more checks against pixel_ptr being negative.
CC:libav-stable@libav.org
(cherry picked from commit e106592447)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-02-23 14:41:10 +01:00
Anton Khirnov
e2cf32ca5f mlpdec: do not try to allocate a zero-sized output buffer.
CC:libav-stable@libav.org
(cherry picked from commit 0dff40bfb9)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-02-23 14:40:48 +01:00
Anton Khirnov
612b28194b flicvideo: avoid an infinite loop in byte run compression
When byte_run is 0, pixel_countdown is not touched and the loop will run
forever.

CC:libav-stable@libav.org
(cherry picked from commit ddfe1246d9)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-02-23 14:40:13 +01:00
Anton Khirnov
8bce2c60b8 lagarith: avoid infinite loop in lag_rac_refill()
range == 0 happens with corrupted files

CC:libav-stable@libav.org
(cherry picked from commit de6dfa2bb8)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-02-23 14:40:04 +01:00
Anton Khirnov
b786ddc0f2 loco: check that there is data left after decoding a plane.
CC:libav-stable@libav.org
(cherry picked from commit 067432c1c9)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-02-23 14:39:19 +01:00
Matt Wolenetz
bc9d341be8 x86: h264: Don't use redzone in AVX h264_deblock on Win64
This fixes crashes in chromium on win64 on machines with AVX
(crashes that apparently aren't triggered by fate).

Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit 311443f6c7)

Signed-off-by: Martin Storsjö <martin@martin.st>
2013-02-22 23:48:35 +02:00
Michael Niedermayer
ab434bf0d0 vqavideo: check chunk sizes before reading chunks
Fixes out of array writes

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit ab6c9332bf)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 13093f9767)

CC: libav-stable@libav.org

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit f7d18deb73)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-02-19 08:10:24 +01:00
Michael Niedermayer
52b18c1fde roqvideodec: check dimensions validity
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 3ae6104511)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit fee26d352a)

CC: libav-stable@libav.org

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit 488f87be87)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-02-19 08:10:03 +01:00
Michael Niedermayer
0b2b8ab979 qdm2: check array index before use, fix out of array accesses
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

(cherry picked from commit a7ee6281f7)

CC: libav-stable@libav.org

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit 39bec05ed4)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-02-19 08:09:48 +01:00
Martin Storsjö
65bf4c9c45 mpegvideo: Do REBASE_PICTURE with byte pointers
REBASE_PICTURE (more specifically, this half of it) takes a Picture
pointer that points into one larger struct, finds the offset of
that Picture within the struct and finds the corresponding field
within another instance of a similar struct.

The pointer difference "pic - (Picture*)old_ctx" is a value given
in sizeof(Picture) units, and when applied back on
(Picture*)new_ctx gets multiplied back with sizeof(Picture). Many
compilers seem to optimize out this division/multiplication, but
not all do.

GCC 4.2 on OS X doesn't seem to remove the division/multiplication,
therefore the new pointer didn't turn out to point to exactly
the right place in the new struct since it only had sizeof(Picture)
granularity (and the Picture is not aligned on a sizeof(Picture)
boundary within the encompassing struct). This bug has been present
before 47318953d as well - with H264, pointers to h->ref_list[0][0]
pointed to 88 bytes before h->ref_list[0][0] after the rebase. After
shrinking Picture, the difference ended up even larger, making
writes via such a Picture pointer overwrite other fields at random
in H264Context, ending up in crashes later.

This fixes H264 multithreaded decoding on OS X with GCC 4.2.

Fixes Bug: #439

Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit a65f965c04)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-02-19 08:09:08 +01:00
Matti Hamalainen
d61c6ebccf svq3: unbreak decoding
a7d2861d36 removed necessary braces.
2013-02-18 02:49:45 +01:00