Commit graph

16787 commits

Author SHA1 Message Date
Michael Niedermayer
3926ebc702 vorbisdec: Check bark_map_size.
This fixes potential divisions by zero and out of array accesses.

Reported-by: Dale Curtis <dalecurtis@chromium.org>
Found-by: inferno@chromium.org
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 2c16bf2de0)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-03-21 04:23:59 +01:00
Michael Niedermayer
85162a60b8 avcodec/snow: split block clipping checks
Fixes out of array read
Fixes: d4476f68ca1c1c57afbc45806f581963-asan_heap-oob_2266b27_8607_cov_4044577381_snow_chroma_bug.avi
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 61d59703c9)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-03-21 04:23:59 +01:00
Michael Niedermayer
573d5fdeda avcodec/msrle: use av_image_get_linesize() to calculate the linesize
Fixes out of array access
Fixes: 14a74a0a2dc67ede543f0e35d834fbbe-asan_heap-oob_49572c_556_cov_215466444_44_001_engine_room.mov
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit c919e1ca2e)

Conflicts:

	libavcodec/msrle.c
(cherry picked from commit bc1c8ec5e6)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-03-21 04:23:59 +01:00
Michael Niedermayer
954ce59a0a avcodec/wmalosslessdec: fix mclms_coeffs* array size
Fixes corruption of context
Fixes: 8835659dde6a4f7dcdf341de6a45c6c8-signal_sigsegv_1dce67b_4564_cov_2504444599_classical_22_16_1_14000_v3c_0_extend_0_29.wma
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit ec9578d54d)

Conflicts:

	libavcodec/wmalosslessdec.c
(cherry picked from commit 5d683807be71002a6c3facef3e26350762a760e2)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-03-21 04:23:59 +01:00
Michael Niedermayer
356f8634e9 wmalosslessdec: make mclms arrays big enough for whats written into them.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit a0abefb0af)

Conflicts:

	libavcodec/wmalosslessdec.c
2014-03-21 04:23:59 +01:00
Michael Niedermayer
605c3f0515 avcodec/vc1: reset fcm/field_mode in non advanced header parsing
Fixes NULL pointer dereference
Fixes: signal_sigsegv_1ab8bf4_2847_cov_4254117347_SA10091.vc1
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit b51e935477)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-03-21 04:23:59 +01:00
Michael Niedermayer
e0f351a979 avcodec/vmnc: Check that rectangles are within the picture
Prevents out of array accesses with CODEC_FLAG_EMU_EDGE

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 6ba02602aa)

Conflicts:

	libavcodec/vmnc.c

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 7c17207ab9)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-03-21 04:23:59 +01:00
Michael Niedermayer
c44ac4d81e dnxhdenc: fix mb_rc size
Fixes out of array access with RC_VARIANCE set to 0

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit f1caaa1c61)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-03-21 04:23:59 +01:00
Michael Niedermayer
84e1aa7768 avcodec/jpeglsdec: check err value for ls_get_code_runterm()
Fixes infinite loop
Fixes Ticket3086

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit cc0e47b550)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-03-21 04:23:59 +01:00
Michael Niedermayer
87d335c007 avcodec/parser: reset indexes on realloc failure
Fixes Ticket2982

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit f31011e9ab)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-03-21 04:23:59 +01:00
Martin Storsjö
320df1c7c0 arm: Don't clobber callee saved registers in scalarproduct
q4-q7/d8-d15 are supposed to not be clobbered by the callee.

CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit d307e408d4)
2013-12-21 10:00:28 +01:00
Michael Niedermayer
ff3e385d84 avcodec/h264: reduce noisiness of "mmco: unref short failure"
Do not consider it an error if we have no frames and should discard one.
This condition can easily happen when decoding is started from an I frame

Fixes Ticket2811

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 08a8976196)

Conflicts:
	libavcodec/h264_refs.c
2013-10-26 01:29:28 +02:00
Michael Niedermayer
1b3803e8c1 avcodec/h264_refs: modify key frame detection heuristic to detect more cases
Fixes Ticket2968

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 5ac6b6028f)

Conflicts:
	libavcodec/h264_refs.c
2013-10-24 11:52:05 +02:00
Michael Niedermayer
a6388942f5 avcodec/ffv1enc: update buffer check for 16bps
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 3728603f18)

Conflicts:

	libavcodec/ffv1enc.c
(cherry picked from commit c900c6e5c26cd86cf34f9c8d4347cedbd01f3935)
2013-09-23 00:02:30 +02:00
Michael Niedermayer
a62852bae0 avcodec/dsputil: fix signedness in sizeof() comparissions
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 454a11a1c9)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-09-23 00:02:30 +02:00
Michael Niedermayer
40d4e82e2e avcodec/pngdsp: fix (un)signed type in end comparission
Fixes out of array accesses
Fixes Ticket2919

Found_by: ami_stuff
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 86736f59d6)

Conflicts:

	libavcodec/pngdsp.c

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-09-23 00:02:30 +02:00
Michael Niedermayer
a4b4be7493 jpeg2000: check log2_cblk dimensions
Fixes out of array access
Fixes Ticket2895

Found-by: Piotr Bandurski <ami_stuff@o2.pl>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 9a271a9368)

Conflicts:

	libavcodec/jpeg2000dec.c

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

Conflicts:

	libavcodec/j2kdec.c

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-09-22 22:52:12 +02:00
Michael Niedermayer
4490b3db23 avcodec/rpza: Perform pointer advance and checks before using the pointers
Fixes out of array accesses
Fixes Ticket2850

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 3819db745d)

Conflicts:

	libavcodec/rpza.c
2013-09-22 22:52:12 +02:00
Michael Niedermayer
edb4aad93d avcodec/flashsv: check diff_start/height
Fixes out of array accesses
Fixes Ticket2844

Found-by: ami_stuff
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 880c73cd76)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-09-22 22:52:12 +02:00
Michael Niedermayer
e4ad9e9377 update all trac links to use the trac subdomain
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-09-22 22:52:12 +02:00
Michael Niedermayer
21dd8f5baa huffyuvdec: Skip len==0 cases
Fixes vlc decoding for hypothetical files that would contain such cases.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 0dfc01c2bb)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 5ff41ffeb4cb9ea6df49757dc859619dc3d3ab4f)

Conflicts:

	libavcodec/huffyuv.c
(cherry picked from commit 9bc70fe1ae50fd2faa0b9429d47cfbda01a92ebc)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-09-22 22:52:12 +02:00
Michael Niedermayer
03e2e95beb huffyuvdec: Check init_vlc() return codes.
Prevents out of array writes

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit f67a0d1152)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 95ab8d33e1a680f30a5a9605175112008ab81afc)

Conflicts:

	libavcodec/huffyuv.c
(cherry picked from commit 277def59fce10d91e3113e5c0f63e22bc4abfa88)

Conflicts:

	libavcodec/huffyuv.c
2013-09-22 22:52:12 +02:00
Michael Niedermayer
d0a225d6c1 mpeg12dec: avoid reinitialization on PS changes when possible.
Fixes Ticket2574

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 970c8df735)

Conflicts:
	libavcodec/mpeg12dec.c
2013-07-09 00:55:55 +02:00
Michael Niedermayer
92abb12863 alacenc: Fix missing sign_extend()
Fixes ticket #2497

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 8aea2f05dc)
2013-06-13 00:17:43 +02:00
Michael Niedermayer
8e7e12dcbf h264_cavlc: fix reading skip run
Fixes Ticket2606

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 826b3a75cd)

Conflicts:
	libavcodec/h264_cavlc.c
2013-05-30 23:30:43 +02:00
Michael Niedermayer
0147e9f7c6 smacker: remove av_clip_int16()
Fixes Ticket2425

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 2211c76287)
2013-05-21 00:14:34 +02:00
Carl Eugen Hoyos
6642804c22 Fix type of shared flac table ff_flac_blocksize_table[].
Fixes ticket #2533.
(cherry picked from commit a07ac1f788)
2013-05-05 20:39:38 +02:00
Carl Eugen Hoyos
8b597077ae Do not (re-)set libx264 parameter b_tff if interlaced encoding was not requested.
Reconfiguring can break x264 lossless encoding.

Fixes ticket #2165.
(cherry picked from commit 75c7e4583f)
2013-03-18 02:23:20 +01:00
Piotr Bandurski
8b48e2c654 tiffdec: Use the correct height field.
Fixes Ticket913

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 4784a135b2)

Conflicts:
	libavcodec/tiff.c
2013-01-07 00:38:13 +01:00
Clément Bœsch
42abae1125 lavc/ass_split: check for NULL pointer in ff_ass_split_override_codes().
This is consistent with the other ff_ass_split_* functions.

It also fixes a crash when trying to split a dialog with text=NULL
(which seems to happen when the text of the dialog is empty); basically,
this commit fixes crashes when trying to encode an empty text subtitle
dialog (see subrip and mov_text encoders).

Fixes Ticket2048.
(cherry picked from commit c83002a4f8)
2013-01-01 18:23:54 +01:00
Ronald S. Bultje
3c3ee65ebb dxva2: include dxva.h if found
Apparently, some build environments require dxva.h even for dxva2,
while others lack this header entirely.  Including it conditionally
allows building in both cases.

Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit fa84506177)

Conflicts:

	configure

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2012-09-13 04:44:19 +02:00
Michael Niedermayer
5d6982c13b indeo4: check that num_mbs matches
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit d3db8988d5)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2012-05-03 01:31:07 +02:00
Michael Niedermayer
e11fa0879a dsp: fix diff_bytes_mmx() with small width
Fixes Ticket1068

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 73089eccd3e48539555349b36d8aabbf1cea416e)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2012-05-03 01:27:33 +02:00
Michael Niedermayer
e70d202275 vqavideodev: Check image dimensions
Fixes out of heap array read

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 3583c8706d)
Independently-Found-by: Fabian Yamaguchi
Fixes: CVE-2012-0947

Conflicts:

	libavcodec/vqavideo.c
2012-05-03 00:29:18 +02:00
Michael Niedermayer
9de0c8c60c sonic: update to new API
Fixes Ticket1075

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 6f9803e5e0)

Conflicts:

	libavcodec/sonic.c

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2012-05-03 00:28:35 +02:00
Michael Niedermayer
db041fd115 qpeg: Fix out of array writes.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2012-05-03 00:28:35 +02:00
Fabian Greffrath
7a877418e3 srtdec: fix a format string vulnerability.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit aaa1173de7)
2012-05-03 00:28:35 +02:00
ami_stuff
cf9b04c6f2 Replace SSE2 instruction in scalarproduct_float_sse() by SSE equivalent.
Fixes an AAC decoding issue with the sample from ticket #213 on machines
with SSE but without SSE2.
Based on 89411a by Reimar.

(cherry picked from commit f6b7863808)
2012-04-04 09:16:02 +02:00
Michael Niedermayer
89f2d6c349 h264: fix seeking in low delay streams without IDR
Fixes Ticket1165

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 3360b8517a)
2012-04-04 08:41:14 +02:00
Franz Brauße
32a79b5649 smacker audio: sign-extend the initial 16-bit predicted value
Fixes Bug #265

Signed-off-by: Justin Ruggles <justin.ruggles@gmail.com>
(cherry picked from commit 12cbbbb4ab)
2012-04-01 14:00:13 +02:00
Michael Niedermayer
63945e2226 indeo4: fix LE reader define for backport
Fixes Ticket920

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2012-01-16 22:26:52 +01:00
Carl Eugen Hoyos
ad6eefc0d8 Fix compilation with yasm-0.6.2. 2012-01-12 16:53:42 +01:00
John Brooks
e540446f6c vc1dec: fix invalid memory access for small video dimensions
For small video dimensions, these calculations of the upper bound
for pixel access may have a negative result. Using an unsigned
comparison to bound a potentially negative value only works if
the greater operand is non-negative. Fixed by doing edge emulation
when the upper bound is probably negative, everywhere that this
pattern appears.

Signed-off-by: Kostya Shishkov <kostya.shishkov@gmail.com>
(cherry picked from commit d209c27b09)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2012-01-05 02:23:08 +01:00
John Brooks
fc9c5ad9ea rv34: fix invalid memory access for small video dimensions
For small video dimensions calculations of the upper bound for pixel
access may result in negative value. Using an unsigned comparison
works only if the greater operand is non-negative. This is fixed by
doing edge emulation explicitly for such conditions.

Signed-off-by: Kostya Shishkov <kostya.shishkov@gmail.com>
(cherry picked from commit aacf6b3a2f)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2012-01-05 02:23:02 +01:00
Michael Niedermayer
58afa73338 vorbis: make sure ch is non zero before calling vorbis_residue_decode
This possibly makes part of the CVE-2011-3895 fix unneeded.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit ff7f198d7f)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2012-01-04 22:19:02 +01:00
Chris Evans
68ee43468e vorbis: An additional defense in the Vorbis codec.
BUG=101458
Review URL: http://codereview.chromium.org/8414025

Fixes second part of CVE-2011-3895

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit f74ce3a60d)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2012-01-04 22:19:00 +01:00
Chris Evans
80440c5b1d vorbis: Fix decoder bug.
BUG=101458
Review URL: http://codereview.chromium.org/8413019

This fixes part of 2011-3895

bigned-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 68226ed9ec)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2012-01-04 22:19:00 +01:00
Michael Niedermayer
89bd49b25c vorbisdec: Make sure blocksize is not set to an invalid value.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 405e99bdfd)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2012-01-04 22:18:59 +01:00
Michael Niedermayer
d7e5301e43 vorbis: Fix last quarter of CVE-2011-3893
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 6fcf2bb8af)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2012-01-04 22:18:58 +01:00
Chris Evans
6a3fed8749 ogg: Avoid the possibility to read out-of-bounds of a static global array in Vorbis
decoding.

BUG=100543
Review URL: http://codereview.chromium.org/8365014
This fixes 25% of CVE-2011-3893

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 7149fce2ca)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2012-01-04 22:18:56 +01:00