Commit graph

15052 commits

Author SHA1 Message Date
Martin Storsjö
8925c44db1 arm: Don't clobber callee saved registers in scalarproduct
q4-q7/d8-d15 are supposed to not be clobbered by the callee.

CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit d307e408d4)
2013-12-21 10:00:31 +01:00
Michael Niedermayer
bd66456866 avcodec/ffv1enc: update buffer check for 16bps
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 3728603f18)

Conflicts:

	libavcodec/ffv1enc.c
(cherry picked from commit c900c6e5c26cd86cf34f9c8d4347cedbd01f3935)
2013-10-06 03:13:28 +02:00
Michael Niedermayer
90ee388b28 avcodec/dsputil: fix signedness in sizeof() comparissions
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 454a11a1c9)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-10-06 03:13:28 +02:00
Michael Niedermayer
e3d643cf75 avcodec/pngdsp: fix (un)signed type in end comparission
Fixes out of array accesses
Fixes Ticket2919

Found_by: ami_stuff
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 86736f59d6)

Conflicts:

	libavcodec/pngdsp.c

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-10-06 03:13:28 +02:00
Michael Niedermayer
38ca79b04d jpeg2000: check log2_cblk dimensions
Fixes out of array access
Fixes Ticket2895

Found-by: Piotr Bandurski <ami_stuff@o2.pl>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 9a271a9368)

Conflicts:

	libavcodec/jpeg2000dec.c

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

Conflicts:

	libavcodec/j2kdec.c

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-10-06 03:13:28 +02:00
Michael Niedermayer
0dff3171ce avcodec/rpza: Perform pointer advance and checks before using the pointers
Fixes out of array accesses
Fixes Ticket2850

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 3819db745d)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-10-06 02:38:45 +02:00
Michael Niedermayer
eec833b10d avcodec/parser: reset indexes on realloc failure
Fixes Ticket2982

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit f31011e9ab)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-09-26 23:25:00 +02:00
Michael Niedermayer
9b0736c08a Merge remote-tracking branch 'qatar/release/0.7' into release/0.8
* qatar/release/0.7:
  Update changelog for 0.7.8 release
  aac: check the maximum number of channels
  oggdec: fix faulty cleanup prototype
  qdm2: check that the FFT size is a power of 2
  rv10: check that extradata is large enough
  lavf: make sure stream probe data gets freed.
  dfa: check for invalid access in decode_wdlt().
  avfiltergraph: check for sws opts being non-NULL before using them.

Conflicts:
	Changelog
	libavformat/utils.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-09-22 17:43:33 +02:00
Michael Niedermayer
70a1182a48 Merge commit 'f844cb9bce' into release/0.8
* commit 'f844cb9bce':
  iff: validate CMAP palette size
  wmaprodec: require block_align to be set.
  lzo: fix overflow checking in copy_backptr()
  flacdec: simplify bounds checking in flac_probe()
  atrac3: avoid oversized shifting in decode_bytes()
  lavf: fix arithmetic overflows in avformat_seek_file()

Conflicts:
	libavformat/iff.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-09-22 17:36:39 +02:00
Michael Niedermayer
49d597f058 Merge commit '9c713f30e4' into release/0.8
* commit '9c713f30e4':
  parser: fix large overreads
  dsputil: fix invalid array indexing
  shorten: use the unsigned type where needed

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-09-22 17:35:20 +02:00
Michael Niedermayer
44ebb2556d Merge commit '5ebb5a32bd' into release/0.8
* commit '5ebb5a32bd':
  shorten: report meaningful errors
  shorten: set invalid channels count to 0

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-09-22 17:25:51 +02:00
Michael Niedermayer
0a41da3e9d Merge commit 'd785f69401' into release/0.8
* commit 'd785f69401':
  shorten: validate that the channel count in the header is not <= 0
  matroskadec: request a read buffer for the wav header
  h264: check for luma and chroma bit depth being equal
  xxan: fix invalid memory access in xan_decode_frame_type0()
  wmadec: require block_align to be set.

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-09-22 17:22:53 +02:00
Michael Niedermayer
afe09e490a Merge commit '5025dbc577' into release/0.8
* commit '5025dbc577':
  wmaprodec: return an error, not 0, when the input is too small.
  vorbisdec: Error on bark_map_size equal to 0.
  Update RELEASE file for 0.7.8
  update year to 2013
  oggdec: make sure the private parse data is cleaned up
  indeo5: update AVCodecContext width/height on size change
  doc: filters: Correct BNF FILTER description

Conflicts:
	RELEASE

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-09-22 17:17:10 +02:00
Michael Niedermayer
f8c4dbe45e mpeg12dec: avoid reinitialization on PS changes when possible.
Fixes Ticket2574

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 970c8df735)

Conflicts:
	libavcodec/mpeg12.c
2013-07-09 00:59:24 +02:00
Michael Niedermayer
23d835f611 alacenc: Fix missing sign_extend()
Fixes ticket #2497

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 8aea2f05dc)

Conflicts:
	libavcodec/alacenc.c
2013-06-13 00:21:41 +02:00
Michael Niedermayer
d04dc7b5a7 h264_cavlc: fix reading skip run
Fixes Ticket2606

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 826b3a75cd)

Conflicts:
	libavcodec/h264_cavlc.c
2013-05-30 23:37:15 +02:00
Reinhard Tartler
ade4f3e746 aac: check the maximum number of channels
Broken bitstreams could report a larger than specified number of
channels and cause outbound writes.

CC:libav-stable@libav.org
(cherry picked from commit a943a132f3)

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>

Conflicts:
	libavcodec/aacdec.c
2013-05-09 20:05:53 +02:00
Anton Khirnov
96481c5e18 qdm2: check that the FFT size is a power of 2
Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC:libav-stable@libav.org
(cherry picked from commit 34f87a5853)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-05-09 20:05:53 +02:00
Anton Khirnov
9b052bfb86 rv10: check that extradata is large enough
Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC:libav-stable@libav.org

(cherry picked from commit 01d376f598)

Conflicts:

	libavcodec/rv10.c
2013-05-09 20:05:53 +02:00
Anton Khirnov
280998b13c wmaprodec: require block_align to be set.
Avoids an infinite loop in the calling programs with decoder not
consuming any input and not returning output.

CC:libav-stable@libav.org
(cherry picked from commit cacad1c058)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit 20373a66ec)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-05-09 20:05:52 +02:00
Anton Khirnov
96cf80609a dfa: check for invalid access in decode_wdlt().
This can happen when the number of skipped lines is not consistent with
the number of coded lines.

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 3623589edc)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-05-09 20:05:52 +02:00
Michael Niedermayer
9c713f30e4 parser: fix large overreads
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Justin Ruggles <justin.ruggles@gmail.com>

(cherry picked from commit 096abfa150)

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-05-09 11:29:05 +02:00
Mans Rullgard
799000af70 dsputil: fix invalid array indexing
Indexing outside an array is invalid and causes errors with
gcc 4.8.

Signed-off-by: Mans Rullgard <mans@mansr.com>
(cherry picked from commit 0a07f2b346)

Signed-off-by: Diego Biurrun <diego@biurrun.de>
2013-05-09 11:29:05 +02:00
Xi Wang
78889be3fb atrac3: avoid oversized shifting in decode_bytes()
When `off' is 0, `0x537F6103 << 32' in the following expression invokes
undefined behavior, the result of which is not necessarily 0.

    (0x537F6103 >> (off * 8)) | (0x537F6103 << (32 - (off * 8)))

Avoid oversized shifting.

CC: libav-stable@libav.org

Signed-off-by: Xi Wang <xi.wang@gmail.com>
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>

(cherry picked from commit eba1ff3130)

Conflicts:
	libavcodec/atrac3.c
2013-05-09 11:29:05 +02:00
Luca Barbato
6d4d186e9e shorten: use the unsigned type where needed
get_uint returns an unsigned value, use an unsigned to store
blocksize to make sure the comparison logic is correct and report
correctly the error for the channel count not supported.

CC: libav-stable@libav.org

(cherry picked from commit 5cf7c72757)
(cherry picked from commit 88089eecfd)
(cherry picked from commit f42d03746a)

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
Signed-off-by: Reinhard Tartler <siretart@tauware.de>

Conflicts:
	libavcodec/shorten.c
2013-05-09 11:29:01 +02:00
Reinhard Tartler
5ebb5a32bd shorten: report meaningful errors
(cherry picked from commit 4c364eb2b8)
(cherry picked from commit 0daf1428e8)

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>

Conflicts:
	libavcodec/shorten.c
2013-05-09 11:28:28 +02:00
Reinhard Tartler
a694b2b158 shorten: set invalid channels count to 0
Prevent the loop shorten_decode_close from writing and freeing out of
the array boundary.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>

CC: libav-stable@libav.org

(cherry picked from commit c10da30d84)
(cherry picked from commit 21d568be17)

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>

Conflicts:
	libavcodec/shorten.c
2013-05-09 11:28:28 +02:00
Justin Ruggles
d785f69401 shorten: validate that the channel count in the header is not <= 0
(cherry picked from commit 4c53f4aed3)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>

Conflicts:
	libavcodec/shorten.c
2013-05-09 11:28:25 +02:00
Anton Khirnov
5025dbc577 wmaprodec: return an error, not 0, when the input is too small.
Returning 0 may result in an infinite loop in valid calling programs. A
decoder should never return 0 without producing any output.

CC:libav-stable@libav.org
(cherry picked from commit 4c0080b7e7)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit 60dd8b5733)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-05-09 11:20:11 +02:00
Michael Niedermayer
d86a5ce03f vorbisdec: Error on bark_map_size equal to 0.
The value is used to calculate output LSP curve and a division by zero
and out of array accesses would occur.

CVE-2013-0894

CC: libav-stable@libav.org

Reported-by: Dale Curtis <dalecurtis@chromium.org>
Found-by: inferno@chromium.org
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
(cherry picked from commit 11dcecfcca)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit 494ddd377a)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-05-09 11:20:11 +02:00
Reinhard Tartler
7d4c38d58d h264: check for luma and chroma bit depth being equal
The decoder assumes a single bit depth for all the planes while
the specification allows different bit depths for luma and chroma.

Avoid the possible problems described in CVE-2013-2277

Conflicts:
	libavcodec/h264.c
2013-05-09 11:20:11 +02:00
Reinhard Tartler
7e6625a9af xxan: fix invalid memory access in xan_decode_frame_type0()
The loop a few lines below the xan_unpack() call accesses up to
dec_size * 2 bytes into y_buffer, so dec_size must be limited to
buffer_size / 2.

CC:libav-stable@libav.org
(cherry picked from commit 8a49d2bcbe)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit 62a657de16)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>

Conflicts:
	libavcodec/xxan.c
2013-05-09 11:20:11 +02:00
Anton Khirnov
f13f6f82c6 wmadec: require block_align to be set.
Avoids an infinite loop in the calling programs with decoder not
consuming any input and not returning output.

CC:libav-stable@libav.org
(cherry picked from commit ea1136baaf)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit c1f479e8df)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-05-09 11:20:11 +02:00
Michael Niedermayer
b6783b8826 indeo5: update AVCodecContext width/height on size change
Fixes CVE-2012-2787

Note that in 0.7, there is only indeo 5, no indeo 4 decoder

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Anton Khirnov <anton@khirnov.net>
(cherry picked from commit b146d74730)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit 2bc1e4fcb9)

Conflicts:

	libavcodec/ivi_common.c
2013-05-09 11:20:10 +02:00
Carl Eugen Hoyos
e2d529424f Fix type of shared flac table ff_flac_blocksize_table[].
Fixes ticket #2533.
(cherry picked from commit a07ac1f788)
2013-05-05 20:39:34 +02:00
Michael Niedermayer
dfeef3a209 cook: check js_subband_start for validity
Fixes out of array read

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit c69315a5de)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-02-21 02:00:40 +01:00
Michael Niedermayer
2b6f3be082 avcodec_align_dimensions2: Ensure cinepak has large enough buffers.
This is partly redundant with the following patches, but its safer

Found-by: u-bo1b@0w.se
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit f5c00b347d)

Conflicts:

	libavcodec/utils.c

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-02-20 04:33:01 +01:00
Michael Niedermayer
17966ae5bb qdm2: increase noise_table size
This prevents out of array reads. An alternative solution would be
to check the index but this would require several checks in the
inner loops

Yet another alternative would be to change the index reset logic
but this likely would introduce a difference to the binary decoder

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 8c4aebb58d)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-02-20 01:23:54 +01:00
Michael Niedermayer
5af2fd317d wma: check byte_offset_bits
Fixes assertion failure

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 984add64a4)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-02-20 01:23:54 +01:00
Michael Niedermayer
8aedb75156 tiff: check bppcount
Fixes division by 0

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit a34418c28e)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-02-20 01:23:54 +01:00
Michael Niedermayer
1fd86f9a21 vqavideo: fix return type
Fixes Ticket2281

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-02-20 01:23:54 +01:00
Michael Niedermayer
41eda87048 pngdec/filter: dont access out of array elements at the end
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 1ac0fa50ef)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-02-17 23:23:05 +01:00
Michael Niedermayer
e6ac11e417 aacdec: check channel count
Prevent out of array accesses

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 96f452ac64)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-02-17 23:22:44 +01:00
Michael Niedermayer
2cac35086c vqavideo: check chunk sizes before reading chunks
Fixes out of array writes

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit ab6c9332bf)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-02-17 23:22:42 +01:00
Michael Niedermayer
af343f5cdd eamad: fix out of array accesses
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 63ac64864c)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-02-17 23:22:21 +01:00
Michael Niedermayer
391e0fc6c9 roqvideodec: check dimensions validity
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 3ae6104511)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-02-17 23:22:18 +01:00
Michael Niedermayer
caeca53a09 qdm2: check array index before use, fix out of array accesses
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit a7ee6281f7)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-02-17 23:22:13 +01:00
Michael Niedermayer
760929117d alsdec: check block length
Fix writing over the end

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 0ceca269b6)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-02-17 23:22:12 +01:00
Michael Niedermayer
acada70ffb Merge remote-tracking branch 'qatar/release/0.7' into release/0.8
* qatar/release/0.7:
  Update changelog for 0.7.7 release
  mpeg12: do not decode extradata more than once.
  indeo4/5: check empty tile size in decode_mb_info().
  dfa: improve boundary checks in decode_dds1()
  indeo5dec: Make sure we have had a valid gop header.
  rv34: error out on size changes with frame threading

Conflicts:
	Changelog

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-02-14 14:12:14 +01:00
Michael Niedermayer
4f91c45644 huffyuvdec: Skip len==0 cases
Fixes vlc decoding for hypothetical files that would contain such cases.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 0dfc01c2bb)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 5ff41ffeb4cb9ea6df49757dc859619dc3d3ab4f)

Conflicts:

	libavcodec/huffyuv.c
(cherry picked from commit 9bc70fe1ae50fd2faa0b9429d47cfbda01a92ebc)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-02-14 14:11:58 +01:00