Commit graph

22204 commits

Author SHA1 Message Date
Michael Niedermayer
3ee967c1d8 msrledec: merge switches
More speedup and fixes 'may be used uninitialized in this function' warnings

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit d2e0a276d5)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-03-14 04:58:01 +01:00
Michael Niedermayer
e44f89371c msrledec: move loop into switch
speeds up code and allows more simplifications

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit dbaae33c2c)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-03-14 04:57:44 +01:00
Michael Niedermayer
e586e4d93b msrledec: move output pointer test up
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit c2992b7053)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-03-14 04:57:36 +01:00
Michael Niedermayer
f156dc54f8 mpegaudio_parser: fix off by 1 error
See:
commit 29d8cd265a
Author: Alexander Kojevnikov <alexander@kojevnikov.com>
Date:   Tue Feb 26 21:47:11 2013 -0800

    mp3dec: Fix VBR bit rate parsing

    When parsing the Xing/Info tag, don't set the bit rate if it's an Info tag.

    When parsing the stream, don't override the bit rate if it's already set,
    otherwise calculate the mean bit rate from parsed frames. This way, the bit
    rate will be set correctly both for CBR and VBR streams.

    Signed-off-by: Alexander Kojevnikov <alexander@kojevnikov.com>
    Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-03-14 04:42:08 +01:00
Michael Niedermayer
685f50b374 Merge remote-tracking branch 'qatar/release/9' into release/1.1
* qatar/release/9:
  eamad: allocate a dummy reference frame when the real one is missing
  libmp3lame: use the correct remaining buffer size when flushing
  png: use av_mallocz_array() for the zlib zalloc function
  wmaprodec: require block_align to be set.
  ffv1: fix calculating slice dimensions for version 2
  xxan: fix invalid memory access in xan_decode_frame_type0()
  wmadec: require block_align to be set.
  ivi_common: do not call MC for intra frames when dc_transform is unset

Conflicts:
	libavcodec/ffv1dec.c
	libavcodec/ivi_common.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-03-14 04:27:35 +01:00
Michael Niedermayer
6086a4d74d Merge commit '747fbe0c21' into release/1.1
* commit '747fbe0c21':
  roqvideodec: fix a potential infinite loop in roqvideo_decode_frame().
  mp3dec: Fix VBR bit rate parsing
  wmaprodec: return an error, not 0, when the input is too small.
  vmdaudio: fix invalid reads when packet size is not a multiple of chunk size
  h264: check for luma and chroma bit dept being equal
  Prepare for 9.4 Release

Conflicts:
	RELEASE
	libavcodec/vmdav.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-03-14 02:49:31 +01:00
Anton Khirnov
0cb3cab343 eamad: allocate a dummy reference frame when the real one is missing
Fixes invalid reads when the first frame is not an I-frame.

CC:libav-stable@libav.org
(cherry picked from commit 7b89cd20d8)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2013-03-09 19:05:42 +01:00
Justin Ruggles
b77d9cbbd5 libmp3lame: use the correct remaining buffer size when flushing
CC:libav-stable@libav.org
(cherry picked from commit e984f47873)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-03-09 18:10:16 +01:00
Justin Ruggles
905f5c8a1e png: use av_mallocz_array() for the zlib zalloc function
Fixes valgrind uninitialized memory errors when decoding png.

CC:libav-stable@libav.org
(cherry picked from commit 486f0b0cfc)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-03-09 18:10:16 +01:00
Anton Khirnov
20373a66ec wmaprodec: require block_align to be set.
Avoids an infinite loop in the calling programs with decoder not
consuming any input and not returning output.

CC:libav-stable@libav.org
(cherry picked from commit cacad1c058)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-03-09 18:10:16 +01:00
Anton Khirnov
d48da91373 ffv1: fix calculating slice dimensions for version 2
It got broken in 0f13cd3187.

CC:libav-stable@libav.org
(cherry picked from commit d243896987)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-03-09 18:10:16 +01:00
Anton Khirnov
62a657de16 xxan: fix invalid memory access in xan_decode_frame_type0()
The loop a few lines below the xan_unpack() call accesses up to
dec_size * 2 bytes into y_buffer, so dec_size must be limited to
buffer_size / 2.

CC:libav-stable@libav.org
(cherry picked from commit 8a49d2bcbe)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-03-09 18:10:16 +01:00
Anton Khirnov
747fbe0c21 roqvideodec: fix a potential infinite loop in roqvideo_decode_frame().
When there is just 1 byte remanining in the buffer, nothing will be read
and the loop will continue forever. Check that there are at least 8
bytes, which are always read at the beginning.

CC:libav-stable@libav.org
(cherry picked from commit 3e2f200237)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-03-09 18:01:09 +01:00
Anton Khirnov
c1f479e8df wmadec: require block_align to be set.
Avoids an infinite loop in the calling programs with decoder not
consuming any input and not returning output.

CC:libav-stable@libav.org
(cherry picked from commit ea1136baaf)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-03-09 18:01:09 +01:00
Alexander Kojevnikov
d3b40af01f mp3dec: Fix VBR bit rate parsing
When parsing the Xing/Info tag, don't set the bit rate if it's an Info tag.

When parsing the stream, don't override the bit rate if it's already set,
otherwise calculate the mean bit rate from parsed frames. This way, the bit
rate will be set correctly both for CBR and VBR streams.

CC:libav-stable@libav.org

Signed-off-by: Anton Khirnov <anton@khirnov.net>
(cherry picked from commit eae0879d96)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-03-09 18:01:09 +01:00
Anton Khirnov
74880e78d8 ivi_common: do not call MC for intra frames when dc_transform is unset
CC:libav-stable@libav.org
(cherry picked from commit 3ba40ebb6c)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-03-09 18:01:09 +01:00
Anton Khirnov
60dd8b5733 wmaprodec: return an error, not 0, when the input is too small.
Returning 0 may result in an infinite loop in valid calling programs. A
decoder should never return 0 without producing any output.

CC:libav-stable@libav.org
(cherry picked from commit 4c0080b7e7)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-03-09 18:01:09 +01:00
Anton Khirnov
77cf052e39 vmdaudio: fix invalid reads when packet size is not a multiple of chunk size
CC:libav-stable@libav.org
(cherry picked from commit f86d66bcfa)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-03-09 18:01:09 +01:00
Luca Barbato
146eac0a0c h264: check for luma and chroma bit dept being equal
The decoder assumes a single bit depth for all the planes
while the specification allows different bit depths for luma
and chroma.

Avoid the possible problems described in CVE-2013-2277

CC: libav-stable@libav.org
(cherry picked from commit 4987faee78)

Conflicts:

	libavcodec/h264.c
2013-03-09 18:01:09 +01:00
Michael Niedermayer
41313bdcc5 aacsbr: Check for envelope scalefactors overflowing
This prevents various values from becoming stuck at NAN and
output to become silent
If someone knows a cleaner solution, thats welcome!

Fixes Ticket2335

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 8978c743fb)
2013-03-08 20:03:42 +01:00
Michael Niedermayer
088ba9bc3e psymodel: dont apply lowpass filters with a cutoff close to the nyquist
The IIR filter numerically diverges in such cases, this could easily be
fixed but would make the filter slower on some platforms

Fixes Ticket2246

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit fee5da6b0a)
2013-03-07 19:58:05 +01:00
Michael Niedermayer
7c8beec48c buildsys: only include log2_tab per library for shared builds
Fix linking failures with -all_load due to multiple log2_tabs

Signed-off-by: Carl Eugen Hoyos <cehoyos@ag.or.at>
(cherry picked from commit 03148fd174)
2013-03-05 01:17:55 +01:00
Michael Niedermayer
992957ac30 Merge remote-tracking branch 'qatar/release/9' into release/1.1
* qatar/release/9:
  update Changelog
  h264: set ref_count to 0 for intra slices.
  h264: on reference overflow, reset the reference count to 0, not 1.
  flvdec: Check the return value of a malloc

Conflicts:
	Changelog
	libavcodec/h264.c
	libavformat/flvdec.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-03-03 12:15:14 +01:00
Michael Niedermayer
b3c8fd1f0e Merge commit '1b0082eabc' into release/1.1
* commit '1b0082eabc':
  flvdec: Don't read the VP6 header byte when setting codec type based on metadata
  vorbisdec: Accept 0 amplitude_bits
  vorbisdec: Error on bark_map_size equal to 0.
  vorbisdec: Add missing checks
  ac3dec: validate channel output mode against channel count

Conflicts:
	libavcodec/ac3dec.c
	libavformat/flvdec.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-03-03 11:56:42 +01:00
Anton Khirnov
704952fee5 h264: set ref_count to 0 for intra slices.
CC:libav-stable@libav.org
(cherry picked from commit 437211ae73)

Fixes deadlocks waiting for non-existing references with some fuzzed files.

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-03-02 11:20:59 +01:00
Anton Khirnov
b6f5a1ca58 h264: on reference overflow, reset the reference count to 0, not 1.
Since decode_slice_header() returns before the reference lists are
constructed, there are zero valid references.

CC:libav-stable@libav.org
(cherry picked from commit 668e16a0dd)

Conflicts:

	libavcodec/h264.c
2013-03-02 11:20:59 +01:00
Luca Barbato
c6c4dc6935 vorbisdec: Accept 0 amplitude_bits
The specification does not prevent an encoder to write the amplitude 0
as 0 amplitude_bits.

Our get_bits() implementation might not support a zero sized read
properly, thus the additional branch.
(cherry picked from commit 23bd9ef4b2)

Conflicts:

	libavcodec/vorbisdec.c
2013-02-26 20:21:01 +01:00
Michael Niedermayer
494ddd377a vorbisdec: Error on bark_map_size equal to 0.
The value is used to calculate output LSP curve and a division by zero
and out of array accesses would occur.

CVE-2013-0894

CC: libav-stable@libav.org

Reported-by: Dale Curtis <dalecurtis@chromium.org>
Found-by: inferno@chromium.org
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
(cherry picked from commit 11dcecfcca)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-02-26 20:21:01 +01:00
Luca Barbato
37e99e384e vorbisdec: Add missing checks
Rate and order must not be 0 even if the specification does not say that
explicitly.
(cherry picked from commit 5b47c19bfd)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-02-26 20:21:01 +01:00
Justin Ruggles
73d6f4651e ac3dec: validate channel output mode against channel count
Damaged frames can lead to a mismatch, which can cause a segfault
due to using an incorrect channel mapping.

CC:libav-stable@libav.org
(cherry picked from commit d7c450436f)

Conflicts:

	libavcodec/ac3dec.c
2013-02-26 20:21:01 +01:00
Michael Niedermayer
6e8ed38fab aac: reconfigure output on pop
Fixes Ticket1918

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 6f77122bf5)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-02-26 03:39:44 +01:00
Michael Niedermayer
f64e4a8c9a Merge remote-tracking branch 'qatar/release/9' into release/1.1
* qatar/release/9:
  doc: developer: Allow tabs in the vim configuration for Automake files
  doc: filters: Correct BNF FILTER description
  Prepare for 9.3 Release
  update Changelog
  cavs: initialize various context tables to 0
  4xm: check the return value of read_huffman_tables().
  qtrle: add more checks against pixel_ptr being negative.
  mlpdec: do not try to allocate a zero-sized output buffer.
  av_memcpy_backptr: avoid an infinite loop for back = 0
  flicvideo: avoid an infinite loop in byte run compression
  lagarith: avoid infinite loop in lag_rac_refill()
  mov: use the format context for logging.
  loco: check that there is data left after decoding a plane.
  update Changelog
  x86: h264: Don't use redzone in AVX h264_deblock on Win64

Conflicts:
	Changelog
	RELEASE
	libavcodec/4xm.c
	libavcodec/loco.c
	libavcodec/qtrle.c
	libavutil/mem.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-02-26 03:09:41 +01:00
James Almer
d92a7870d7 lavc/bink: Chech for malloc failure
Based on commit 8ab2173ed1
2013-02-25 05:53:20 -03:00
Anton Khirnov
77493bfd97 cavs: initialize various context tables to 0
Avoids crashes with corrupted files.

CC:libav-stable@libav.org
(cherry picked from commit 4f3b058c84)

Conflicts:

	libavcodec/cavs.c

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-02-23 14:46:54 +01:00
Anton Khirnov
bb3f1cad17 4xm: check the return value of read_huffman_tables().
CC:libav-stable@libav.org
(cherry picked from commit 8097fc9a2d)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-02-23 14:41:24 +01:00
Anton Khirnov
a6403a3b69 qtrle: add more checks against pixel_ptr being negative.
CC:libav-stable@libav.org
(cherry picked from commit e106592447)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-02-23 14:41:10 +01:00
Anton Khirnov
e2cf32ca5f mlpdec: do not try to allocate a zero-sized output buffer.
CC:libav-stable@libav.org
(cherry picked from commit 0dff40bfb9)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-02-23 14:40:48 +01:00
Anton Khirnov
612b28194b flicvideo: avoid an infinite loop in byte run compression
When byte_run is 0, pixel_countdown is not touched and the loop will run
forever.

CC:libav-stable@libav.org
(cherry picked from commit ddfe1246d9)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-02-23 14:40:13 +01:00
Anton Khirnov
8bce2c60b8 lagarith: avoid infinite loop in lag_rac_refill()
range == 0 happens with corrupted files

CC:libav-stable@libav.org
(cherry picked from commit de6dfa2bb8)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-02-23 14:40:04 +01:00
Anton Khirnov
b786ddc0f2 loco: check that there is data left after decoding a plane.
CC:libav-stable@libav.org
(cherry picked from commit 067432c1c9)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-02-23 14:39:19 +01:00
Matt Wolenetz
5bed920971 Fix Win64 AVX h264_deblock by not using redzone on Win64
Thanks-to: "Ronald S. Bultje" <rsbultje@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 82a4a4e7ca)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-02-23 01:47:05 +01:00
Michael Niedermayer
02d1efdd5b h264: check that luma and chroma depth match
Fixes out of array access

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit bdeb61ccc6)

Conflicts:

	libavcodec/h264_ps.c

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-02-22 22:53:11 +01:00
Michael Niedermayer
469cb61193 avcodec_decode_audio4: check got_frame_ptr before handling initial skip
Fixes out of array accesses

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 8a6449167a)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-02-22 22:53:11 +01:00
Michael Niedermayer
a642be972d h264: ensure that get_format() is called when changing format but not otherwise.
Fixes Ticket2288

Tested-by: Stefano Pigozzi <stefano.pigozzi@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 04220b473e)

Conflicts:

	libavcodec/h264.c
2013-02-22 22:53:11 +01:00
Matt Wolenetz
bc9d341be8 x86: h264: Don't use redzone in AVX h264_deblock on Win64
This fixes crashes in chromium on win64 on machines with AVX
(crashes that apparently aren't triggered by fate).

Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit 311443f6c7)

Signed-off-by: Martin Storsjö <martin@martin.st>
2013-02-22 23:48:35 +02:00
Michael Niedermayer
80ddf7889e Merge remote-tracking branch 'qatar/release/9' into release/1.1
* qatar/release/9:
  doc: Fix some obsolete references to av* tools as ff* tools
  vqavideo: check chunk sizes before reading chunks
  roqvideodec: check dimensions validity
  qdm2: check array index before use, fix out of array accesses
  mpegvideo: Do REBASE_PICTURE with byte pointers

Conflicts:
	libavcodec/qdm2.c
	libavcodec/roqvideodec.c
	libavcodec/vqavideo.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-02-22 22:44:15 +01:00
Carl Eugen Hoyos
4be63111d1 Fix bits_per_coded_sample when encoding png with frame-level multithreading.
Fixes ticket #2290.
(cherry picked from commit c4dc6c4c86)
2013-02-21 09:04:05 +01:00
Michael Niedermayer
ab434bf0d0 vqavideo: check chunk sizes before reading chunks
Fixes out of array writes

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit ab6c9332bf)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 13093f9767)

CC: libav-stable@libav.org

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit f7d18deb73)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-02-19 08:10:24 +01:00
Michael Niedermayer
52b18c1fde roqvideodec: check dimensions validity
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 3ae6104511)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit fee26d352a)

CC: libav-stable@libav.org

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit 488f87be87)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-02-19 08:10:03 +01:00
Michael Niedermayer
0b2b8ab979 qdm2: check array index before use, fix out of array accesses
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

(cherry picked from commit a7ee6281f7)

CC: libav-stable@libav.org

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit 39bec05ed4)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2013-02-19 08:09:48 +01:00