Commit graph

29196 commits

Author SHA1 Message Date
Michael Niedermayer
fdc8f4e5b4 avcodec/vorbisdec: Fix off by 1 error in ptns_to_read
Fixes read of uninitialized memory
Fixes: asan_heap-uaf_18dac2b_9_asan_heap-uaf_22eb375_208_beta3_test_small.ogg

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 8c50704ebf)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-04 04:14:20 +02:00
Michael Niedermayer
603f4ecd14 avcodec/svq3: Dont memcpy AVFrame
This avoids out of array accesses

Fixes: asan_heap-uaf_21f42e4_9_asan_heap-uaf_21f42e4_278_gl2.mov

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 075a165d27)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-04 01:15:09 +02:00
Michael Niedermayer
1ac4ae2a32 avcodec/smc: fix off by 1 error
Fixes out of array access
Fixes: asan_heap-oob_1685bf0_5_asan_heap-oob_1f35116_430_smc.mov

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit c727401aa9)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-03 23:24:14 +02:00
Michael Niedermayer
3998071039 avcodec/qpeg: fix off by 1 error in MV bounds check
Fixes out of array access
Fixes: asan_heap-oob_153760f_4_asan_heap-oob_1d7a4cf_164_VWbig6.avi

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit dd3bfe3cc1)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-03 21:32:04 +02:00
Michael Niedermayer
6fb9bfb1a3 avcodec/gifdec: factorize interleave end handling out
also change it to a loop
Fixes out of array access
Fixes: asan_heap-oob_ca5410_8_asan_heap-oob_ca5410_97_ID_LSD_Size_Less_Then_Data_Inter_3.gif

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 8f1457864b)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-03 20:28:35 +02:00
Michael Niedermayer
693faadd30 avcodec/cinepak: fix integer underflow
Fixes out of array access
Fixes: asan_heap-oob_4da0ba_6_asan_heap-oob_4da0ba_241_cvid_crash.avi

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit e7e5114c50)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-03 19:47:33 +02:00
Michael Niedermayer
9806028fbb avcodec/pngdec: Calculate MPNG bytewidth more defensively
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit e830902934)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-03 18:29:13 +02:00
Michael Niedermayer
433e15bb87 avcodec/pngdec: Check bits per pixel before setting monoblack pixel format
Fixes out of array accesses
Fixes: asan_heap-oob_14dbfcf_4_asan_heap-oob_1ce5767_179_add_method_small.png

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 3e2b745020)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-03 18:29:11 +02:00
Michael Niedermayer
c3af801c63 avcodec/tiff: more completely check bpp/bppcount
Fixes pixel format selection
Fixes out of array accesses
Fixes: asan_heap-oob_1766029_6_asan_heap-oob_20aa045_332_cov_1823216757_m2-d1d366d7965db766c19a66c7a2ccbb6b.tif

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit e1c0cfaa41)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-03 16:47:21 +02:00
Michael Niedermayer
01817d508b avcodec/mmvideo: Bounds check 2nd line of HHV Intra blocks
Fixes out of array access
Fixes: asan_heap-oob_4da4f3_8_asan_heap-oob_4da4f3_419_scene1a.mm

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 8b0e96e1f2)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-03 15:09:43 +02:00
Michael Niedermayer
b5ef1eee45 avcodec/h263dec: Fix decoding messenger.h263
Fixes http://samples.mplayerhq.hu/V-codecs/h263/h263-raw/messenger.h263
Fixes regression since b239f3f6

Found-by: Josh Allmann
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit d225b0f7aa)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-03 13:10:08 +02:00
Michael Niedermayer
e2c5f88237 avcodec/utils: Add case for jv to avcodec_align_dimensions2()
Fixes out of array accesses
Fixes: asan_heap-oob_12304aa_8_asan_heap-oob_4da4f3_300_intro.jv

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 105654e376)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-03 05:07:15 +02:00
Michael Niedermayer
57bdb3f3dd avcodec/mjpegdec: check bits per pixel for changes similar to dimensions
Fixes out of array accesses
Fixes: asan_heap-oob_16668e9_2_asan_heap-oob_16668e9_346_miss_congeniality_pegasus_mjpg.avi

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 5c378d6a6d)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-03 03:40:59 +02:00
Michael Niedermayer
5e34dded10 avcodec/jpeglsdec: Check run value more completely in ls_decode_line()
previously it could have been by 1 too large
Fixes out of array access
Fixes: asan_heap-oob_12240f5_1_asan_heap-oob_12240f5_448_t8c1e3.jls
Fixes: asan_heap-oob_12240f5_1_asan_heap-oob_12240f5_448_t8nde0.jls
Fixes: asan_heap-oob_12240fa_1_asan_heap-oob_12240fa_448_t16e3.jls

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 06e7d58410)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-03 03:40:59 +02:00
Michael Niedermayer
ca2ccd85d7 avcodec/ac3enc_template: fix out of array read
Found-by: Andreas Cadhalpun
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit d85ebea3f3)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-03 03:40:59 +02:00
Pascal Massimino
aa40bbb492 avcodec/webp: fix default palette color 0xff000000 -> 0x00000000
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit e5b3112996)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-03 03:40:59 +02:00
Gianluigi Tiesi
8e276fc96a avcodec/libilbc: support for latest git of libilbc
in the latest git commits of libilbc developers removed WebRtc_xxx typedefs

This commit uses int types instead,
it's safe to apply also for previous versions since
WebRtc_Word16 was always a typedef of int16_t and
WebRtc_UWord16 a typedef of uint16_t

Reviewed-by: Timothy Gu <timothygu99@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 59af5383c1)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-03 03:40:59 +02:00
Pascal Massimino
4a4e30a6d8 libavcodec/webp: treat out-of-bound palette index as translucent black
See https://code.google.com/p/webp/issues/detail?id=206
for a description of the problem/fix.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

This patch makes the decoder follow the recommendation of the spec.
There is some disagreement (see "[FFmpeg-devel] [PATCH]: libavcodec/webp")
about what would be best to be written in the spec, so in case the spec
is changed again, this potentially would need to be amended or reverted
(cherry picked from commit 4fd21d58a7)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-03 03:40:59 +02:00
Michael Niedermayer
1c733a440a avcodec/x86/vp9lpf: Always include x86util.asm
Fixes executable stack

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 41d82b85ab)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-03 03:40:59 +02:00
Michael Niedermayer
6952f6f39b avcodec/cabac_functions: fix "warning: UNCHECKED_BITSTREAM_READER is not defined, evaluates to 0"
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit a67b6c9541)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-03 03:40:59 +02:00
Michael Niedermayer
beb55b3981 avcodec/mpegvideo: Set err on failure in ff_mpv_common_frame_size_change()
Found-by: ubitux
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit cfce6f7efd)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-03 03:40:59 +02:00
Michael Niedermayer
80aec733ad avcodec/mpegvideo: check that the context is initialized in ff_mpv_common_frame_size_change()
The function otherwise would initialize the context without setting context_initialized
alternatively we could set context_initialized

Fixes valgrind anomalies related to ticket 3928

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 0d0f7f0ba4)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-03 03:40:59 +02:00
Michael Niedermayer
77bb6b5bcc avcodec/mpegvideo: Use "goto fail" for all error paths in ff_mpv_common_frame_size_change()
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 2762323c37)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-03 03:40:59 +02:00
Michael Niedermayer
f68395f7fc avcodec/h264: Allow partial escaping
Fixes Ticket3923

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 033a5334ba)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-03 03:40:59 +02:00
Michael Niedermayer
f903147f2d avcodec/mjpegdec: fix green line at the bottom with upscale v
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit f0d4f00f24)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-03 03:40:58 +02:00
Michael Niedermayer
9a840d5e17 avcodec/mjpegdec: fix green vertical line at the right with upscale h
Fixes Ticket3891

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 63a52ca134)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-03 03:40:58 +02:00
Michael Niedermayer
e13e928baa avcodec/snow: check coeffs for validity
Fixes deadlock
Fixes integer overflow
Fixes Ticket 3892

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 596636a474)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-03 03:40:58 +02:00
Michael Niedermayer
17a6ca7d31 avcodec/vc1dec: fix null pointer dereference
Fixes Ticket3860

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 30f680ee0a)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-03 03:16:38 +02:00
Michael Niedermayer
8a20224059 avcodec/utils: add GBRP16 to avcodec_align_dimensions2()
Fixes Ticket3869

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 3fe9e7be4c)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-03 03:16:37 +02:00
Michael Niedermayer
29ee8b72c4 avcodec: fix aac/ac3 parser bitstream buffer size
Buffers containing copies of the AAC and AC3 header bits were not padded
before parsing, violating init_get_bits() buffer padding requirement,
leading to potential buffer read overflows.
This change adds FF_INPUT_BUFFER_PADDING_SIZE bytes to the bit buffer
for parsing the header in each of aac_parser.c and ac3_parser.c.

Based on patch by: Matt Wolenetz <wolenetz@chromium.org>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit fccd85b9f3)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-03 03:16:37 +02:00
Christophe Gisquet
25864cf562 wavpackenc: proper buffer allocation
The allocation didn't account for headers, that can be easily 79 bytes.
As a result, buffers allocated for a few samples (e.g. 5 in the original
bug) could be undersized.

Fixed ticket #2881.

Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 2ba58bec20)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-03 02:51:57 +02:00
Christophe Gisquet
f74206cb40 wavpack: report if there is no bits left
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 11a39bdf53)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-03 02:51:57 +02:00
Michael Niedermayer
3b6bde3b3d avcodec/h264_mb: fix grayscale only decoding with weighted prediction
Fixes Ticket3412

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 94f60b6544)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-08-17 15:09:05 +02:00
Michael Niedermayer
10c2d22ba1 avcodec/mjpegdec: Support AV_PIX_FMT_YUV420P16 with upscale_h
Fixes assertion failure
Fixes: test42f.jpg
Found-by: Piotr Bandurski <ami_stuff@o2.pl>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 5c7899a483)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-08-14 16:53:36 +02:00
Christophe Gisquet
35738e5898 proresenc_kostya: properly account for alpha
The packet buffer allocation considered as dct-coded, while it is
actually run-coded and thus requires a larger buffer.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 117bc8e6ff)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-08-14 15:44:54 +02:00
Christophe Gisquet
60bfa9154d proresenc_kostya: report buffer overflow
If the allocated size, despite best efforts, is too small, exit
with the appropriate error.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 52b81ff463)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-08-12 02:40:15 +02:00
Christophe Gisquet
9794727ccd proresenc_kostya: remove unneeded parameters
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit bf10f09bcc)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-08-12 02:40:14 +02:00
Michael Niedermayer
11420649d0 avcodec/snow: fix null pointer dereference in cleanup after allocation failure
Fixes: snowf.avi
Found-by: Piotr Bandurski <ami_stuff@o2.pl>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 9a162146ca)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-08-11 15:45:42 +02:00
Michael Niedermayer
dbf5d7e5cd avcodec/huffyuvdec: fix overread checks
Fixes: ffvhuff_f.avi
Found-by: Piotr Bandurski <ami_stuff@o2.pl>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit ba47d519e5)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-08-11 15:32:00 +02:00
Michael Niedermayer
6badd558ce avcodec/get_bits: add BITS_LEFT() for finding the bits left with an opened reader
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 11512d70fa)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-08-11 15:31:59 +02:00
Michael Niedermayer
a1fe3b4150 avcodec/vc1dec: do not crash when flushing without an allocated frame
Fixes Ticket3837
Found-by: Piotr Bandurski <ami_stuff@o2.pl>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 6801eb0a09)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-08-11 02:26:29 +02:00
Michael Niedermayer
fc57959fd5 avcodec/iff: Factorize av_pix_fmt_desc_get() out
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 9bcdb98839)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-08-10 22:40:50 +02:00
Michael Niedermayer
83956309cc avcodec/iff: check pixfmt for rgb8 / rgbn
Fixes out of array access

Found-by: Piotr Bandurski <ami_stuff@o2.pl>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 3539d6c63a)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-08-10 22:40:50 +02:00
Felix Abecassis
6928193493 h264: fix interpretation of interleved stereo modes
Column and row frame packing arrangements were inverted.

Signed-off-by: Vittorio Giovara <vittorio.giovara@gmail.com>
(cherry picked from commit a7e541c992)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-08-10 22:40:50 +02:00
Anton Khirnov
a72b7286e6 cdgraphics: do not return 0 from the decode function
0 means no data consumed, so it can trigger an infinite loop in the
caller.

CC:libav-stable@libav.org
(cherry picked from commit c7d9b473e2)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-08-10 22:40:50 +02:00
Michael Niedermayer
fdfe94f4b1 avcodec/wavpackenc: Fix log2sample() result value
Found-by: CSA
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit e706fe7640)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-08-10 22:40:50 +02:00
Michael Niedermayer
638c3aca64 avcodec/dvdsub_parser: print message if packet is smaller than the packet size field
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit bcc898dd26)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-08-02 01:27:04 +02:00
Michael Niedermayer
76d59f1b34 avcodec/dvdsub_parser: Check buf_size before reading 32bit packet size
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 81c1657a59)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-08-02 01:27:03 +02:00
Michael Niedermayer
aa0a8ef50e avcodec/dvdsub_parser: never return 0 when the input isnt 0
Fixes a infinite loop
Fixes Ticket3804

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit cfdb30d2f1)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-08-02 01:27:02 +02:00
Michael Niedermayer
d6173ae341 avcodec/mpeg12dec: fix support for interlaced mpeg2 with missing last slice
Fixes Ticket3809

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit f95298c913)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-08-01 21:20:43 +02:00