Commit graph

29217 commits

Author SHA1 Message Date
Michael Niedermayer
a0605792c2 avcodec/mpegaudio_parser: fix off by 1 error in bitrate calculation
Fixes Ticket3918

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 817663897e)
2014-11-06 01:06:17 +01:00
Michael Niedermayer
4b8cb3fe51 avcodec/h264_parser: rewrite the parse_nal_units() loop logic based on h264.c
Fixes Ticket4011

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 69a9a90d2e)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-01 17:11:41 +01:00
Michael Niedermayer
48bf926bad avcodec/h264: simplify find_start_code()
this also uses avpriv_find_start_code(), though no speed change is expected as
the area searched is generally small

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 3b678da5e3)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-01 17:09:16 +01:00
Michael Niedermayer
ab43652c67 Move get_avc_nalsize() and find_start_code() to h264.h
This allows sharing them with the h264 parser

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 4898440f6b)

Conflicts:

	libavcodec/h264.c
2014-11-01 17:05:35 +01:00
Clément Bœsch
193b949f71 avcodec/mjpegdec: Fix chroma width rounding
Fixes vertical line at the right side
Fixes Ticket 3929

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit b96d864fd6)
2014-11-01 15:51:41 +01:00
Michael Niedermayer
9395a3a96b avcodec/svq1dec: zero terminate embedded message before printing
Fixes out of array access
Fixes: asan_stack-oob_49b1e5_10_009.mov
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit e91ba2efa9)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-30 18:38:45 +01:00
Michael Niedermayer
25d2a4dde7 avcodec/cook: check that the subpacket sizes fit in block_align
Fixes out of array read
Fixes: asan_heap-oob_fb5c50_19_018.rmvb
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 10e32618ac)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-30 17:23:37 +01:00
Michael Niedermayer
bfee1e9072 avcodec/g2meet: check tile dimensions to avoid integer overflow
Fixes out of array access
Fixes: asan_heap-oob_12a55d3_30_029.wmv
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 32e666c354)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-30 01:51:30 +01:00
Michael Niedermayer
0db579445f avcodec/utils: Align dimensions by at least their chroma sub-sampling factors.
Fixes: out of array accesses
Fixes: asan_heap-oob_112c6b3_13_012.avi
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit df74811cd5)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-29 16:23:36 +01:00
Michael Niedermayer
e3275571c7 avcodec/dnxhddec: treat pix_fmt like width/height
Fixes out of array accesses
Fixes: asan_heap-oob_22c9a39_16_015.mxf
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit f3c0e0bf6f)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-29 01:31:49 +01:00
Michael Niedermayer
7b7d12ea04 avcodec/dxa: check dimensions
Fixes out of array access
Fixes: asan_heap-oob_11222fb_21_020.dxa
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit e70312dfc2)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-28 16:05:18 +01:00
Michael Niedermayer
e26fd791ef avcodec/dirac_arith: fix integer overflow
Fixes: asan_heap-oob_1078676_9_008.drc
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 39680cacee)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-28 15:00:05 +01:00
Michael Niedermayer
ad98b2891c avcodec/diracdec: Tighter checks on CODEBLOCKS_X/Y
Fixes very long but finite loop
Fixes: asan_heap-oob_107866c_42_041.drc
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 5145d22b88)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-28 15:00:04 +01:00
Michael Niedermayer
1ec6a3c768 avcodec/diracdec: Use 64bit in calculation of codeblock coordinates
Fixes integer overflow
Fixes out of array read
Fixes: asan_heap-oob_107866c_42_041.drc
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 526886e606)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-28 15:00:02 +01:00
Michael Niedermayer
5c791b1c9c avcodec/sgidec: fix count check
Fixes: asan_heap-oob_22b30d4_39_038.sgi
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit a050cf0c45)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-27 21:16:55 +01:00
Michael Niedermayer
4a03c31728 avcodec/sgidec: fix linesize for 16bit
Fixes: asan_heap-oob_22b30d4_39_038.sgi
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 3b20ed8548)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-27 21:16:54 +01:00
Christophe Gisquet
c3c8857263 avcodec/tiffenc: properly compute packet size
The bytes per row is a better indication of it.

Helps resolving ticket #3874 by fixing ffmpeg's encoder and transforming
the issue in a issue with non-compliant decoders. ffmpeg's one is ok,
but unfortunately, many others aren't handling correctly unusual chroma
samplings.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 0e8bfd3c93)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-22 22:31:44 +02:00
Christophe Gisquet
f2fba07494 utvideoenc: properly set slice height/last line
Mimic decoder and obey sampling.

Does not affect fate tests for utvideo.
Fixes ticket #3949.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit cb530dda7d)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-10 20:30:12 +02:00
Michael Niedermayer
5f7e48a113 avcodec/on2avc: Check number of channels
Fixes out of array access
Fixes: asan_heap-oob_4da4f3_7_asan_heap-oob_4da4f3_173_Xmen_avc_500.vp6

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 550f3e9df3)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-04 22:26:33 +02:00
Michael Niedermayer
e2e0c5b7f6 avcodec/hevc: fix chroma transform_add size
Fixes accessing misaligned address
Fixes: signal_sigsegv_1feb99c_10_signal_sigsegv_2d1d35c_79_cov_691940146_EXT_A_ericsson_3.bit

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit df8394c693)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-04 17:56:37 +02:00
Michael Niedermayer
279003eb9e avcodec/h264: Check mode before considering mixed mode intra prediction
Fixes out of array read
Fixes: asan_heap-oob_e476fc_2_asan_heap-oob_1333ec6_61_CAMACI3_Sony_C.jsv

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 9734a7a1de)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-04 15:04:33 +02:00
Michael Niedermayer
fdc8f4e5b4 avcodec/vorbisdec: Fix off by 1 error in ptns_to_read
Fixes read of uninitialized memory
Fixes: asan_heap-uaf_18dac2b_9_asan_heap-uaf_22eb375_208_beta3_test_small.ogg

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 8c50704ebf)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-04 04:14:20 +02:00
Michael Niedermayer
603f4ecd14 avcodec/svq3: Dont memcpy AVFrame
This avoids out of array accesses

Fixes: asan_heap-uaf_21f42e4_9_asan_heap-uaf_21f42e4_278_gl2.mov

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 075a165d27)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-04 01:15:09 +02:00
Michael Niedermayer
1ac4ae2a32 avcodec/smc: fix off by 1 error
Fixes out of array access
Fixes: asan_heap-oob_1685bf0_5_asan_heap-oob_1f35116_430_smc.mov

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit c727401aa9)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-03 23:24:14 +02:00
Michael Niedermayer
3998071039 avcodec/qpeg: fix off by 1 error in MV bounds check
Fixes out of array access
Fixes: asan_heap-oob_153760f_4_asan_heap-oob_1d7a4cf_164_VWbig6.avi

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit dd3bfe3cc1)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-03 21:32:04 +02:00
Michael Niedermayer
6fb9bfb1a3 avcodec/gifdec: factorize interleave end handling out
also change it to a loop
Fixes out of array access
Fixes: asan_heap-oob_ca5410_8_asan_heap-oob_ca5410_97_ID_LSD_Size_Less_Then_Data_Inter_3.gif

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 8f1457864b)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-03 20:28:35 +02:00
Michael Niedermayer
693faadd30 avcodec/cinepak: fix integer underflow
Fixes out of array access
Fixes: asan_heap-oob_4da0ba_6_asan_heap-oob_4da0ba_241_cvid_crash.avi

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit e7e5114c50)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-03 19:47:33 +02:00
Michael Niedermayer
9806028fbb avcodec/pngdec: Calculate MPNG bytewidth more defensively
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit e830902934)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-03 18:29:13 +02:00
Michael Niedermayer
433e15bb87 avcodec/pngdec: Check bits per pixel before setting monoblack pixel format
Fixes out of array accesses
Fixes: asan_heap-oob_14dbfcf_4_asan_heap-oob_1ce5767_179_add_method_small.png

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 3e2b745020)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-03 18:29:11 +02:00
Michael Niedermayer
c3af801c63 avcodec/tiff: more completely check bpp/bppcount
Fixes pixel format selection
Fixes out of array accesses
Fixes: asan_heap-oob_1766029_6_asan_heap-oob_20aa045_332_cov_1823216757_m2-d1d366d7965db766c19a66c7a2ccbb6b.tif

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit e1c0cfaa41)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-03 16:47:21 +02:00
Michael Niedermayer
01817d508b avcodec/mmvideo: Bounds check 2nd line of HHV Intra blocks
Fixes out of array access
Fixes: asan_heap-oob_4da4f3_8_asan_heap-oob_4da4f3_419_scene1a.mm

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 8b0e96e1f2)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-03 15:09:43 +02:00
Michael Niedermayer
b5ef1eee45 avcodec/h263dec: Fix decoding messenger.h263
Fixes http://samples.mplayerhq.hu/V-codecs/h263/h263-raw/messenger.h263
Fixes regression since b239f3f6

Found-by: Josh Allmann
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit d225b0f7aa)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-03 13:10:08 +02:00
Michael Niedermayer
e2c5f88237 avcodec/utils: Add case for jv to avcodec_align_dimensions2()
Fixes out of array accesses
Fixes: asan_heap-oob_12304aa_8_asan_heap-oob_4da4f3_300_intro.jv

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 105654e376)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-03 05:07:15 +02:00
Michael Niedermayer
57bdb3f3dd avcodec/mjpegdec: check bits per pixel for changes similar to dimensions
Fixes out of array accesses
Fixes: asan_heap-oob_16668e9_2_asan_heap-oob_16668e9_346_miss_congeniality_pegasus_mjpg.avi

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 5c378d6a6d)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-03 03:40:59 +02:00
Michael Niedermayer
5e34dded10 avcodec/jpeglsdec: Check run value more completely in ls_decode_line()
previously it could have been by 1 too large
Fixes out of array access
Fixes: asan_heap-oob_12240f5_1_asan_heap-oob_12240f5_448_t8c1e3.jls
Fixes: asan_heap-oob_12240f5_1_asan_heap-oob_12240f5_448_t8nde0.jls
Fixes: asan_heap-oob_12240fa_1_asan_heap-oob_12240fa_448_t16e3.jls

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 06e7d58410)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-03 03:40:59 +02:00
Michael Niedermayer
ca2ccd85d7 avcodec/ac3enc_template: fix out of array read
Found-by: Andreas Cadhalpun
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit d85ebea3f3)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-03 03:40:59 +02:00
Pascal Massimino
aa40bbb492 avcodec/webp: fix default palette color 0xff000000 -> 0x00000000
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit e5b3112996)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-03 03:40:59 +02:00
Gianluigi Tiesi
8e276fc96a avcodec/libilbc: support for latest git of libilbc
in the latest git commits of libilbc developers removed WebRtc_xxx typedefs

This commit uses int types instead,
it's safe to apply also for previous versions since
WebRtc_Word16 was always a typedef of int16_t and
WebRtc_UWord16 a typedef of uint16_t

Reviewed-by: Timothy Gu <timothygu99@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 59af5383c1)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-03 03:40:59 +02:00
Pascal Massimino
4a4e30a6d8 libavcodec/webp: treat out-of-bound palette index as translucent black
See https://code.google.com/p/webp/issues/detail?id=206
for a description of the problem/fix.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

This patch makes the decoder follow the recommendation of the spec.
There is some disagreement (see "[FFmpeg-devel] [PATCH]: libavcodec/webp")
about what would be best to be written in the spec, so in case the spec
is changed again, this potentially would need to be amended or reverted
(cherry picked from commit 4fd21d58a7)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-03 03:40:59 +02:00
Michael Niedermayer
1c733a440a avcodec/x86/vp9lpf: Always include x86util.asm
Fixes executable stack

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 41d82b85ab)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-03 03:40:59 +02:00
Michael Niedermayer
6952f6f39b avcodec/cabac_functions: fix "warning: UNCHECKED_BITSTREAM_READER is not defined, evaluates to 0"
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit a67b6c9541)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-03 03:40:59 +02:00
Michael Niedermayer
beb55b3981 avcodec/mpegvideo: Set err on failure in ff_mpv_common_frame_size_change()
Found-by: ubitux
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit cfce6f7efd)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-03 03:40:59 +02:00
Michael Niedermayer
80aec733ad avcodec/mpegvideo: check that the context is initialized in ff_mpv_common_frame_size_change()
The function otherwise would initialize the context without setting context_initialized
alternatively we could set context_initialized

Fixes valgrind anomalies related to ticket 3928

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 0d0f7f0ba4)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-03 03:40:59 +02:00
Michael Niedermayer
77bb6b5bcc avcodec/mpegvideo: Use "goto fail" for all error paths in ff_mpv_common_frame_size_change()
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 2762323c37)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-03 03:40:59 +02:00
Michael Niedermayer
f68395f7fc avcodec/h264: Allow partial escaping
Fixes Ticket3923

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 033a5334ba)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-03 03:40:59 +02:00
Michael Niedermayer
f903147f2d avcodec/mjpegdec: fix green line at the bottom with upscale v
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit f0d4f00f24)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-03 03:40:58 +02:00
Michael Niedermayer
9a840d5e17 avcodec/mjpegdec: fix green vertical line at the right with upscale h
Fixes Ticket3891

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 63a52ca134)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-03 03:40:58 +02:00
Michael Niedermayer
e13e928baa avcodec/snow: check coeffs for validity
Fixes deadlock
Fixes integer overflow
Fixes Ticket 3892

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 596636a474)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-03 03:40:58 +02:00
Michael Niedermayer
17a6ca7d31 avcodec/vc1dec: fix null pointer dereference
Fixes Ticket3860

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 30f680ee0a)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-03 03:16:38 +02:00
Michael Niedermayer
8a20224059 avcodec/utils: add GBRP16 to avcodec_align_dimensions2()
Fixes Ticket3869

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 3fe9e7be4c)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-03 03:16:37 +02:00