Commit graph

24816 commits

Author SHA1 Message Date
Michael Niedermayer
b4e0acfa04 avcodec/smc: fix off by 1 error
Fixes out of array access
Fixes: asan_heap-oob_1685bf0_5_asan_heap-oob_1f35116_430_smc.mov

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit c727401aa9)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-03 23:24:23 +02:00
Michael Niedermayer
d36bb362a6 avcodec/qpeg: fix off by 1 error in MV bounds check
Fixes out of array access
Fixes: asan_heap-oob_153760f_4_asan_heap-oob_1d7a4cf_164_VWbig6.avi

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit dd3bfe3cc1)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-03 21:32:12 +02:00
Michael Niedermayer
be7105dff6 avcodec/gifdec: factorize interleave end handling out
also change it to a loop
Fixes out of array access
Fixes: asan_heap-oob_ca5410_8_asan_heap-oob_ca5410_97_ID_LSD_Size_Less_Then_Data_Inter_3.gif

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 8f1457864b)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-03 20:28:43 +02:00
Michael Niedermayer
97fdbd12f9 avcodec/cinepak: fix integer underflow
Fixes out of array access
Fixes: asan_heap-oob_4da0ba_6_asan_heap-oob_4da0ba_241_cvid_crash.avi

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit e7e5114c50)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-03 19:47:40 +02:00
Michael Niedermayer
db48767d7c avcodec/pngdec: Calculate MPNG bytewidth more defensively
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit e830902934)

Conflicts:

	libavcodec/pngdec.c
2014-10-03 18:35:17 +02:00
Michael Niedermayer
ae81d9a7da avcodec/pngdec: Check bits per pixel before setting monoblack pixel format
Fixes out of array accesses
Fixes: asan_heap-oob_14dbfcf_4_asan_heap-oob_1ce5767_179_add_method_small.png

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 3e2b745020)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-03 18:33:44 +02:00
Michael Niedermayer
abbfc4d87e avcodec/tiff: more completely check bpp/bppcount
Fixes pixel format selection
Fixes out of array accesses
Fixes: asan_heap-oob_1766029_6_asan_heap-oob_20aa045_332_cov_1823216757_m2-d1d366d7965db766c19a66c7a2ccbb6b.tif

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit e1c0cfaa41)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-03 16:47:32 +02:00
Michael Niedermayer
16775c7aaa avcodec/mmvideo: Bounds check 2nd line of HHV Intra blocks
Fixes out of array access
Fixes: asan_heap-oob_4da4f3_8_asan_heap-oob_4da4f3_419_scene1a.mm

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 8b0e96e1f2)

Conflicts:

	libavcodec/mmvideo.c
2014-10-03 15:10:48 +02:00
Michael Niedermayer
ac82e318bb avcodec/utils: Add case for jv to avcodec_align_dimensions2()
Fixes out of array accesses
Fixes: asan_heap-oob_12304aa_8_asan_heap-oob_4da4f3_300_intro.jv

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 105654e376)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-03 05:07:29 +02:00
Michael Niedermayer
49d69844f5 avcodec/mjpegdec: check bits per pixel for changes similar to dimensions
Fixes out of array accesses
Fixes: asan_heap-oob_16668e9_2_asan_heap-oob_16668e9_346_miss_congeniality_pegasus_mjpg.avi

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 5c378d6a6d)

Conflicts:

	libavcodec/mjpegdec.c
(cherry picked from commit 94371a404c663c3dae3d542fa43951567ab67f82)

Conflicts:

	libavcodec/mjpegdec.c
2014-10-03 02:26:25 +02:00
Michael Niedermayer
3f9a148022 avcodec/jpeglsdec: Check run value more completely in ls_decode_line()
previously it could have been by 1 too large
Fixes out of array access
Fixes: asan_heap-oob_12240f5_1_asan_heap-oob_12240f5_448_t8c1e3.jls
Fixes: asan_heap-oob_12240f5_1_asan_heap-oob_12240f5_448_t8nde0.jls
Fixes: asan_heap-oob_12240fa_1_asan_heap-oob_12240fa_448_t16e3.jls

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 06e7d58410)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-03 02:25:22 +02:00
Michael Niedermayer
52fd0cda2c avcodec/ac3enc_template: fix out of array read
Found-by: Andreas Cadhalpun
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit d85ebea3f3)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-09-27 20:58:41 +02:00
Gianluigi Tiesi
f238de1990 avcodec/libilbc: support for latest git of libilbc
in the latest git commits of libilbc developers removed WebRtc_xxx typedefs

This commit uses int types instead,
it's safe to apply also for previous versions since
WebRtc_Word16 was always a typedef of int16_t and
WebRtc_UWord16 a typedef of uint16_t

Reviewed-by: Timothy Gu <timothygu99@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 59af5383c1)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-09-21 17:04:15 +02:00
James Almer
992ce9777c x86/dsputil: add emms to ff_scalarproduct_int16_mmxext()
Also undo the changes to ra144enc.c from previous commits.
Should fix ticket #3429

Signed-off-by: James Almer <jamrial@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 9e0e1f9067)

Conflicts:

	libavcodec/ra144enc.c

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-09-20 14:14:26 +02:00
Michael Niedermayer
beb83f0a40 avcodec/mpegvideo: Set err on failure in ff_mpv_common_frame_size_change()
Found-by: ubitux
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit cfce6f7efd)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-09-07 14:37:56 +02:00
Michael Niedermayer
2b415192fd avcodec/mpegvideo: check that the context is initialized in ff_mpv_common_frame_size_change()
The function otherwise would initialize the context without setting context_initialized
alternatively we could set context_initialized

Fixes valgrind anomalies related to ticket 3928

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 0d0f7f0ba4)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-09-07 13:40:23 +02:00
Michael Niedermayer
6e70816f71 avcodec/mpegvideo: Use "goto fail" for all error paths in ff_mpv_common_frame_size_change()
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 2762323c37)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-09-07 13:40:23 +02:00
Michael Niedermayer
d76c9b5665 avcodec/snow: check coeffs for validity
Fixes deadlock
Fixes integer overflow
Fixes Ticket 3892

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 596636a474)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-09-07 13:40:22 +02:00
Michael Niedermayer
d4253b3a5b avcodec/utils: add GBRP16 to avcodec_align_dimensions2()
Fixes Ticket3869

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 3fe9e7be4c)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-09-07 13:40:22 +02:00
Michael Niedermayer
9fc7de8d80 avcodec: fix aac/ac3 parser bitstream buffer size
Buffers containing copies of the AAC and AC3 header bits were not padded
before parsing, violating init_get_bits() buffer padding requirement,
leading to potential buffer read overflows.
This change adds FF_INPUT_BUFFER_PADDING_SIZE bytes to the bit buffer
for parsing the header in each of aac_parser.c and ac3_parser.c.

Based on patch by: Matt Wolenetz <wolenetz@chromium.org>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit fccd85b9f3)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-09-07 13:40:22 +02:00
Christophe Gisquet
2958b8b86e wavpack: report if there is no bits left
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 11a39bdf53)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-08-19 18:16:23 +02:00
Christophe Gisquet
caf08defa6 proresenc_kostya: properly account for alpha
The packet buffer allocation considered as dct-coded, while it is
actually run-coded and thus requires a larger buffer.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 117bc8e6ff)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-08-14 15:45:05 +02:00
Christophe Gisquet
57a6cd8ab1 proresenc_kostya: report buffer overflow
If the allocated size, despite best efforts, is too small, exit
with the appropriate error.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 52b81ff463)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-08-12 02:40:37 +02:00
Christophe Gisquet
b469fce85d proresenc_kostya: remove unneeded parameters
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit bf10f09bcc)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-08-12 02:40:35 +02:00
Michael Niedermayer
656f930160 avcodec/iff: check pixfmt for rgb8 / rgbn
Fixes out of array access

Found-by: Piotr Bandurski <ami_stuff@o2.pl>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 3539d6c63a)

Conflicts:

	libavcodec/iff.c
2014-08-10 22:24:05 +02:00
Anton Khirnov
c56d6f3552 cdgraphics: do not return 0 from the decode function
0 means no data consumed, so it can trigger an infinite loop in the
caller.

CC:libav-stable@libav.org
(cherry picked from commit c7d9b473e2)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-08-10 22:24:05 +02:00
Michael Niedermayer
65f581472c avcodec/dvdsub_parser: print message if packet is smaller than the packet size field
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit bcc898dd26)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-08-02 01:27:26 +02:00
Michael Niedermayer
c7fac44ef8 avcodec/dvdsub_parser: Check buf_size before reading 32bit packet size
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 81c1657a59)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-08-02 01:27:26 +02:00
Michael Niedermayer
8d48223a32 avcodec/dvdsub_parser: never return 0 when the input isnt 0
Fixes a infinite loop
Fixes Ticket3804

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit cfdb30d2f1)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-08-02 01:27:25 +02:00
Michael Niedermayer
01e2dd53a0 cavsdec: fix qp fixed slice handling
Fixes Ticket3400

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 0accf24b15)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-06-25 16:35:25 +02:00
Michael Niedermayer
b94621ac6b avcodec/alsdec: Clear MPEG4AudioConfig so that no use of uninitialized memory is possible
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 6e6bd5481c)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-06-25 16:35:24 +02:00
Michael Niedermayer
9fe7e45030 avcodec/aic: fix quantization table permutation
Fixes Ticket3700

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 0a2004b6d1)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-06-25 16:35:24 +02:00
Michael Niedermayer
3c169251ee avcodec/mss4: Fix () in MKVAL() macro
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit cf7ff0146c)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-06-25 16:35:24 +02:00
Michael Niedermayer
81a2740919 avcodec/mss34dsp: fix () in SOP* macros
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 6e720c5c81)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-06-25 16:35:24 +02:00
Michael Niedermayer
a144d305dd avcodec/mlpdec: fix () in MSB_MASK() macro
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit fa160af08b)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-06-25 16:35:24 +02:00
Michael Niedermayer
4965da92ef avcodec/ivi_dsp: add some missing () to macros
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit f276bf303c)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-06-25 16:35:24 +02:00
Michael Niedermayer
a83304814c avcodec/h264: fix () in macros
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit af62b42736)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-06-25 16:35:24 +02:00
Michael Niedermayer
db6f3be941 avcodec/golomb-test: fix () in EXTEND() macro
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 97e6b5ee3a)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-06-25 16:35:24 +02:00
Michael Niedermayer
2f1f407044 avcodec/aac: fix () in IS_CODEBOOK_UNSIGNED macro
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit fa915d4193)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-06-25 16:35:24 +02:00
Michael Niedermayer
bdd2f7c7a7 avcodec/libvorbisenc: dont add the duration to AV_NOPTS_VALUE
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 19e66c7232)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-06-25 16:35:24 +02:00
Michael Niedermayer
62b9c99aec avcodec/diracdec: move mc buffer allocation to per frame
Fixes out of array accesses for non default buffers with large strides

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 4a30f08505)
(cherry picked from commit 9c9fc79d9237d28e33161cb2e75082d8ad232b2e)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-06-25 16:35:24 +02:00
Michael Niedermayer
d9d66d2d16 avcodec/mjpegdec: Fix undefined shift
Fixes CID1194388

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit b432960528)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-06-25 16:35:24 +02:00
Michael Niedermayer
4d9aad95b5 iavcodec/vc1dec: Fix missing {}
Fixes part of Ticket3466
Found-by: Andrey_Karpov / PVS-Studio
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit cb53beb81a)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-06-25 16:35:24 +02:00
Michael Niedermayer
66ca450fcb avcodec/g723_1: add assert to help static code analyzers
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 1457f3fd90)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-06-25 16:18:40 +02:00
Michael Niedermayer
8162b69091 avcodec/diracdec: fix undefined behavior with shifts
Fixes part of Ticket3466
Found-by: Andrey_Karpov / PVS-Studio
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit b8598f6ce6)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-06-25 16:18:40 +02:00
Michael Niedermayer
6f6f5f9c8b avcodec/mjpegen: Fix declared argument size
Fixes part of Ticket3466
Found-by: Andrey_Karpov / PVS-Studio
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 256f530603)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-06-25 16:18:40 +02:00
Michael Niedermayer
2ca68b98ba avcodec/x86/idct_sse2_xvid: fix non C99 inline function
Found-by: Matt Oliver <protogonoi@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 46d5625f44)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-06-25 16:18:40 +02:00
Anthoine Bourgeois
853bbe1aed avcodec/dirac_arith: Fix build with PIC and stack-check options
Fixes Ticket3540

The function dirac_get_arith_bit in libavcodec/dirac_arith.h can't be
built with PIC and check-stack because the asm code needs 6 registers
and PIC and check-stack options take 1 each and x86 is quite limited
in this area.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit d8ab7f31dd)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-06-25 16:18:40 +02:00
Michael Niedermayer
491e1ec95d avcodec/msrle: check return code for success before use
The check is possibly redundant, but better to check for errors
that dont occur than to skip the check and crash

Fixes CID1197060

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 754f84663e)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-06-25 16:18:40 +02:00
Michael Niedermayer
5805c0c2b6 avcodec/wma: use av_freep(), do not leave stale pointers in memory
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit d167faafe9)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-06-25 16:18:40 +02:00