Commit graph

24754 commits

Author SHA1 Message Date
Michael Niedermayer
13ce367368 avcodec/wmalosslessdec: fix mclms_coeffs* array size
Fixes corruption of context
Fixes: 8835659dde6a4f7dcdf341de6a45c6c8-signal_sigsegv_1dce67b_4564_cov_2504444599_classical_22_16_1_14000_v3c_0_extend_0_29.wma
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit ec9578d54d)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-02-07 15:43:07 +01:00
Michael Niedermayer
33c47d3976 avcodec/vc1: reset fcm/field_mode in non advanced header parsing
Fixes NULL pointer dereference
Fixes: signal_sigsegv_1ab8bf4_2847_cov_4254117347_SA10091.vc1
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit b51e935477)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-02-01 19:22:23 +01:00
Michael Niedermayer
d0d0924947 avcodec/takdec: always check bits_per_raw_sample
Fixes out of array access
Fixes: asan_heap-oob_19c7a94_6470_cov_1453611734_luckynight-partial.tak
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit f58eab1512)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-01-31 19:55:25 +01:00
Michael Niedermayer
ca5d6c615e avcodec/jpeg2000dec: fix error detection in pix_fmt_match()
Fixes out of array accesses with CODEC_FLAG_EMU_EDGE

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 8001e9f7d1)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-01-21 17:05:16 +01:00
Michael Niedermayer
7c17207ab9 avcodec/vmnc: Check that rectangles are within the picture
Prevents out of array accesses with CODEC_FLAG_EMU_EDGE

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 6ba02602aa)

Conflicts:

	libavcodec/vmnc.c

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-01-21 17:02:19 +01:00
Michael Niedermayer
9847f02faf dnxhdenc: fix mb_rc size
Fixes out of array access with RC_VARIANCE set to 0

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit f1caaa1c61)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-01-18 01:13:22 +01:00
Dale Curtis
33aa2c5d6b h264: Clear ERContext.cur_pic when unref'ing current picture.
Signed-off-by: Dale Curtis <dalecurtis@chromium.org>
(cherry picked from commit 4feca2214a)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-01-03 00:44:29 +01:00
Michael Niedermayer
66a9edfcf6 do O(1) instead of O(n) atomic operations in register functions
about 1ms faster startup time

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 133fbfc781)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-12-24 01:05:47 +01:00
Paul B Mahol
a9382fc15c avcodec/libopusenc: change default frame duration to 20 ms
20 ms is used by libopus encoder.

Signed-off-by: Paul B Mahol <onemda@gmail.com>
(cherry picked from commit 74906d3727)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-12-24 01:05:47 +01:00
Michael Niedermayer
bd9dcb411d avcodec/jpeg2000dec: Check precno before using it in JPEG2000_PGOD_CPRL
Fixes out of array reads
Fixes: asan_heap-oob_f0de57_6823_mjp2.mov

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 3d5a5e86be)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-12-24 01:05:47 +01:00
Michael Niedermayer
ae81a0e32d avcodec: move end zeroing code from av_packet_split_side_data() to avcodec_decode_subtitle2()
This code changes the input packet, which is read only and can in
rare circumstances lead to decoder errors. (i run into one of these in
the audio decoder, which corrupted the packet during av_find_stream_info()
so that actual decoding that single packet failed later)
Until a better fix is implemented, this commit limits the problem.
A better fix might be to make the subtitle decoders not depend on
data[size] = 0 or to copy their input when this is not the case.
(cherry picked from commit 01923bab98)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-12-24 01:05:47 +01:00
Michael Niedermayer
0cd61c7f7d rename new lls code to lls2 to avoid conflict with the old which has a different ABI
also remove failed attempt at a compatibility layer, the code simply cannot work

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit c3814ab654)

Conflicts:

	libavcodec/version.h
2013-12-24 01:05:47 +01:00
Michael Niedermayer
28ac4e91dc avutil: rename lls to lls2
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit bbe66ef912)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-12-24 01:05:47 +01:00
Diego Biurrun
6b683be641 mpeg12dec: Remove incomplete and wrong UV swapping code for VCR2
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 3215140425)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-12-24 01:05:47 +01:00
Kostya Shishkov
3a3b5ae4c0 mpegvideo: Fix swapping of UV planes for VCR2
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit bae14f38d9)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-12-24 01:05:47 +01:00
Michael Niedermayer
70fcea3b77 h264: Do not treat the initial frame special in handling of frame gaps
The not handling of frame gaps has lead to the lack of a dummy reference
frame, which has lead to the failure of decode_slice_header() which has
lead to one SEI recovery message being skiped which had introduced a
slightly suboptimal recovery point for at least 1 h264 file compared to
JM.

Found-by: Carl & BugMaster
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 9e5ef1c5c3)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-12-24 01:05:47 +01:00
Michael Niedermayer
b545d11d49 avcodec/jpeg2000dec: non zero image offsets are not supported
Fixes out of array accesses
Fixes Ticket3080
Found-by: ami_stuff
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 780669ef7c)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-12-24 01:05:47 +01:00
Michael Niedermayer
bdf6e6fff4 avcodec/jpeglsdec: check err value for ls_get_code_runterm()
Fixes infinite loop
Fixes Ticket3086

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit cc0e47b550)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-12-24 01:05:47 +01:00
Michael Niedermayer
fce2cfbdcf avcodec/utils: add some saftey checks to add_metadata_from_side_data()
This fixes potential overreads with crafted files.

Found-by: wm4
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 838f461b07)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-12-24 01:05:47 +01:00
Michael Niedermayer
72f1907c96 avcodec/avpacket/av_packet_split_side_data: ensure that side data padding is initialized
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 240fd8c96f)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-12-24 01:05:46 +01:00
Michael Niedermayer
47f8497837 avcodec/jpeg2000dec: fix context consistency with too large lowres
Fixes out of array accesses
Fixes Ticket2898

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit a1b9004b76)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-12-24 01:05:46 +01:00
Michael Niedermayer
93f26b7992 avcodec/jpeg2000dec: prevent out of array accesses in pixel addressing
Fixes Ticket2921

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit fe448cd28d)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-12-24 01:05:46 +01:00
Michael Niedermayer
f684bbf224 avcodec/jpeg2000dec: check transform equality in MCT
Fixes null pointer dereference
Fixes Ticket2843

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit ac3b01a9c0)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-12-24 01:05:46 +01:00
Mason Carter
aeac212fda VC1: Fix intensity compensation performance regression
Fix https://trac.ffmpeg.org/ticket/3204

The problem was that intensity compensation was always used once it was
encountered. This is because v->next_use_ic was never set back to zero.
To fix this, when resetting v->next_luty/uv, also reset v->next_use_ic.

This improved (restored) performance by 85% when decoding
http://bit.ly/bbbwmv

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit ed5bed4152)
2013-12-22 16:09:44 +01:00
Martin Storsjö
62f05d6309 arm: Don't clobber callee saved registers in scalarproduct
q4-q7/d8-d15 are supposed to not be clobbered by the callee.

CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit d307e408d4)
2013-12-21 09:58:12 +01:00
Michael Niedermayer
7e73760950 avcodec/cabac: force get_cabac to be not inlined
works around bug in gccs inline asm register assignment
Fixes Ticket3177

gcc from 4.4 to 4.6 is affected at least, no non affected gccs known
clang seems not affected

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 0538b29ae8)
2013-12-09 10:36:00 +01:00
Michael Niedermayer
0639e403be avcodec/error_resilience: check that er is supported before attempting to read the status of the previous slice
Fixes incorrectly set error_occured and improves speed

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 90539cea33)
2013-12-07 11:34:42 +01:00
Michael Niedermayer
5c7d6be5f9 avcodec/error_resilience: factor er_supported() check out
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit afb18c5578)
2013-12-07 11:34:25 +01:00
Michael Niedermayer
beb28bc55d avcodec/bink: fix seeking to frame 0
Fixes Ticket3088

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit cb52d6da0a)

Conflicts:
	libavcodec/bink.c
2013-10-31 00:57:11 +01:00
Michael Niedermayer
1d0e583728 h264: make flush_change() set mmco_reset
This ensures that frames do not get mixed on context reinits

Fixes Ticket2836

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 3c9dd93faa)
2013-10-26 02:41:31 +02:00
Michael Niedermayer
782331be1e avcodec/h264: reduce noisiness of "mmco: unref short failure"
Do not consider it an error if we have no frames and should discard one.
This condition can easily happen when decoding is started from an I frame

Fixes Ticket2811

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 08a8976196)
2013-10-26 00:52:34 +02:00
Michael Niedermayer
cd7d575e90 avcodec/h264: do not trust last_pic_droppable when marking pictures as done
This simplifies the code and fixes a deadlock

Fixes Ticket2927
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 29ffeef5e7)
2013-10-26 00:50:00 +02:00
Michael Niedermayer
59431fc841 avcodec/h264_refs: modify key frame detection heuristic to detect more cases
Fixes Ticket2968

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 5ac6b6028f)
2013-10-24 10:28:20 +02:00
Paul B Mahol
b4ccdf5e68 avcodec/ffv1dec: fix format detection
Fixes crash with carefuly designed files.

Signed-off-by: Paul B Mahol <onemda@gmail.com>
(cherry picked from commit a27227d401)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-10-08 18:13:55 +02:00
Michael Niedermayer
9b02aa2593 avcodec/parser: reset indexes on realloc failure
Fixes Ticket2982

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit f31011e9ab)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-10-08 18:13:55 +02:00
Michael Niedermayer
f089e67d51 avcodec/imgconvert/get_color_type: fix type for PAL8
Fixes Ticket3008

Fate changes as PAL8 gets used instead of BGR8

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 95666b2298)
2013-10-02 23:47:41 +02:00
Michael Niedermayer
2b06f5f8f1 avcodec/g2meet: Fix framebuf size
Currently the code can in some cases draw tiles that hang outside the
allocated buffer. This patch increases the buffer size to avoid out
of array accesses. An alternative would be to fail if such tiles are
encountered.
I do not know if any valid files use such hanging tiles.

Fixes Ticket2971
Found-by: ami_stuff
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit e07ac727c1)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-09-23 21:46:15 +02:00
Michael Niedermayer
0a64b25c77 avcodec/g2meet: Fix order of align and pixel size multiplication.
Fixes out of array accesses
Fixes Ticket2922

Found-by: ami_stuff
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 821a5938d1)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-09-23 21:46:15 +02:00
Michael Niedermayer
40e52bbb63 avcodec/ffv1enc: update buffer check for 16bps
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 3728603f18)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-09-23 21:46:15 +02:00
Michael Niedermayer
aeec1a6430 avcodec/truemotion2: Fix av_freep arguments
Fixes null pointer dereference
Fixes Ticket2944

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit c54aa2fb0f)

Conflicts:

	libavcodec/truemotion2.c
2013-09-23 21:46:15 +02:00
Michael Niedermayer
ef121a88d5 avcodec/mjpegdec: Add some sanity checks to ljpeg_decode_rgb_scan()
These prevent the rgb ljpeg code from being run on parameters that it doesnt
support. No testcase available but it seems possible to trigger these.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 61c68000ed)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-09-23 21:46:15 +02:00
Michael Niedermayer
54bdb5fc86 avcodec/dsputil: fix signedness in sizeof() comparissions
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 454a11a1c9)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-09-23 21:46:15 +02:00
Michael Niedermayer
2fd824b466 ffv1dec: Check bits_per_raw_sample and colorspace for equality in ver 0/1 headers
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit b05cd1ea7e)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-09-23 21:46:15 +02:00
Michael Niedermayer
005b38f8f1 avcodec/ffv1dec: check global header version
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 20b965a1a4)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-09-23 21:46:15 +02:00
Clément Bœsch
0f429392cf avcodec/assenc: fix potential overread.
(cherry picked from commit 860a081058)

Signed-off-by: Alexander Strasser <eclipse7@gmx.net>
2013-09-15 22:21:05 +02:00
Clément Bœsch
2dc6c5d462 avcodec/srtdec: fix potential overread.
(cherry picked from commit 3a54c221d5)

Signed-off-by: Alexander Strasser <eclipse7@gmx.net>
2013-09-15 22:21:05 +02:00
Carl Eugen Hoyos
f5a4bd23e9 Avoid a deadlock when decoding wma.
Fixes ticket #2925.
(cherry picked from commit ec8a4841f7)
2013-09-02 08:54:32 +02:00
Carl Eugen Hoyos
8e1760f37f avcodec/ffv1dec: reorganize thread init/update
This moves some allocations to init, reducing possible failure modes in update.
Always copies from the previous context instead of just during init

Fixes Ticket2923

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 21dc3a3cc2)
2013-09-02 08:49:59 +02:00
Michael Niedermayer
1da5ab751f avcodec/ffv1dec: move initial_states init to init_thread_copy()
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit c72cca5a44)
2013-09-02 08:47:50 +02:00
Michael Niedermayer
be47e93134 avcodec/h264: set er.ref_count earlier
Fixes Ticket2910

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 93cf7b0195)
2013-09-01 09:58:55 +02:00