Commit graph

19317 commits

Author SHA1 Message Date
Michael Niedermayer
bc07ec7b2f avcodec/snow: split block clipping checks
Fixes out of array read
Fixes: d4476f68ca1c1c57afbc45806f581963-asan_heap-oob_2266b27_8607_cov_4044577381_snow_chroma_bug.avi
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 61d59703c9)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-03-08 15:32:44 +01:00
Michael Niedermayer
3a893335bd avcodec/msrle: use av_image_get_linesize() to calculate the linesize
Fixes out of array access
Fixes: 14a74a0a2dc67ede543f0e35d834fbbe-asan_heap-oob_49572c_556_cov_215466444_44_001_engine_room.mov
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit c919e1ca2e)

Conflicts:

	libavcodec/msrle.c
(cherry picked from commit bc1c8ec5e6)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-03-08 15:32:44 +01:00
Michael Niedermayer
c23597ec7e avcodec/wmalosslessdec: fix mclms_coeffs* array size
Fixes corruption of context
Fixes: 8835659dde6a4f7dcdf341de6a45c6c8-signal_sigsegv_1dce67b_4564_cov_2504444599_classical_22_16_1_14000_v3c_0_extend_0_29.wma
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit ec9578d54d)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-03-08 15:32:44 +01:00
Michael Niedermayer
b66a0dce11 avcodec/vc1: reset fcm/field_mode in non advanced header parsing
Fixes NULL pointer dereference
Fixes: signal_sigsegv_1ab8bf4_2847_cov_4254117347_SA10091.vc1
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit b51e935477)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-03-08 15:32:44 +01:00
Michael Niedermayer
c598a46821 avcodec/vmnc: Check that rectangles are within the picture
Prevents out of array accesses with CODEC_FLAG_EMU_EDGE

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 6ba02602aa)

Conflicts:

	libavcodec/vmnc.c

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 7c17207ab9)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-03-08 15:32:44 +01:00
Michael Niedermayer
912b7fe922 dnxhdenc: fix mb_rc size
Fixes out of array access with RC_VARIANCE set to 0

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit f1caaa1c61)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-03-08 15:32:44 +01:00
Michael Niedermayer
84c84204b3 avcodec/jpeglsdec: check err value for ls_get_code_runterm()
Fixes infinite loop
Fixes Ticket3086

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit cc0e47b550)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-03-08 15:32:44 +01:00
Michael Niedermayer
82746c468b avcodec/avpacket/av_packet_split_side_data: ensure that side data padding is initialized
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 240fd8c96f)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 1e48318802b3caa493a40c0584afc30cc866d9d0)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-03-08 15:32:43 +01:00
Michael Niedermayer
2289bffa31 avcodec/parser: reset indexes on realloc failure
Fixes Ticket2982

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit f31011e9ab)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-03-08 15:32:43 +01:00
Michael Niedermayer
b650d5376c avcodec/ffv1enc: update buffer check for 16bps
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 3728603f18)

Conflicts:

	libavcodec/ffv1enc.c
(cherry picked from commit c900c6e5c26cd86cf34f9c8d4347cedbd01f3935)
2014-03-08 15:32:43 +01:00
Michael Niedermayer
ec39abcd37 avcodec/truemotion2: Fix av_freep arguments
Fixes null pointer dereference
Fixes Ticket2944

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit c54aa2fb0f)

Conflicts:

	libavcodec/truemotion2.c

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-03-08 15:32:43 +01:00
Michael Niedermayer
66fcda7e70 avcodec/mjpegdec: Add some sanity checks to ljpeg_decode_rgb_scan()
These prevent the rgb ljpeg code from being run on parameters that it doesnt
support. No testcase available but it seems possible to trigger these.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 61c68000ed)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-03-08 15:32:43 +01:00
Michael Niedermayer
fdf57ad2ff avcodec/dsputil: fix signedness in sizeof() comparissions
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 454a11a1c9)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-03-08 15:32:43 +01:00
Martin Storsjö
69f724a538 arm: Don't clobber callee saved registers in scalarproduct
q4-q7/d8-d15 are supposed to not be clobbered by the callee.

CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit d307e408d4)
2013-12-21 10:00:18 +01:00
Michael Niedermayer
929100ae98 avcodec/cabac: force get_cabac to be not inlined
works around bug in gccs inline asm register assignment
Fixes Ticket3177

gcc from 4.4 to 4.6 is affected at least, no non affected gccs known
clang seems not affected

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 0538b29ae8)
2013-12-09 10:40:08 +01:00
Michael Niedermayer
d49761b396 avcodec/h264: reduce noisiness of "mmco: unref short failure"
Do not consider it an error if we have no frames and should discard one.
This condition can easily happen when decoding is started from an I frame

Fixes Ticket2811

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 08a8976196)

Conflicts:
	libavcodec/h264_refs.c
2013-10-26 01:26:00 +02:00
Michael Niedermayer
87f719a03f avcodec/h264_refs: modify key frame detection heuristic to detect more cases
Fixes Ticket2968

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 5ac6b6028f)

Conflicts:
	libavcodec/h264_refs.c
(cherry picked from commit 6636dd551f)
2013-10-24 11:40:01 +02:00
Carl Eugen Hoyos
a5ef62ede1 Avoid a deadlock when decoding wma.
Fixes ticket #2925.
(cherry picked from commit ec8a4841f7)
2013-09-02 10:03:49 +02:00
Michael Niedermayer
2d945ac68f avcodec/pngdsp: fix (un)signed type in end comparission
Fixes out of array accesses
Fixes Ticket2919

Found_by: ami_stuff
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 86736f59d6)
2013-08-30 23:42:00 +02:00
Michael Niedermayer
1934bb7536 h264: skip error concealment when SPS and slices are mismatching
Fixes out of array accesses

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 695af8eed6)

Conflicts:

	libavcodec/h264.c
2013-08-25 21:05:59 +02:00
Michael Niedermayer
39ed544262 Merge commit '072be3e896'
* commit '072be3e896':
  h264: set parameters from SPS whenever it changes

Conflicts:
	libavcodec/h264.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit b9d887c225)

Conflicts:

	libavcodec/h264.c
2013-08-25 14:49:11 +02:00
Michael Niedermayer
f9e6fb50ce jpeg2000: check log2_cblk dimensions
Fixes out of array access
Fixes Ticket2895

Found-by: Piotr Bandurski <ami_stuff@o2.pl>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 9a271a9368)

Conflicts:

	libavcodec/jpeg2000dec.c

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-08-25 13:11:53 +02:00
Michael Niedermayer
453e2f1528 avcodec/rpza: Perform pointer advance and checks before using the pointers
Fixes out of array accesses
Fixes Ticket2850

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 3819db745d)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-08-24 15:17:49 +02:00
Michael Niedermayer
b25c3063b2 avcodec/flashsv: check diff_start/height
Fixes out of array accesses
Fixes Ticket2844

Found-by: ami_stuff
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 880c73cd76)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-08-24 15:17:49 +02:00
Michael Niedermayer
29b14db916 update all trac links to use the trac subdomain
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-08-24 15:17:48 +02:00
Michael Niedermayer
d94d383f13 xbmdec: fix off by one error in scanf()
Fixes out of array access

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 88c1b0e7a852e48d9f0e3d79c44edaa86e59acfe)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-08-24 15:17:48 +02:00
Michael Niedermayer
2a8c3a7895 avcodec/kmvc: fix MV checks
Fixes Ticket2813
Fixes regression since 70b5583

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 3cd8aaa2b2)
2013-07-31 03:03:05 +02:00
Michael Niedermayer
c9f34c8260 mpeg12dec: avoid reinitialization on PS changes when possible.
Fixes Ticket2574

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 970c8df735)

Conflicts:
	libavcodec/mpeg12dec.c
2013-07-09 00:48:42 +02:00
Hendrik Leppkes
274ec187dc mathops/x86: work around inline asm miscompilation with GCC 4.8.1
The volatile is not required here, and prevents a miscompilation with GCC
4.8.1 when building on x86 with --cpu=i686

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 659df32a9d)
2013-06-24 08:48:31 +02:00
Michael Niedermayer
7a472e0da9 alacenc: Fix missing sign_extend()
Fixes ticket #2497

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 8aea2f05dc)
2013-06-13 00:11:02 +02:00
Claudio Freire
f6bca606f1 AAC encoder: Fix rate control on twoloop.
Fixes a case where multichannel bitrate isn't accurately
targetted by psy model alone, never achieving the target bitrate.
Now fixed.

Fixes ticket #2625.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Carl Eugen Hoyos <cehoyos@ag.or.at>
2013-06-02 16:30:15 +02:00
Michael Niedermayer
5fd83f29f7 h264_cavlc: fix reading skip run
Fixes Ticket2606

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 826b3a75cd)

Conflicts:
	libavcodec/h264_cavlc.c
2013-05-30 23:22:00 +02:00
Michael Niedermayer
1b0028a3c5 smacker: remove av_clip_int16()
Fixes Ticket2425

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 2211c76287)
2013-05-21 00:03:44 +02:00
Carl Eugen Hoyos
7dd042e657 Fix type of shared flac table ff_flac_blocksize_table[].
Fixes ticket #2533.
(cherry picked from commit a07ac1f788)
2013-05-05 20:39:45 +02:00
Michael Niedermayer
fff5f65540 Merge remote-tracking branch 'jamrial/release/0.11' into release/0.11
* jamrial/release/0.11:
  lavc/bink: Chech for malloc failure

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-03-20 22:10:40 +01:00
Alexander Kojevnikov
7cb24ecd4c mp3dec: Fix VBR bit rate parsing
When parsing the Xing/Info tag, don't set the bit rate if it's an Info tag.

When parsing the stream, don't override the bit rate if it's already set,
otherwise calculate the mean bit rate from parsed frames. This way, the bit
rate will be set correctly both for CBR and VBR streams.

Signed-off-by: Alexander Kojevnikov <alexander@kojevnikov.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 29d8cd265a)

Conflicts:

	tests/ref/lavf-fate/mp3

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-03-20 22:02:28 +01:00
Michael Niedermayer
864a7e73b9 huffyuvdec: Skip len==0 cases
Fixes vlc decoding for hypothetical files that would contain such cases.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 0dfc01c2bb)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 5ff41ffeb4cb9ea6df49757dc859619dc3d3ab4f)

Conflicts:

	libavcodec/huffyuv.c
2013-03-20 22:02:28 +01:00
Michael Niedermayer
562aa82d2a huffyuvdec: Check init_vlc() return codes.
Prevents out of array writes

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit f67a0d1152)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 95ab8d33e1a680f30a5a9605175112008ab81afc)

Conflicts:

	libavcodec/huffyuv.c
2013-03-20 22:02:28 +01:00
James Almer
c095137d1b lavc/bink: Chech for malloc failure
Based on commit 8ab2173ed1
2013-03-20 17:51:15 -03:00
Carl Eugen Hoyos
b63dbe2220 Do not (re-)set libx264 parameter b_tff if interlaced encoding was not requested.
Reconfiguring can break x264 lossless encoding.

Fixes ticket #2165.
(cherry picked from commit 75c7e4583f)
2013-03-18 02:19:13 +01:00
Michael Niedermayer
5df2dc0f94 aacsbr: Check for envelope scalefactors overflowing
This prevents various values from becoming stuck at NAN and
output to become silent
If someone knows a cleaner solution, thats welcome!

Fixes Ticket2335

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 8978c743fb)
2013-03-08 20:09:20 +01:00
Michael Niedermayer
ebe645f02b h264: Reset last_pocs in case of reference or frame number inconsistencies
This prevents faulty increasing of has_b_frames
Should fix Ticket 2062

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit c230af9bcc)
2013-02-14 09:24:11 +01:00
Michael Niedermayer
470ee0c660 h264_refs: Print default in case we are missing a reference.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit a39d36146a)
2013-02-14 09:24:08 +01:00
Piotr Bandurski
d338632e9f tiffdec: Use the correct height field.
Fixes Ticket913

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 4784a135b2)
2013-01-07 00:29:28 +01:00
Clément Bœsch
8bc0127b3d lavc/ass_split: check for NULL pointer in ff_ass_split_override_codes().
This is consistent with the other ff_ass_split_* functions.

It also fixes a crash when trying to split a dialog with text=NULL
(which seems to happen when the text of the dialog is empty); basically,
this commit fixes crashes when trying to encode an empty text subtitle
dialog (see subrip and mov_text encoders).

Fixes Ticket2048.
(cherry picked from commit c83002a4f8)
2013-01-01 18:19:50 +01:00
Michael Niedermayer
5f0e5b4048 mpeg1video: fix regression with slices != threads
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit a01679586c)
2012-12-13 00:21:18 +01:00
Michael Niedermayer
e9ded2e1a9 mpeg1video: support multi threaded slice encoding.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 03df972016)

Conflicts:
	libavcodec/mpeg12enc.c
2012-12-13 00:21:09 +01:00
Janne Grunau
69cc119d64 h264: slice-mt: get last_pic_dropable from master context
Fixes fate-h264-conformance-cvnlfi2_sony_h and smllwebdl.mkv from
https://github.com/OpenELEC/OpenELEC.tv/issues/1557 .
(cherry picked from commit 24c62ea7a5)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2012-12-06 16:44:49 +01:00
Michael Niedermayer
54ac3d10d1 mp3dec: Fix possibly exploitable crash
I was sadly unable to find a non fuzzed mp3 that uses the
feature that contained the bug (and i searched hard ...), thus
while this fixes the security issue. It may or may not fix
mixed blocks in 8khz mp3s, i cant say due to lack of samples to test.

Security issue exists since: b37d945dd4

Reported-by: Dale Curtis <dalecurtis@google.com>
(Probably) Found-by: inferno@chromium.org
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 94041febc5)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2012-12-06 16:28:27 +01:00
Paul B Mahol
99dbda3008 aasc: fix out of array write
Closes #1619.

Signed-off-by: Paul B Mahol <onemda@gmail.com>
(cherry picked from commit 8a57ca5c6a)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2012-09-15 00:21:58 +02:00