Commit graph

33637 commits

Author SHA1 Message Date
Michael Niedermayer
147a387fba avcodec/msvideo1: Check side data size before use
Fixes out of array read

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 161ccdaa06)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-11-26 15:12:26 +01:00
Michael Niedermayer
2312d1d979 avcodec/qpeg: Check side data size before use
Fixes out of array read

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 16793504df)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-11-26 15:12:26 +01:00
Michael Niedermayer
1604a2b1e6 avcodec/qtrle: Check side data size before use
Fixes out of array read

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 7d196f2a5a)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-11-26 15:12:26 +01:00
Michael Niedermayer
f39522b6e7 avcodec/msrle: Check side data size before use
Fixes out of array read

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit a6330119a0)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-11-26 15:12:26 +01:00
Michael Niedermayer
09411a7d5e avcodec/kmvc: Check side data size before use
Fixes out of array read

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 2d99101d09)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-11-26 15:12:26 +01:00
Michael Niedermayer
ef32b162e8 avcodec/idcinvideo: Check side data size before use
Fixes out of array read

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit a2b8dde659)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-11-26 15:12:26 +01:00
Michael Niedermayer
41b1ec0810 avcodec/cinepak: Check side data size before use
Fixes out of array read

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 121be31060)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-11-26 15:12:26 +01:00
Michael Niedermayer
e3f8b32279 avcodec/8bps: Check side data size before use
Fixes out of array read

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 042faa847f)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-11-26 15:12:26 +01:00
Michael Niedermayer
fe4c6aeb99 avcodec/dvdsubdec: Fix off by 1 error
Fixes out of array read

Found-by: Thomas Garnier using libFuzzer
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit c92f55847a)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-11-26 15:12:26 +01:00
Michael Niedermayer
3177ea512f avcodec/dvdsubdec: Fix buf_size check
Fixes out of array access

Found-by: Thomas Garnier using libFuzzer
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 25ab1a65f3)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-11-26 15:12:26 +01:00
Ronald S. Bultje
e25441912b vp9: change order of operations in adapt_prob().
This is intended to workaround bug "665 Integer Divide Instruction May
Cause Unpredictable Behavior" on some early AMD CPUs, which causes a
div-by-zero in this codepath, such as reported in Mozilla bug #1293996.

Note that this isn't guaranteed to fix the bug, since a compiler is free
to reorder instructions that don't depend on each other. However, it
appears to fix the bug in Firefox, and a similar patch was applied to
libvpx also (see Chrome bug #599899).

(cherry picked from commit be885da342)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-11-26 15:12:26 +01:00
Michael Niedermayer
abe16359ba avcodec/interplayvideo: Check side data size before use
Fixes out of array read

Found-by: Thomas Garnier using libFuzzer
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 85d23e5cbc)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-11-26 15:12:26 +01:00
Michael Niedermayer
8328c07fb1 avcodec/mpegvideo_enc: Clear mmx state in ff_mpv_reallocate_putbitbuffer()
This function must be called from the mb or slice encoding loop and MMX state may not
be clean there

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 03ec6b780c)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-11-26 15:12:25 +01:00
Michael Niedermayer
865e3f5553 avcodec/utils: Clear MMX state before returning from avcodec_default_execute*()
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 4f96f9d111)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-11-26 15:12:25 +01:00
Moritz Barsnick
bb83ff8b41 lavc: fix typos
Signed-off-by: Moritz Barsnick <barsnick@gmx.net>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 3305f71025)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-11-26 15:12:25 +01:00
Ronald S. Bultje
62b2b2195b videodsp: fix 1-byte overread in top/bottom READ_NUM_BYTES iterations.
This can overread (either before start or beyond end) of the buffer in
Nx1 (i.e. height=1) images.

Fixes mozilla bug 1240080.

(cherry picked from commit 0f88b3f82f)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-10-05 03:06:45 +02:00
Sasi Inguva
ca216c71c7 lavc/movtextdec.c: Avoid infinite loop on invalid data.
Signed-off-by: Sasi Inguva <isasi@google.com>
(cherry picked from commit 7e9e1b7070)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-09-28 11:12:20 +02:00
Michael Niedermayer
ab737ab31d avcodec/ansi: Check dimensions
Fixes: 1.avi

Found-by: 连一汉 <lianyihan@360.cn>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 69449da436)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-09-26 21:43:04 +02:00
Michael Niedermayer
69b00a7fb6 avcodec/cavsdsp: use av_clip_uint8() for idct
Fixes out of array read
Fixes: 1.swf

Found-by: 连一汉 <lianyihan@360.cn>
Tested-by: 连一汉 <lianyihan@360.cn>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 0e318f110b)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-09-23 18:19:56 +02:00
Michael Niedermayer
f8dcc9e718 avcodec/g726: Add missing ADDB output mask
Fixes: 1.poc
Fixes out of array read

Found-by: 连一汉 <lianyihan@360.cn>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit a5af1240fc)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-09-18 00:18:11 +02:00
Michael Niedermayer
175a95bfdd avcodec/avpacket: clear side_data_elems
Fixes null pointer dereference

Found-by: 连一汉 <lianyihan@360.cn>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 5e1bf9d8c0)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-09-18 00:18:11 +02:00
Michael Niedermayer
ac60619acd avcodec/ccaption_dec: Use simple array instead of AVBuffer
This is simpler and fixes an out of array read, fixing it with AVBuffers
would be more complex

Fixes: e00d9e6e50e5495cc93fea41147b97bb/asan_heap-oob_12dcdbb_8798_b32a97ea722dd37bb5066812cc674552.mov

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 752e6dfa3e)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-09-18 00:18:11 +02:00
Michael Niedermayer
562f2ba4ed avcodec/aacenc: Tighter input checks
Fixes occurance of NaN/Inf leading to assertion failures and out of array access
Fixes: d1c38a09acc34845c6be3a127a5aacaf/signal_sigsegv_3982225_6121_d18bd5451d4245ee09408f04badd1b83.wmv

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 77bf96b047)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-08-23 12:36:25 +02:00
Michael Niedermayer
5a96b4b443 libavcodec/wmalosslessdec: Check the remaining bits
Fixes assertion failure
Fixes: 24ebfda03228b5cc1ef792608cfba458/signal_sigabrt_7ffff6ae7c37_6473_3fa8a111dbc752b1a7c411c5ab79aaa4.wma

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 67318187fb)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-08-21 21:11:47 +02:00
Michael Niedermayer
33ec0280f3 avcodec/diracdec: Check numx/y
Fixes division by 0
Fixes: 60261c4469ba3e11059890fb2832a515/asan_generic_135e694_2790_beb94eaa0aeb7d11c0437375a8964a99.drc

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit a31e08fa1a)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-08-20 19:40:26 +02:00
Michael Niedermayer
2b135f266d avcodec/indeo2: check ctab
Fixes out of array access
Fixes: 6b73fa392ac808f02e95a4e0a5770026/asan_static-oob_1b15f9a_1969_e7778535e5f27225fe0d6ded14721430.AVI

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 9ffe44c5c7)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-08-19 14:47:39 +02:00
Michael Niedermayer
7a2329fac1 avcodec/h264: Put context_count check back
Fixes assertion failure

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit f883f0b0bd)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-08-15 20:25:14 +02:00
Michael Niedermayer
3988470ee3 avcodec/raw: Fix decoding of ilacetest.mov
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit bbec14de31)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-08-15 20:25:13 +02:00
Michael Niedermayer
3a6b27caf8 avcodec/ffv1enc: Fix assertion failure with non zero bits per sample
Fixes Ticket5736
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

(cherry picked from commit c1bfeda5a3)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-08-15 20:25:13 +02:00
Michael Niedermayer
da12d544bf avcodec/vp9_parser: Check the input frame sizes for being consistent
Suggested-by: BBB
Fixed-by: BBB
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 77b0f3f26d)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-08-15 20:25:13 +02:00
Sasi Inguva
0f6e244bb0 libx264: Increase x264 opts character limit to 4096
Signed-off-by: Sasi Inguva <isasi@google.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 282477bf45)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-08-15 20:25:13 +02:00
Michael Niedermayer
9fefd76eec avcodec/flac_parser: Raise threshold for detecting invalid data
Fixes regression from Ticket5428

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit b0b3676e13)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-08-15 18:54:34 +02:00
Michael Niedermayer
40ffbe7678 avcodec/mpc8: Correct end truncation
Fixes Ticket5478

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit b21f674876)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-08-15 18:54:34 +02:00
Michael Niedermayer
ed7fe48445 avcodec/mpegvideo: Do not clear the parse context during init
It is allocated before, this cannot work
Fixes Ticket5613

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 24f5136196)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-08-15 18:54:34 +02:00
Michael Niedermayer
bfca58ee2f avcodec/h264: Fix off by 1 context count
Fixes fate-h264-xavc-4389 with slice threads
(cherry picked from commit 27c1eae55f)

Conflicts:

	libavcodec/h264.c

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-08-15 18:54:34 +02:00
Michael Niedermayer
8a32f19d5b avcodec/alsdec: Check r to prevent out of array read
No testcase known

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit c36fc857b5)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-08-15 18:54:34 +02:00
Umair Khan
1dd34bdb09 avcodec/alsdec: fix max bits in ltp prefix code
The maximum number of bits int the prefix code for
p(0) is 4. By setting it as 3, we were missing the
last 0 bit.
This fixes bug #4715 present on the trac.

Signed-off-by: Umair Khan <omerjerk@gmail.com>
Reviewed-by: Thilo Borgmann <thilo.borgmann@mail.de>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 5d64ba9d18)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-08-15 18:54:34 +02:00
Michael Niedermayer
04987035ff avcodec/utils: check skip_samples signedness
Fixes Ticket5528

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 153ab83bd3)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-08-15 18:54:34 +02:00
Michael Niedermayer
003fa5c3e3 avcodec/bmp_parser: Check fsize
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 43a4276c69)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-08-15 18:54:34 +02:00
Michael Niedermayer
166921c23e avcodec/bmp_parser: reset state
Fixes part of ticket 5598

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 37005e65eb)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-08-15 18:54:34 +02:00
Michael Niedermayer
591c0b527c avcodec/bmp_parser: Fix remaining size
Fixes part of ticket 5598

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 250b620d29)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-08-15 18:54:34 +02:00
Michael Niedermayer
731848ef80 avcodec/bmp_parser: Fix frame_start_found in cross frame cases
Fixes part of ticket 5598

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit bfe945ac3a)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-08-15 18:54:34 +02:00
Michael Niedermayer
ffb503c9a1 avcodec/mpegvideo: Deallocate last/next picture earlier
Fixes regression with mplayers direct rendering and reduces buffer count
pressure in some cases

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 39c0b22df4)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-08-15 18:54:34 +02:00
Michael Niedermayer
2bbbd3e50a avcodec/bmp_parser: Fix state
Fixes Ticket5598

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit d0388bd32e)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-08-15 18:54:34 +02:00
Michael Niedermayer
0aaf66fb2e avcodec/mjpegdec: Do not try to detect last scan but apply idct after all scans for progressive jpeg
Fixes: IMG-20160418-WA0002.jpg

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit deaf58abf2)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-08-15 18:54:34 +02:00
Michael Niedermayer
4e4afe29b9 avcodec/motion_est: Attempt to fix "short data segment overflowed" on IA64
This decreases the MV related encoding table sizes
This should have little effect on real world video encoding performance

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit d7c75a5db0)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-04-29 13:04:17 +02:00
Michael Niedermayer
58a7500492 avcodec/ttaenc: Reallocate packet if its too small
Fixes assertion failure
Fixes Ticket5394

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 005c61c6b8)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-04-27 04:40:12 +02:00
Jan Ekström
3003277103 pgssubdec: fix subpicture output colorspace and range
Functionality used before didn't widen the values from limited to
full range. Additionally, now the decoder uses BT.709 where it
should be used according to the video resolution.

Default for not yet set colorimetry is BT.709 due to most observed
HDMV content being HD.

BT.709 coefficients were gathered from the first two parts of BT.709
to BT.2020 conversion guide in ARIB STD-B62 (Pt. 1, Chapter 6.2.2).
They were additionally confirmed by manually calculating values.

Fixes #4637
(cherry picked from commit 9779b62624)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-04-27 04:40:12 +02:00
Michael Niedermayer
05b33258e3 avcodec/ac3dec: Reset SPX when switching from EAC3 to AC3
Fixes Ticket5319

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 9ac154d1fa)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-04-27 04:40:12 +02:00
Paul B Mahol
e80a4ce69f avcodec/takdec: add code that got somehow lost in process of REing
Signed-off-by: Paul B Mahol <onemda@gmail.com>
(cherry picked from commit 38797a8033)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-04-27 04:40:12 +02:00