Commit graph

40378 commits

Author SHA1 Message Date
Michael Niedermayer
b95c209a61 avcodec/scpr: Check for min > max in decompress_p()
Fixes: Timeout
Fixes: 9342/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_SCPR_fuzzer-4795990841229312

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 3378194ce8)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2018-10-07 21:34:00 +02:00
Michael Niedermayer
cd42c19f53 avcodec/shorten: Fix signed 32bit overflow in shift in shorten_decode_frame()
Fixes: runtime error: left shift of 1 by 31 places cannot be represented in type 'int'
Fixes: 9480/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_SHORTEN_fuzzer-6647324284551168 -rss_limit_mb=2000

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 9b604e96a5)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2018-10-07 21:34:00 +02:00
Michael Niedermayer
32353f8bcb avcodec/shorten: Fix integer overflow in residual/LPC combination
Fixes: signed integer overflow: -540538872 + -2012739576 cannot be represented in type 'int'
Fixes: 9255/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_SHORTEN_fuzzer-5758630052757504

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit db7e9082e1)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2018-10-07 21:34:00 +02:00
Michael Niedermayer
2c3ea34082 avcodec/shorten: Check verbatim length
Fixes: Timeout
Fixes: 9252/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_SHORTEN_fuzzer-5780720709533696

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 7007dabec0)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2018-10-07 21:34:00 +02:00
Michael Niedermayer
c0a6febf32 avcodec/mpegaudio_parser: Initialize poutbuf*
Possibly fixes: null pointer dereference
Possibly fixes: 9352/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_MP3ADUFLOAT_fuzzer-5146068961460224
Fixes: Heap-use-after-free
Fixes: 9453/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_MP3ADUFLOAT_fuzzer-5137954375729152

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 0f4c3b0b8e)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2018-10-07 21:34:00 +02:00
Michael Niedermayer
cc0817af0d avcodec/aacpsdsp_template: Fix integer overflow in ps_stereo_interpolate_c()
Fixes: signed integer overflow: -1813244069 + -1407981383 cannot be represented in type 'int'
Fixes: 8823/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_AAC_FIXED_fuzzer-5643295618236416

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 47db5763e2)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2018-10-07 21:34:00 +02:00
Nikolas Bowe
8591d16ce5 lavc/svq3: Fix regression decoding some files.
Fixes some SVQ3 encoded files which fail to decode correctly after 6d6faa2a2d.
These files exhibit lots of artifacts and logs show "Media key encryption is not implemented".
However they decode without artifacts before 6d6faa2a2d.
The attatched patch allows these files to successfully decode, but also reject media key files.

Tested on the files in #6094 and http://samples.mplayerhq.hu/V-codecs/SVQ3/Vertical400kbit.sorenson3.mov

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 5aeb3b0080)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2018-10-07 21:34:00 +02:00
Michael Niedermayer
07255282d0 avcodec/mlp_parser: Check if synccode is within buffer
Fixes: undefined shift
Fixes: 9216/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_TRUEHD_fuzzer-6281404575907840

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 51ac3f43b8)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2018-10-07 21:34:00 +02:00
Michael Niedermayer
f9235773d6 avcodec/qtrle: Check remaining bytestream in qtrle_decode_XYbpp()
Fixes: Timeout
Fixes: 9213/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_QTRLE_fuzzer-5649753332252672

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 7dd836a3f9)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2018-10-07 21:34:00 +02:00
Michael Niedermayer
4f51a21c30 avcodec/diracdec: Check bytes count in else branch in decode_lowdelay() too
Fixes: signed integer overflow: 8 * 340018243 cannot be represented in type 'int'
Fixes: 9441/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_DIRAC_fuzzer-5194665207791616

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit bed125b710)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2018-10-07 21:34:00 +02:00
Michael Niedermayer
6cf72a56e7 avcodec/diracdec: Check slice numbers for overflows in relation to picture dimensions
Fixes: signed integer overflow: 88 * 33685506 cannot be represented in type 'int'
Fixes: 9433/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_DIRAC_fuzzer-5725943535501312

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit f457c0ad7f)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2018-10-07 21:34:00 +02:00
Michael Niedermayer
88093d2c1f avcodec/diracdec: Change frame_number to 64bit as its a 32bit from the bitstream and we also have a -1 special case
Fixes: signed integer overflow: 2147483647 + 1 cannot be represented in type 'int'
Fixes: 9291/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_DIRAC_fuzzer-6324345860259840

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 462d1be6de)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2018-10-07 21:34:00 +02:00
Michael Niedermayer
8147da2bad avcodec/dirac_dwt_template: Fix several integer overflows in horizontal_compose_daub97i()
Fixes: signed integer overflow: 2147483647 + 1 cannot be represented in type 'int'
Fixes: 8926/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_DIRAC_fuzzer-6047609228623872

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 69cac9e130)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2018-10-07 21:34:00 +02:00
Michael Niedermayer
f291acafbb avcodec/diracdec: Prevent integer overflow in intermediate in global_mv()
Fixes: signed integer overflow: -393471 * 5460 cannot be represented in type 'int'
Fixes: 8890/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_DIRAC_fuzzer-6299775379963904

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 5129040646)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2018-10-07 21:34:00 +02:00
Thilo Borgmann
49a90d5d31 lavc/videotoolboxenc: Fix compilation on osx 10.10.5 Yosemite
Signed-off-by: Aman Gupta <aman@tmm1.net>
(cherry picked from commit 72d9b8f4c5)
2018-08-14 10:02:05 -07:00
Aman Gupta
fab3418cb9 avcodec/mediacodecdec: fix SEGV on modern nvidia decoders
This code came originally from gstreamer, where it was added in [1]
as a work-around for the Tegra 3. (The alignment was changed in [2]
as a response to [3], from 32-bit to 16-bit).

gstreamer only used this workaround in the case where the decoder
didn't return a slice-height property, but when the code was copied
into avcodec the conditional got lost. This commit restores the guard
and prefers the slice-height from the decoder when it is available.

This fixes segfaults decoding 1920x1080 h264 and mpeg2 videos on the
NVidia SHIELD after upgrading to Android Oreo.

[1] a870e6a5c3
[2] 21ff3ae0b0
[3] https://bugzilla.gnome.org/show_bug.cgi?id=748867

Signed-off-by: Aman Gupta <aman@tmm1.net>
(cherry picked from commit 476fd6ba3a)
2018-08-03 11:07:31 -07:00
James Almer
9cc5337247 avcodec/bitstream_filters: check the input argument of av_bsf_get_by_name() for NULL
Fixes crashes like "ffmpeg -h bsf" caused by passing NULL to strcmp()

Signed-off-by: James Almer <jamrial@gmail.com>
(cherry picked from commit 3258cc6507)
2018-07-28 22:43:09 -03:00
Thomas Guillem
db923b3fbd avcodec/videotoolboxenc: fix undefined behavior with rc_max_rate=0
On macOS, a zero rc_max_rate cause an error from
VTSessionSetProperty(kVTCompressionPropertyKey_DataRateLimits).

on iOS (depending on device/version), a zero rc_max_rate cause invalid
arguments from the vtenc_output_callback after few frames and then a crash
within the VideoToolbox library.

Signed-off-by: Aman Gupta <aman@tmm1.net>
(cherry picked from commit 93e157f40f)
2018-07-19 09:26:30 -07:00
Michael Niedermayer
3ef38c414e avcodec/dvdsub_parser: Allocate input padding
Fixes: out of array read
Fixes: 9350/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_DVDSUB_fuzzer-5746777750765568

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit cd86b5cfe2)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2018-07-16 19:02:12 +02:00
Michael Niedermayer
40ed40902a avcodec/dvdsub_parser: Init output buf/size
No testcase

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 9e6c843776)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2018-07-16 19:02:12 +02:00
Michael Niedermayer
0561cde128 avcodec/dirac_dwt_template: Fix signedness regression in interleave()
Found-by: <jdarnley>
Tested-by: James Darnley <james.darnley@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 181435a4de)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2018-07-16 19:02:12 +02:00
Michael Niedermayer
d8c4b2ae57 avcodec/mjpegdec: Check for odd progressive RGB
Fixes: out of array access
Fixes: 9225/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_JPEGLS_fuzzer-5684770334834688

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit ee1e3ca5eb)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2018-07-16 19:02:12 +02:00
Michael Niedermayer
fc92ca5b8e avcodec/vp8_parser: Do not leave data/size uninitialized
This is identical to what the VP9 parser does

Fixes: 9215/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_LIBVPX_VP8_fuzzer-5768227253649408
Fixes: out of memory access

This may also fix oss fuzz issue 9212

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 284dde24da)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2018-07-16 19:02:12 +02:00
Michael Niedermayer
4439d6aa69 avcodec/mpeg4videodec: Check for bitstream end in read_quant_matrix_ext()
Fixes: out of array read
Fixes: asff-crash-0e53d0dc491dfdd507530b66562812fbd4c36678

Found-by: Paul Ch <paulcher@icloud.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 5aba5b89d0)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2018-07-16 19:02:12 +02:00
Michael Niedermayer
3bf80c7b22 avcodec/indeo4: Check for end of bitstream in decode_mb_info()
Fixes: Timeout
Fixes: 8776/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_INDEO4_fuzzer-5361788798369792

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 267ba2aa96)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2018-07-16 19:02:12 +02:00
Michael Niedermayer
1361e4abb8 avcodec/ac3dec: Check channel_map index
Fixes: out of array read
Fixes: 8924/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_EAC3_fuzzer-5851861780267008

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 00f98d23b1)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2018-07-16 19:02:12 +02:00
Michael Niedermayer
5fd1dce39a avcodec/mpeg4videodec: Remove use of FF_PROFILE_MPEG4_SIMPLE_STUDIO as indicator of studio profile
The profile field is changed by code inside and outside the decoder,
its not a reliable indicator of the internal codec state.
Maintaining it consistency with studio_profile is messy.
Its easier to just avoid it and use only studio_profile

Fixes: assertion failure
Fixes: ffmpeg_crash_9.avi

Found-by: Thuan Pham, Marcel Böhme, Andrew Santosa and Alexandru Razvan Caciulescu with AFLSmart
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit bd27a9364c)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2018-07-16 19:02:12 +02:00
Michael Niedermayer
de0a1d01ba avcodec/shorten: Fix undefined addition in shorten_decode_frame()
Fixes: signed integer overflow: 1139785606 + 1454196085 cannot be represented in type 'int'
Fixes: 8937/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_SHORTEN_fuzzer-6202943597445120

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 3b10bb8772)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2018-07-16 19:02:12 +02:00
Michael Niedermayer
c4b23793d4 avcodec/shorten: Fix undefined integer overflow
Fixes: signed integer overflow: 8454144 * 256 cannot be represented in type 'int'
Fixes: 8788/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_SHORTEN_fuzzer-5728205041303552

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 70832333bb)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2018-07-16 19:02:12 +02:00
Michael Niedermayer
e21e5c95c1 avcodec/jpeg2000dec: Fixes invalid shifts in jpeg2000_decode_packets_po_iteration()
Fixes: shift exponent 47 is too large for 32-bit type 'int'
Fixes: 9163/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_JPEG2000_fuzzer-5661750182543360

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 652d7c6348)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2018-07-16 19:02:12 +02:00
Michael Niedermayer
2b13c136c4 avcodec/jpeg2000dec: Check that there are enough bytes for all tiles
Fixes: OOM
Fixes: 8781/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_JPEG2000_fuzzer-5810709081358336

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 0898a3d990)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2018-07-16 19:02:12 +02:00
Michael Niedermayer
679d749eab avcodec/mpeg4videodec: Check read profile before setting it
Fixes: null pointer dereference
Fixes: ffmpeg_crash_7.avi

Found-by: Thuan Pham, Marcel Böhme, Andrew Santosa and Alexandru Razvan Caciulescu with AFLSmart
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 2aa9047486)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2018-07-16 19:02:12 +02:00
Michael Niedermayer
0003ace83b avcodec/ac3_parser: Check init_get_bits8() for failure
Fixes: null pointer dereference
Fixes: ffmpeg_crash_6.avi

Found-by: Thuan Pham, Marcel Böhme, Andrew Santosa and Alexandru Razvan Caciulescu with AFLSmart
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 00e8181bd9)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2018-07-16 19:02:12 +02:00
Michael Niedermayer
841e1399e6 avcodec/dpx: Check elements in 12bps planar path
Fixes: null pointer dereference
Fixes: 8946/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_DPX_fuzzer-5078915222601728

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Reviewed-by: Carl Eugen Hoyos <ceffmpeg@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 75a2db5524)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2018-07-16 19:02:12 +02:00
Michael Niedermayer
9f76f0fab8 avcodec/escape124: Fix spelling errors in comment
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit f59c4e4391)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2018-07-16 19:02:12 +02:00
Michael Niedermayer
bf3e331b76 avcodec/ra144: Fix integer overflow in ff_eval_refl()
Fixes: signed integer overflow: -4096 * -524288 cannot be represented in type 'int'
Fixes: 8650/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_RA_144_fuzzer-5734816036159488

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit b31189881a)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2018-07-16 19:02:12 +02:00
Michael Niedermayer
21732c1adc avcodec/cscd: Check output buffer size for lzo.
Fixes: Timeout
Fixes: 8665/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_CSCD_fuzzer-5768442610188288

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
(cherry picked from commit 78167b498f)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2018-07-16 19:02:12 +02:00
Michael Niedermayer
d00548f2c1 avcodec/escape124: Check buf_size against num_superblocks
Fixes: Timeout
Fixes: 8722/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_ESCAPE124_fuzzer-4843268402577408

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 6677c98626)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2018-07-16 19:02:12 +02:00
Michael Niedermayer
10e5302db4 avcodec/h264_parser: Reduce needed history for parsing mb index
This fixes a bug/regression with very small packets
Fixes: output_file

Regression since: 0782fb6bcb

Reported-by: Thierry Foucu <tfoucu@google.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit d25c945247)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2018-07-16 19:02:12 +02:00
Michael Niedermayer
84280dc7cf avcodec/magicyuv: Check bits left in flags&1 branch
Fixes: Timeout
Fixes: 8690/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_MAGICYUV_fuzzer-6542020913922048

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 7719b8ccc7)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2018-07-16 19:02:12 +02:00
Michael Niedermayer
2c404cc11a avcodec/mjpegdec: Check for end of bitstream in ljpeg_decode_rgb_scan()
Fixes: Timeout
Fixes: 8648/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_MJPEG_fuzzer-5108395525799936

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 540e8c2d64)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2018-07-16 19:02:12 +02:00
Aman Gupta
789bac72ed Revert "avcodec/mediacodecdec: wait on first frame after input buffers are full"
@xyz reported a regression on his Sony Xperia Z3 Tablet Compact where
playback would intermittently fail to start, essentially deadlocking in
the decoder. Bisecting narrowed down the issue to this commit, which was
meant as an optimization but is not necessary.

This reverts commit a75bb5496a.

Signed-off-by: Aman Gupta <aman@tmm1.net>
(cherry picked from commit 37c2cb6a68)
2018-06-18 12:01:12 -07:00
Thomas Guillem
33fcbb4372 avcodec/videotoolboxenc: fix invalid session on iOS
Cf. comment. Restart the VT session when the APP goes from foreground to
background and vice versa.

Signed-off-by: Aman Gupta <aman@tmm1.net>
(cherry picked from commit 513e6a30fb)
2018-06-18 11:49:38 -07:00
Thomas Guillem
a56eb4d56c avcodec/videotoolboxenc: split initialization
Split vtenc_init() into vtenc_init() (VTEncContext initialization) and
vtenc_configure_encoder() (creates the vt session).

This commit will allow to restart the vt session while encoding.

Signed-off-by: Aman Gupta <aman@tmm1.net>
(cherry picked from commit 9e11d27c25)
2018-06-18 11:49:38 -07:00
Thomas Guillem
70799fae35 avcodec/videotoolboxenc: fix mutex/cond leak in error path
The leak could happen when the vtenc_create_encoder() function failed.

Signed-off-by: Aman Gupta <aman@tmm1.net>
(cherry picked from commit ce2330bdf8)
2018-06-18 11:49:38 -07:00
Michael Niedermayer
83a737aa70 avcodec/aacdec_fixed: Fix undefined integer overflow in apply_independent_coupling_fixed()
Fixes: signed integer overflow: 1195517 * 2048 cannot be represented in type 'int'
Fixes: 8636/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_AAC_FIXED_fuzzer-4695836326887424

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 8bd514d934)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2018-06-15 22:31:13 +02:00
Michael Niedermayer
2deeb2eaef avcodec/dirac_dwt_template: Fix undefined behavior in interleave()
Fixes: signed integer overflow: 2147483647 + 1 cannot be represented in type 'int'
Fixes: 8697/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_DIRAC_fuzzer-5197148130902016

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 575d8ca026)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2018-06-15 22:31:13 +02:00
Michael Niedermayer
897524954b avcodec/mpeg4videodec: Clear bits_per_raw_sample if it has originated from a previous instance
Fixes: assertion failure
Fixes: ffmpeg_crash_5.avi

Found-by: Thuan Pham <thuanpv@comp.nus.edu.sg>, Marcel Böhme, Andrew Santosa and Alexandru Razvan Caciulescu with AFLSmart
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 2fc108f60f)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2018-06-15 22:31:13 +02:00
Michael Niedermayer
e42ab0115e avcodec/h264_slice: Fix overflow in recovery_frame computation
Fixes: signed integer overflow: 15 + 2147483646 cannot be represented in type 'int'
Fixes: 8381/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_H264_fuzzer-6225533137321984

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 8c20ea8ee0)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2018-06-15 22:31:13 +02:00
Michael Niedermayer
10f68641ae avcodec/h264_ps: Move MAX_LOG2_MAX_FRAME_NUM to header so it can be used in h264_sei
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit b796c5ae92)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2018-06-15 22:31:13 +02:00