The earlier code was based on the assumption that AVFrame.linesize can
not be negative.
Fixes ticket #8280.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit 6770057ac9)
Allocating an AVCodecContext's priv_data used to be the first object
allocated in avcodec_open2(), so it was unnecessary to goto free_and_end
(which does the cleanup) upon error here. But this is no longer so since
f3a29b750a.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit d1dcc20126)
Don't check for AVCodec.priv_data_size (which is always true if
AVCodec.priv_class is set). Instead check for AVCodecContext.priv_data
to actually exist.
(Note: av_opt_free(NULL) is a no-op.)
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit c6e54d14c5)
avcodec_open2() also called the AVCodec's close function if an error
happened before init had ever been called if the AVCodec has the
FF_CODEC_CAP_INIT_CLEANUP flag set. This is against the documentation of
said flag: "The codec allows calling the close function for deallocation
even if the init function returned a failure."
E.g. the SVQ3 decoder is not ready to be closed if init has never been
called.
Fixes: NULL dereference
Fixes: 25762/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_SVQ3_fuzzer-5716279070294016
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit 5bc74d06da)
After all, allocating an AVFloatDSPContext might have failed.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit dac9e88a99)
The JPEG2000 encoder did not clean up after itself on error.
This commit fixes this by modifying the cleanup function to be able to
handle only partially allocated structures and by setting the
FF_CODEC_CAP_INIT_CLEANUP flag.
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit 3d83de4187)
Up until now, the flashsv encoder tried to allocate two buffers in its
init function; if only one of these allocations succeeds, the other
buffer leaks. Fix this by making one of these buffers part of the
context (its size is a compile-time constant).
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit ec6f4c5158)
The ac3 encoders (fixed- and floating-point AC-3 as well as the EAC-3
encoder) all allocate an array whose elements are pointers to other
buffers. The array is not zeroed initially so that if an allocation of
one of the subbuffers fails, the other pointers are uninitialized.
This causes problems when cleaning, so zero the array initially.
(Only the fixed-point AC-3 encoder was affected by this, because
the other two don't clean up at all in case of errors during init.)
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit ae36fad624)
The very first thing the SVQ3 decoder currently does is allocating several
SVQ3Frames, a structure which contains members that need to be freed on
their own. If one of these allocations fails, the decoder calls its own
close function to not leak the already allocated SVQ3Frames. Yet said
function presumes that the SVQ3Frames have been successfully allocated
as there is no check before freeing the members that need to be freed.
This commit fixes this by making these frames part of the SVQ3Context,
thereby avoiding the allocations altogether. Notice that the pointers
to the frames have been retained in order to allow to just swap them as
the code already does.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit 96061c5a4f)
The Sonic decoder and encoders allocate several buffers in their init
function and return immediately if one of these allocations fails; this
will lead to leaks if there was an earlier successfull allocation. Fix
this by setting the FF_CODEC_CAP_INIT_CLEANUP flag.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit 7d91f9271e)
The qtrle encoder allocates several buffers and an AVFrame in its init
function. If one of these allocations fails, but others succeed, the
successfully allocated objects leak. This is fixed by setting the
FF_CODEC_CAP_INIT_CLEANUP flag.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit 2a71cbeb01)
The FFV1 encoder has so far not cleaned up after itself in this case;
but it can be done easily by setting the FF_CODEC_CAP_INIT_CLEANUP flag.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit f9215d0bb2)
When allocating FFV1 slice contexts fails, ff_ffv1_init_slice_contexts()
frees everything that it has allocated, yet it does not reset the
counter for the number of allocated slice contexts. This inconsistent
state leads to segfaults lateron in ff_ffv1_close(), because said
function presumes that the slice contexts have been allocated.
Fix this by making sure that the number of slice contexts on error is
consistent (namely zero).
(This issue only affected the FFV1 decoder, because the encoder does not
clean up after itself on init failure.)
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit a0750f412a)
If an error happens during init after an allocation has succeeded,
the already allocated data leaked up until now. Fix this by setting the
FF_CODEC_CAP_INIT_CLEANUP flag.
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit b78031cf16)
ff_ivi_init_planes() might error out after having allocated some arrays.
Set the FF_CODEC_CAP_INIT_CLEANUP flag in order to free these arrays in
this case.
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit d8fc69bd6a)
If allocating the tiles array for indeo 4/5 fails, the context is in an
inconsistent state, because the counter for the number of tiles is > 0.
This will lead to a segfault when freeing the tiles' substructures.
Fix this by setting the number of tiles to zero if the allocation was
unsuccessful.
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit e411a3af11)
If an error happens during init after an allocation has succeeded,
the already allocated data leaked up until now. Fix this by setting the
FF_CODEC_CAP_INIT_CLEANUP flag.
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit a8ebb56320)
If one of several allocations the gif encoder performs in its init
function fails, the successful allocations leak. Fix this by adding the
FF_CODEC_CAP_INIT_CLEANUP flag.
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit 8da8774d8e)
If ff_codec_open2_recursive() fails, the already allocated
AVCodecContext leaks. Fix this by setting the FF_CODEC_CAP_INIT_CLEANUP
flag.
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit 3c2128df7f)
Move it to the context instead.
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit 2777bae7f2)
The init function first allocates an AVFrame and then some buffers; if
one of the buffers couldn't be allocated, the AVFrame leaks. Solve this
by setting the FF_CODEC_CAP_INIT_CLEANUP flag.
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit 4f67288948)
The Winnov WNV1 format is designed for a little-endian bitstream reader;
yet our decoder reversed every byte bitwise (in a buffer only
allocated for this purpose) to use a big-endian bitstream reader. This
commit stops this.
Two things needed to be done to achieve this: The codes in the table used
to initialize a VLC reader needed to be reversed bitwise (when
initializing a VLC in LE mode, it is expected that the first bit to be
read is in the least significant bit; with BE codes the first bit to be
read is the most significant bit of the code) and the following
expression needed to be adapted:
ff_reverse[get_bits(&w->gb, 8 - w->shift)]
But this is easy: When only the bits read are reversed, they coincide
with what a little-endian bitstream reader reads that reads the
original, not-reversed data. But ff_reverse always reverses the full
eight bits and this also performs a shift by (8 - (8 - w->shift)) on top
of reversing the bits read. So the above line needs to be changed to
get_bits(&w->gb, 8 - w->shift) << w->shift
and this also shows why the variable shift is named the way it is.
Finally, this also fixes a hypothetical memleak: For gigantic packets,
initializing a GetBitContext can fail and in this case, the buffer
containing the reversed data would leak.
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit 0166b1d1a6)
TrueMotion 2.0 uses Huffmann trees. To parse them, the decoder allocates
arrays for the codes, their lengths and their value; afterwards a VLC
table is initialized using these values. If everything up to this point
succeeds, a new buffer of the same size as the already allocated arrays
for the values is allocated and upon success the values are copied into
the new array; all the old arrays are then freed. Yet if allocating the
new array fails, the old arrays get freed, but the VLC table doesn't.
This leak is fixed by not allocating a new array at all; instead the old
array is simply reused, ensuring that nothing can fail after the
creation of the VLC table.
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit 5ff2ff6bd9)
Fixes: out of array access
Fixes: 29345/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_HAP_fuzzer-5401813482340352
Fixes: 30745/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_HAP_fuzzer-5762798221131776
Suggested-by: Anton
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 89fe1935b1)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: Timeout (long -> 5sec)
Fixes: 30269/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_VP7_fuzzer-5430325004075008
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Reviewed-by: Peter Ross <pross@xvid.org>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 6a797ceafe)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: signed integer overflow: 9223372036854775807 - -30069403896 cannot be represented in type 'long'
Fixes: 30046/clusterfuzz-testcase-minimized-ffmpeg_dem_FITS_fuzzer-5807144773484544
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit c000a91288)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: out of array access
Fixes: 30135/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_PBM_fuzzer-4997145650397184
Fixes: 30208/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_PGMYUV_fuzzer-5605891665690624.fuzz
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 5314a4996c)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: Timeout
Fixes: left shift of 33046 by 16 places cannot be represented in type 'int'
Fixes: 29258/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_MJPEG_fuzzer-4889231489105920
Fixes: 29515/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_MJPEG_fuzzer-6161940391002112
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 980900d991)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: out of array access
Fixes: 29392/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_HEVC_fuzzer-4821602850177024.fuzz
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 0791a515d3)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: signed integer overflow: 2147483641 + 32 cannot be represented in type 'int'
Fixes: 27452/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_VP9_fuzzer-5078752576667648
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 4dfb7ff528)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: out of array access
Fixes: 27434/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_RSCC_fuzzer-5196757675540480
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
(cherry picked from commit a5ed6da9bd)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: signed integer overflow: -1498310196 - 902891776 cannot be represented in type 'int'
Fixes: 28445/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_VC1IMAGE_fuzzer-5075163389493248
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 57f7e5caa3)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: Timeout
Fixes: 29226/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_THEORA_fuzzer-6195092572471296
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Reviewed-by: Peter Ross <pross@xvid.org>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 869fe41d10)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: signed integer overflow: -2147483648 - 4 cannot be represented in type 'int'
Fixes: 26907/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_CFHD_fuzzer-5746202330267648
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 386faeda5f)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
The entry read is not used in subsequent computation, thus its
value is not important.
Fixes: out of array read
Fixes: 28578/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_SIREN_fuzzer-6332019122503680
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit f3e4ebb007)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: signed integer overflow: 131203586 * 28 cannot be represented in type 'int'
Fixes: 26817/clusterfuzz-testcase-minimized-ffmpeg_dem_MSF_fuzzer-6296902548848640
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 2488ba85a0)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: left shift of negative value -25824
Fixes: 27754/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_XMA2_fuzzer-5760255962906624
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 69aeba8a19)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: signed integer overflow: -210824 * 16384 cannot be represented in type 'int'
Fixes: 28670/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_ALS_fuzzer-5682310846480384
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 7ce40dde03)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: assertion failure
Fixes: 28447/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_FFV1_fuzzer-5369575948550144
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 5cae71d2b7)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: signed integer overflow: 617890810133996544 * 16 cannot be represented in type 'long'
Fixes: 26565/clusterfuzz-testcase-minimized-ffmpeg_dem_MV_fuzzer-5092054700654592
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 8aadae670f)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: signed integer overflow: 2147483640 + 8 cannot be represented in type 'int'
Fixes: 28449/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_HEVC_fuzzer-5686013259284480
Reviewed-by: James Almer <jamrial@gmail.com>
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit f1700bd8bb)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
