Affected the ffmpeg-filter_colorkey FATE-test (but only if the C version
of idct8_add is used and not e.g. the x86 SSE2 version).
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Fixes: left shift of negative value -107
Fixes: 20398/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_CAVS_fuzzer-5725389278412800
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 0f0f2ab0c3)
by replacing it with a multiplication. Said multiplication can't
overflow an int32_t because lpc_coefs is limited to 16 bit precision.
Fixes the FACE-test acodec-ra144 as well as part of #8217.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit e3fb9af6f1)
Replace "((a << shift) + b) >> shift" by "a + (b >> shift)". This avoids
a left shift which also happens to trigger undefined behaviour in case "a"
is negative. This affected the FATE-tests acodec-adpcm-adx and
acodec-adpcm-adx-trellis; it also fixes ticket #8008.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 59a9d65e0d)
Affected the adpcm-afc, adpcm-ea-1, adpcm-ea-2, adpcm-ea-maxis-xa,
adpcm-thp and ea-cdata FATE-tests. Also fixes ticket #8487.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 3ad8af51b7)
This fixes ticket #7997 as well as the vsynth*-prores_# FATE-tests
(where * ranges over { 1, 2, 3, _lena } and # over { , _int, _444,
_444_int }).
(Given that prev_dc is in the range -0xC000..0x3FFF, no overflow can
happen upon multiplication with 2.)
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 198081efb7)
Fixes: left shift of 1 by 31 places cannot be represented in type 'int'.
Affected the FATE-tests vsynth1-cinepak, vsynth2-cinepak and
vsynth_lena-cinepak. Also fixes ticket #8220.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit e3650dcfc9)
Affected the vsynth*-jpeg2000 and the vsynth*-jpeg2000-97 FATE tests
(where * ranges over { 1, 2, 3, _lena }) as well as ticket #7983.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 5cf593adcd)
Affected the FATE-tests exr-rgb-scanline-pxr24-half-uint32-13x9 and
exr-rgb-scanline-pxr24-uint32.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 8b0f949906)
Fixes ticket #8161 and the acodec-wavpack FATE-test.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit 84974c6fb5)
Fixes the FATE-tests unknown_layout-ac3, ac3-fixed-encode, ac3-encode
and eac3-encode. It furthermore fixes the ac3-encoder bugs mentioned in
tickets #7994, #8144 and #8159.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 2f289ec914)
Fixes the FATE-tests g723_1-dec-1, g723_1-dec-2 and g723_1-dec-4.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 54e1efda6d)
Fixes the tdsc FATE-test.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 86bef10e7a)
ttaenc contained (1 << unary) - 1 as an argument for a function
expecting an unsigned int. unary can be as big as 31 in this case.
The type of the shift and the whole expression is int, because 1 fits
into an integer, so that the behaviour is undefined if unary == 31
as the result of the shift can't be represented in an int §. Subtraction
by 1 (which makes the result of the whole expression representable in
an int) doesn't change that this is undefined (it usually leads to
signed integer overflow which is undefined, too).
The solution is simple: Make 1 unsigned to change the type of the
whole expression to unsigned int (as the function expects anyway).
Fixes ticket #8153.
§: This of course presupposes the common int range of -2^31..2^31-1
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 3ab488a540)
3469cfab added a check for whether the extradata coincided with the
beginning of the packet's data in order not to add extradata to packets
that already have it. But the check used was buggy for packets whose
size is smaller than the extradata's size. This commit fixes this.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit a88a3cdb4b)
zmbvenc allocates a buffer for a picture with padding on all four sides:
The stride is choosen so large that it already contains padding on the
right; the height also includes padding rows. The padding on the right
of each row is also reused as padding for the left of the next row. So
one still needs to add padding on the left for the first row. This is done
by offsetting the actual pointer used to access the picture from the
pointer returned by av_mallocz and the formula for this offset was
wrong, because it ignored that a pixel can take more than one byte when
calculating the offset resulting from the left padding of the first row.
This fixes accesses outside of the allocated buffer that were reported
in tickets #7980 and #7994. No writes were ever attempted outside of
the buffer.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Reviewed-by: Tomas Härdin <tjoppen@acc.umu.se>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit def04022f4)
The maximum allowed index for an array access is FF_ARRAY_ELEMS - 1; yet
the current code allowed FF_ARRAY_ELEMS. This wasn't dangerous in practice,
as parameter sets with invalid ids were already filtered out during
reading.
Found via PVS-Studio (see ticket #8156).
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit f3333c3c67)
Also the patch makes this code consistent with mpeg4videodec.c
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit f163d30de2)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: signed integer overflow: 2147483647 + 1 cannot be represented in type 'int'
Fixes: 19950/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_BINKAUDIO_DCT_fuzzer-5765514337189888
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Suggested-by: Paul
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit f603d10b1e)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
The reference (thp.txt) uses floats so wrap around would seem incorrect.
Fixes: signed integer overflow: 1073741824 + 1073741824 cannot be represented in type 'int'
Fixes: 20658/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_ADPCM_THP_fuzzer-5646302555930624
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit b12b05374f)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: out of array access
Fixes: 20659/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_RALF_fuzzer-5739471895265280
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit f0c0471075)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: Out of array access
Fixes: 20659/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_IFF_ILBM_fuzzer-5658548592967680
Fixes: 20659/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_IFF_ILBM_fuzzer-5723561177382912
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit a035fd88ae)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: signed integer overflow: 2147483594 + 142 cannot be represented in type 'int'
Fixes: 20492/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_UTVIDEO_fuzzer-5658568101724160
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 876cfa67f3)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: signed integer overflow: 1931744255 + 252497024 cannot be represented in type 'int'
Fixes: 20492/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_TTA_fuzzer-5763348114440192
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 8627885172)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: signed integer overflow: 289082077 - -2003141111 cannot be represented in type 'int'
Fixes: 20492/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_RALF_fuzzer-5196077752123392
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 0c4330847c)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: signed integer overflow: 65312 * 65312 cannot be represented in type 'int'
Fixes: 20492/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_NUV_fuzzer-5740176118906880
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 1ac106bf56)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: negation of -2147483648 cannot be represented in type 'int32_t' (aka 'int'); cast to an unsigned type to negate this value to itself
Fixes: signed integer overflow: -2147483648 - 1 cannot be represented in type 'int32_t' (aka 'int')
Fixes: 20492/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_IFF_ILBM_fuzzer-5764066459254784
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 7a92147f87)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
The fixed point integer reference specifies the multiplication used
to have 16bit input and clips so we need to clip the input
The floating point implementation does not seem to do that.
Fixes: signed integer overflow: 6317568 * 410 cannot be represented in type 'int'
Fixes: 20492/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_G729_fuzzer-5700189272932352
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 82d4c7b95e)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: signed integer overflow: 1020048 * 4096 cannot be represented in type 'int'
Fixes: 20492/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_ALAC_fuzzer-5753877751660544
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 22e51e95ac)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: out of array access
Fixes: 20659/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_DST_fuzzer-5735812071424000
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 5727b1f13f)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: 21089/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_APNG_fuzzer-5135981419429888
Fixes: out of array read
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 79e5c2ee2b)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: out of array access
Fixes: 21469/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_G2M_fuzzer-5199357982015488
Alternatively the arrays can be made bigger or the index can be clipped.
In case a real file with such huge tiles exist we ask the user to upload it.
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 5501bb28dd)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: signed integer overflow: -193177 * 11585 cannot be represented in type 'int'
Fixes: 20557/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_VP9_fuzzer-5704852816789504
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit e16e3e63f0)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: left shift of negative value -1
Fixes: 21390/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_ALAC_fuzzer-6242539519868928
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 49ae034b42)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: left shift of negative value -8321365
Fixes: 20506/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_WMALOSSLESS_fuzzer-4798062906310656
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 62e4003780)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: signed integer overflow: -16 * 134217879 cannot be represented in type 'int'
Fixes: 20492/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_DST_fuzzer-5639509530378240
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 2d465a401d)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: out of array access
Fixes: 20446/clusterfuzz-testcase-minimized-ffmpeg_BSF_HEVC_METADATA_fuzzer-5707770718584832
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 588114cea4)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: out of array read
Fixes: 20796/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_IFF_ILBM_fuzzer-5111364702175232.fuzz
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit b4a33387cb)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Reviewed-by: Peter Ross <pross@xvid.org>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit bc41a29a5a)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Found while reviewing a patch fixing a similar issue
Reviewed-by: Anton Khirnov <anton@khirnov.net>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 4c7bcaa385)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: signed integer overflow: 40550400 * 128 cannot be represented in type 'int'
Fixes: 20331/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_RV40_fuzzer-5676685725007872
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 13171ad2e3)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
