Commit graph

41421 commits

Author SHA1 Message Date
Michael Niedermayer
20c0ea160e avcodec/wmavoice: sanity check block_align
This limit is roughly based on the bitreader limit, its likely a much tighter limit
could be used

Fixes: left shift of 1965039647 by 1 places cannot be represented in type 'int'
Fixes: 19545/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_WMAVOICE_fuzzer-5695391899320320

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 6847e22c8c)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2020-07-03 12:10:23 +02:00
Michael Niedermayer
958db2c438 avcodec/pcm: Fix invalid shift in pcm_decode_frame for LXF
Fixes: left shift of 32 by 28 places cannot be represented in type 'int'
Fixes: 19472/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_PCM_LXF_fuzzer-5704364320096256

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 985d3666f6)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2020-07-03 12:10:23 +02:00
Michael Niedermayer
bc77d29d9e avcodec/snappy: Sanity check bytestream2_get_levarint()
Fixes: left shift of 79 by 28 places cannot be represented in type 'int'
Fixes: 20202/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_HAP_fuzzer-5719004081815552
Fixes: 20219/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_HAP_fuzzer-5641738677125120
Fixes: 20389/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_HAP_fuzzer-5680721517871104

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit be54da2117)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2020-07-03 12:10:23 +02:00
Michael Niedermayer
5e83098085 avcodec/mlpdsp: Fix a invalid shift in ff_mlp_rematrix_channel()
Fixes: left shift of negative value -2
Fixes: 20305/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_TRUEHD_fuzzer-5677196618498048

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Reviewed-by: Jai Luthra <me@jailuthra.in>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit fcc9f13717)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2020-07-03 12:10:23 +02:00
Michael Niedermayer
51c084bf72 avcodec/avdct: Clear IDCTDSPContext context
Fixes use of uninitialized variable and segfault

Reviewed-by: Paul B Mahol <onemda@gmail.com>
Reviewed-by: James Almer <jamrial@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit b82825eba8)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2020-07-03 12:10:23 +02:00
Michael Niedermayer
9ed9c8cc51 avcodec/x86/diracdsp: Fix high bits on Windows x86_64
Found-by: james
(cherry picked from commit 24af459d1e)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2020-07-03 12:10:23 +02:00
Michael Niedermayer
862c0048ec avcodec/wmalosslessdec: Fix multiple integer overflows
Fixes: left shift of 3329 by 20 places cannot be represented in type 'int'
Fixes: signed integer overflow: -199378355 + -1948950833 cannot be represented in type 'int'
Fixes: 19837/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_WMALOSSLESS_fuzzer-5752565837070336
Fixes: 19839/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_WMALOSSLESS_fuzzer-5767483265122304

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 422202516c)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2020-07-03 12:10:23 +02:00
Michael Niedermayer
109cab1487 avcodec/apedec: Fix undefined integer overflow in decode_array_0000()
Fixes: signed integer overflow: -2143289344 - 6246400 cannot be represented in type 'int'
Fixes: 19239/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_APE_fuzzer-5173755680915456

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit a3655bb02c)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2020-07-03 12:10:23 +02:00
Michael Niedermayer
b7c2f4d298 avcodec/smacker: Check space before decoding type
Fixes: Timeout (232sec -> 280ms)
Fixes: 19682/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_SMACKER_fuzzer-5654129649385472

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 6f5c18da59)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2020-07-03 12:10:23 +02:00
Michael Niedermayer
3f42af0342 avcodec/rawdec: Use linesize in b64a
Fixes: out of array access
Fixes: 19750/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_RAWVIDEO_fuzzer-5074834119983104

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 2b5b9d5dac)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2020-07-03 12:10:23 +02:00
Michael Niedermayer
57278dac15 avcodec/iff: Over-allocate ham_palbuf for HAM6 IFF-PBM
IFF-PBM-HAM6 can read out of array without this overallocation
Fixes: Out of array read
Fixes: 19752/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_IFF_ILBM_fuzzer-5675331403120640

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 8652f4e7a1)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2020-07-03 12:10:23 +02:00
Michael Niedermayer
e1bea6d412 avcodec/x86/diracdsp: Fix incorrect src addressing in dequant_subband_32()
Fixes: Segfault (not reproducable with asm, which made this hard to debug)
Fixes: decoding errors
Fixes: 19854/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_DIRAC_fuzzer-5729372837511168

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 0694b60b7b)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2020-07-03 12:10:23 +02:00
Michael Niedermayer
88326e29dc avcodec/hevc_mp4toannexb_bsf: check that nalu size doesnt overflow
Fixes: Out of array access
Fixes: 19299/clusterfuzz-testcase-minimized-ffmpeg_BSF_HEVC_MP4TOANNEXB_fuzzer-5169193398042624

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit a8ceb2a72f)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2020-07-03 12:10:23 +02:00
Michael Niedermayer
a0d0e9b245 avcodec/hevc_mp4toannexb_bsf: Avoid NULL memcpy()
Fixes: invalid memcpy use
Fixes: 19299/clusterfuzz-testcase-minimized-ffmpeg_BSF_HEVC_MP4TOANNEXB_fuzzer-5169193398042624

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 1e23b5a706)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2020-07-03 12:10:23 +02:00
Michael Niedermayer
831ac9ad46 avcodec/cbs_av1: Check leb128 values read
"It is a requirement of bitstream conformance that the value returned from the leb128 parsing process is less than or equal
to (1 << 32) - 1."

Fixes: assertion failure
Fixes: 19293/clusterfuzz-testcase-minimized-ffmpeg_BSF_TRACE_HEADERS_fuzzer-5749508361420800

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit a70d836364)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2020-07-03 12:10:23 +02:00
Michael Niedermayer
623ecb2ec8 avcodec/wmalosslessdec: move channel check up
Fixes: out of array access
Fixes: 2nd part of 18429/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_WMALOSSLESS_fuzzer-6210814364614656

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Reviewed-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 891bcc4acc)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2020-07-03 12:10:23 +02:00
Michael Niedermayer
94a1280fb0 avcodec/cbs_h2645: Skip all 0 NAL units
Fixes: assertion failure
Fixes: 19286/clusterfuzz-testcase-minimized-ffmpeg_BSF_H264_REDUNDANT_PPS_fuzzer-5707990724509696

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 285138ef14)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2020-07-03 12:10:23 +02:00
Michael Niedermayer
7be077fcf8 avcodec/adpcm: Fix overflow in FFABS() IMA_EA_EACS
Fixes: negation of -2147483648 cannot be represented in type 'int'; cast to an unsigned type to negate this value to itself
Fixes: 19235/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_ADPCM_IMA_EA_EACS_fuzzer-5680878952382464

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 794352ae9d)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2020-07-03 12:10:23 +02:00
Michael Niedermayer
b36592ce07 avcodec/alac: Fix integer overflow in LPC coefficient adaption
Fixes: signed integer overflow: 267693597 * 10 cannot be represented in type 'int'
Fixes: 19237/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_ALAC_fuzzer-5755407700328448

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 6a865cec5e)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2020-07-03 12:10:23 +02:00
Michael Niedermayer
84048cf37f avcodec/g729postfilter: Optimize out overflowing multiplication from apply_tilt_comp()
Fixes: signed integer overflow: -1114392282 * 2 cannot be represented in type 'int'
Fixes: 19236/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_G729_fuzzer-5741678938030080

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit c0bd5fa43d)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2020-07-03 12:10:23 +02:00
Michael Niedermayer
fabd42043a avcodec/vc1dec: Check field_mode for sprites
Fixes: Out of array read
Fixes: 19263/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_VC1IMAGE_fuzzer-5389219325542400

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 32fb919836)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2020-07-03 12:10:23 +02:00
Michael Niedermayer
f2a0bde8f3 avcodec/vc1dec: Limit bits by the actual bitstream size
Fixes: Timeout (350 ->19sec)
Fixes: 19249/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_VC1IMAGE_fuzzer-6566896438870016

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit c56a52a82c)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2020-07-03 12:10:23 +02:00
Michael Niedermayer
f061fe9aa1 avcodec/vmdaudio: Check block_align more
Fixes: signed integer overflow: 2147483647 + 1 cannot be represented in type 'int'
Fixes: 19788/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_VMDAUDIO_fuzzer-5743379690553344

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 06f6857b54)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2020-07-03 12:10:23 +02:00
Andreas Rheinhardt
96f18b6427 avcodec/cinepakenc: Fix invalid shifts
Fixes: left shift of 1 by 31 places cannot be represented in type 'int'.
Affected the FATE-tests vsynth1-cinepak, vsynth2-cinepak and
vsynth_lena-cinepak. Also fixes ticket #8220.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit e3650dcfc9)
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
2020-07-02 00:27:48 +02:00
Andreas Rheinhardt
946d2893a9 avcodec/cbs_h2645: Fix potential out-of-bounds array access
The maximum allowed index for an array access is FF_ARRAY_ELEMS - 1; yet
the current code allowed FF_ARRAY_ELEMS. This wasn't dangerous in practice,
as parameter sets with invalid ids were already filtered out during
reading.

Found via PVS-Studio (see ticket #8156).

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit f3333c3c67)
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
2020-07-02 00:25:46 +02:00
Andreas Rheinhardt
5db4c91ef5 h264_redundant_pps: Fix memleak in case of errors
Now the fragment is uninitialized and the input packet freed in case of
errors.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@googlemail.com>
Signed-off-by: Mark Thompson <sw@jkqxz.net>
(cherry picked from commit 40b74abfca)
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
2020-07-02 00:20:34 +02:00
Andreas Rheinhardt
2b35185006 avcodec/cavsdsp: Fix undefined left shifts of negative numbers
Affected the ffmpeg-filter_colorkey FATE-test (but only if the C version
of idct8_add is used and not e.g. the x86 SSE2 version).

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>

Fixes: left shift of negative value -107
Fixes: 20398/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_CAVS_fuzzer-5725389278412800

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 0f0f2ab0c3)
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
2020-07-01 23:18:20 +02:00
Andreas Rheinhardt
d9e4783f06 avcodec/ra144enc: Fix invalid left shift of negative number
by replacing it with a multiplication. Said multiplication can't
overflow an int32_t because lpc_coefs is limited to 16 bit precision.

Fixes the FACE-test acodec-ra144 as well as part of #8217.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit e3fb9af6f1)
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
2020-07-01 23:04:12 +02:00
Andreas Rheinhardt
68e67bb187 avcodec/adxenc: Avoid undefined left shift of negative numbers
Replace "((a << shift) + b) >> shift" by "a + (b >> shift)". This avoids
a left shift which also happens to trigger undefined behaviour in case "a"
is negative. This affected the FATE-tests acodec-adpcm-adx and
acodec-adpcm-adx-trellis; it also fixes ticket #8008.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 59a9d65e0d)
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
2020-07-01 23:03:43 +02:00
Andreas Rheinhardt
d620f2eeb4 avcodec/adpcm: Fix undefined left shifts of negative numbers
Affected the adpcm-afc, adpcm-ea-1, adpcm-ea-2, adpcm-ea-maxis-xa,
adpcm-thp and ea-cdata FATE-tests. Also fixes ticket #8487.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 3ad8af51b7)
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
2020-07-01 23:03:23 +02:00
Andreas Rheinhardt
02368962f5 avcodec/proresenc_anatoliy: Fix invalid left shift of negative number
This fixes ticket #7997 as well as the vsynth*-prores_# FATE-tests
(where * ranges over { 1, 2, 3, _lena } and # over { , _int, _444,
_444_int }).

(Given that prev_dc is in the range -0xC000..0x3FFF, no overflow can
happen upon multiplication with 2.)

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 198081efb7)
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
2020-07-01 23:01:29 +02:00
Andreas Rheinhardt
02d9561eab avcodec/jpeg2000dwt: Fix undefined shifts of negative numbers
Affected the vsynth*-jpeg2000 and the vsynth*-jpeg2000-97 FATE tests
(where * ranges over { 1, 2, 3, _lena }) as well as ticket #7983.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 5cf593adcd)
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
2020-07-01 22:30:42 +02:00
Andreas Rheinhardt
ceed3603e1 avcodec/ituh263dec: Fix undefined left shift of negative number
Fixes ticket #8160.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 324487b596)
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
2020-07-01 22:30:16 +02:00
Andreas Rheinhardt
ab3b008dad avcodec/dnxhdenc: Fix undefined left shifts of negative numbers
Affected 61 FATE-tests: 60 vsynth tests and lavf-mxf_opatom.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit b7f156e8cb)
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
2020-07-01 22:29:36 +02:00
Andreas Rheinhardt
438773db51 avcodec/exr: Fix undefined left shifts of negative numbers
Affected the FATE-tests exr-rgb-scanline-pxr24-half-uint32-13x9 and
exr-rgb-scanline-pxr24-uint32.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 8b0f949906)
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
2020-07-01 22:28:15 +02:00
Andreas Rheinhardt
2b9ddde1d0 avcodec/pcm: Fix undefined shifts
Fixes the acodec-pcm-u16[lb]e FATE-tests.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit 69473bec6f)
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
2020-07-01 22:26:42 +02:00
Andreas Rheinhardt
3497fd2dd7 avcodec/wavpackenc: Fix undefined shifts
Fixes ticket #8161 and the acodec-wavpack FATE-test.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit 84974c6fb5)
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
2020-07-01 22:26:04 +02:00
Andreas Rheinhardt
9083b1054a avcodec/ac3enc: Fix invalid shift
Fixes the FATE-tests unknown_layout-ac3, ac3-fixed-encode, ac3-encode
and eac3-encode. It furthermore fixes the ac3-encoder bugs mentioned in
tickets #7994, #8144 and #8159.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 2f289ec914)
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
2020-07-01 22:25:21 +02:00
Andreas Rheinhardt
0284b72c8a avcodec/tdsc: Fix undefined shifts
Fixes the tdsc FATE-test.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 86bef10e7a)
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
2020-07-01 22:20:51 +02:00
Andreas Rheinhardt
9a3edef421 avcodec/ttaenc: Fix undefined shift
ttaenc contained (1 << unary) - 1 as an argument for a function
expecting an unsigned int. unary can be as big as 31 in this case.
The type of the shift and the whole expression is int, because 1 fits
into an integer, so that the behaviour is undefined if unary == 31
as the result of the shift can't be represented in an int §. Subtraction
by 1 (which makes the result of the whole expression representable in
an int) doesn't change that this is undefined (it usually leads to
signed integer overflow which is undefined, too).

The solution is simple: Make 1 unsigned to change the type of the
whole expression to unsigned int (as the function expects anyway).

Fixes ticket #8153.

§: This of course presupposes the common int range of -2^31..2^31-1

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 3ab488a540)
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
2020-07-01 22:18:52 +02:00
Andreas Rheinhardt
8d90a8cb37 avcodec/cbs_av1: Fix writing uvlc numbers >= INT_MAX
Fixes: assertion failure
Fixes: left shift of 1 by 31 places cannot be represented in type 'int'
Fixes: 23264/clusterfuzz-testcase-minimized-ffmpeg_BSF_AV1_METADATA_fuzzer-6308429248593920

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit 6f06c17a55)
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
2020-07-01 17:16:06 +02:00
Andreas Rheinhardt
0df1210631 avcodec/bitstream: Don't check for undefined behaviour after it happened
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit 5e196dac22)
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
2020-07-01 16:59:01 +02:00
Andreas Rheinhardt
dc84430172 libavcodec/libvpxenc: Don't free user-provided AVPacket
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit 26b4509690)
2020-05-23 20:34:51 +02:00
Andreas Rheinhardt
68f5905f3c libavcodec/libmp3lame: Don't free user-provided AVPacket
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit 7e6941e185)
2020-05-23 20:34:51 +02:00
Andreas Rheinhardt
8efc398218 avcodec/libopusenc: Don't free user-provided AVPacket
Reviewed-by: James Almer <jamrial@gmail.com>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit b803993b6d)
2020-05-23 20:34:51 +02:00
James Almer
6376b6b00b avcodec/cbs_h265: fix writing extension_data bits
We only care about the right most bit.

Signed-off-by: James Almer <jamrial@gmail.com>
(cherry picked from commit 38d1815cc6)
2020-05-03 18:50:22 -03:00
Fei Wang
c96a52b714 avcodec/cbs_av1: avoid reading trailing bits when obu type is OBU_TILE_LIST
Signed-off-by: Fei Wang <fei.w.wang@intel.com>
Signed-off-by: James Almer <jamrial@gmail.com>
(cherry picked from commit 1ea44178f5)
2020-01-06 12:14:02 -03:00
James Almer
3413eb6091 avcodec/av1_parser: skip frames with spatial_id > 0
This fixes marking keyframes in svc samples.

Signed-off-by: James Almer <jamrial@gmail.com>
(cherry picked from commit 5985ca0436)
2020-01-06 12:10:44 -03:00
Mark Thompson
07afe2e3ca cbs_h264: Fix missing inferred colour description fields
With video_signal_type_present_flag set but colour_description_present_flag
unset the colour fields would not have had their correct values inferred.

(cherry picked from commit f9b8503639)
2020-01-06 12:03:38 -03:00
James Almer
1ad3418519 avcodec/cbs_av1: keep separate reference frame state for reading and writing
In scearios where a Temporal Unit is written right after reading it using the same
CBS context (av1_metadata, av1_frame_merge, etc), the reference frame state used
by the writer must not be the state that's the result of the reader having already
parsed the current frame in question.

This fixes writing Switch frames, and frames using short ref signaling.

Signed-off-by: James Almer <jamrial@gmail.com>
(cherry picked from commit 4e2bef6a82)
2020-01-06 11:56:04 -03:00