Commit graph

47975 commits

Author SHA1 Message Date
Michael Niedermayer
63aa0cfd0e
avcodec/bonk: decode multiple passes in intlist_read() at once
This makes the worst case much faster

Fixes: Timeout
Fixes: 51363/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_BONK_fuzzer-5660734784143360
Fixes: 57957/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_BONK_fuzzer-5874095467397120

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 957106a24d)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2023-10-28 19:04:44 +02:00
Michael Niedermayer
1d7dd08199
avcodec/vorbisdec: Check codebook float values to be finite
Fixes: Timeout
Fixes: 55116/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_VORBIS_fuzzer-4572159970508800

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit cadd7e7a75)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2023-10-28 19:04:44 +02:00
Michael Niedermayer
6500c15977
avcodec/g2meet: Replace fake allocation avoidance for framebuf
framebuf is only allocated when the new width/height are larger than the old
but nothing sets the old so its always allocated.
Use av_fast_mallocz() instead.

Fixes: Timeout
Fixes: 55094/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_G2M_fuzzer-5116909932904448

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 38adbc6eeb)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2023-10-28 19:04:44 +02:00
Michael Niedermayer
19db9ffb3b
avcodec/lcldec: More space for rgb24
Fixes: Ticket 10239
Fixes: zlib_306_306_rgb24.av

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit e2c3aa8e2b)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2023-10-28 19:04:43 +02:00
Michael Niedermayer
735f6be311
avcodec/lcldec: Support 4:1:1 and 4:2:2 with odd width
Fixes: Ticket10240
Fixes: zlib_306_306_yuv422.avi
Fixes: zlib_306_306_yuv411.avi

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 0cf1ac905d)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2023-10-28 19:04:43 +02:00
Michael Niedermayer
a06bf3dae8
libavcodec/lcldec: width and height should not be unsigned
Computations like col < width - 3 will not work with unsigned width=1

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 3eb4e28c26)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2023-10-28 19:04:42 +02:00
Michael Niedermayer
0291b86a94
avcodec/escape124: Check that blocks are allocated before use
Fixes: NULL pointer dereference
Fixes: 57819/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_ESCAPE124_fuzzer-5077280228769792

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 5366ae12b9)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2023-10-28 19:04:42 +02:00
Michael Niedermayer
d84e889785
avcodec/rka: Fix signed integer overflow in decode_filter()
Fixes: signed integer overflow: -631553 * 32768 cannot be represented in type 'int'
Fixes: 57814/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_RKA_fuzzer-4614661233573888

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 0ecda4fbce)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2023-10-28 19:04:41 +02:00
Michael Niedermayer
a85d0bc84b
avcodec/huffyuvdec: Fix undefined behavior with shift
Fixes: left shift of negative value -1
Fixes: 57554/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_FFVHUFF_fuzzer-4853603839115264

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 27e7857bd1)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2023-10-28 19:04:41 +02:00
Michael Niedermayer
67919ba682
avcodec/j2kenc: Replace RGB24 special case by generic test
This fixes RGB48 with libavcodec as decoder

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit ad4d647591)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2023-10-28 19:04:41 +02:00
Michael Niedermayer
1ed34709ff
avcodec/j2kenc: Replace BGR48 / GRAY16 test by test for number of bits
BGR48 is not supported and this was probably meant to be RGB48 so this fixes
RGB48 a bit

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 7fb70d27a2)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2023-10-28 19:04:40 +02:00
Michael Niedermayer
4cec33869e
avcodec/j2kenc: simplify pixel format setup
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 644d15716d)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2023-10-28 19:04:40 +02:00
Michael Niedermayer
9a086ad5b1
avcodec/j2kenc: Fix funky bpno errors on decoding
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 816676085e)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2023-10-28 19:04:40 +02:00
Michael Niedermayer
407e6cde75
avcodec/j2kenc: remove misleading pred value
This field is only checked for being 0 or not and not zero means 5/3

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 0adb375377)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2023-10-28 19:04:39 +02:00
Michael Niedermayer
fa35ce082a
avcodec/j2kenc: fix 5/3 DWT identifer
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit f6955b6df4)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2023-10-28 19:04:39 +02:00
Michael Niedermayer
efab690c16
avcodec/vp3: Check width to avoid assertion failure
Fixes: Assertion failure on x86-32
       av_assert2(block_w * sizeof(pixel) <= FFABS(buf_linesize)); in ff_emulated_edge_mc()
Fixes: 39641/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_THEORA_fuzzer-5925660741206016

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit dab1cd2dc0)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2023-10-28 19:04:39 +02:00
Michael Niedermayer
14da53d4c6
avcodec/g729postfilter: Limit shift in long term filter
Fixes: shift exponent 34 is too large for 32-bit type 'int'
Fixes: 57389/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_ACELP_KELVIN_fuzzer-6229522659016704

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 6d1d8609ac)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2023-10-28 19:04:38 +02:00
Michael Niedermayer
b89cda3269
avcodec/wavarc: Fix several integer overflows
Fixes: signed integer overflow: -532410125 + -1759642300 cannot be represented in type 'int'
Fixes: 57045/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_WAVARC_fuzzer-637023665297817

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 1942dbb8ca)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2023-10-28 19:04:38 +02:00
Michael Niedermayer
deeb0b6e94
avcodec/tests/snowenc: Fix 2nd test
(cherry picked from commit 163013c724)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2023-10-28 19:04:38 +02:00
Michael Niedermayer
89c0a651ad
avcodec/tests/snowenc: return a failure if DWT/IDWT mismatches
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 771c266c0b)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2023-10-28 19:04:37 +02:00
Michael Niedermayer
ea6ec6b507
avcodec/snowenc: Fix visual weight calculation
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 5b5fcadea0)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2023-10-28 19:04:37 +02:00
Michael Niedermayer
c2a484cf5b
avcodec/tests/snowenc: unbreak DWT tests
the IDWT data type mismatched current code

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 8b3351bbea)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2023-10-28 19:04:37 +02:00
Michael Niedermayer
1957bd1110
avcodec/mpeg12dec: Check input size
Fixes: Timeout
Fixes: 53599/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_IPU_fuzzer-4950102511058944

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 7c130d6911)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2023-10-28 19:04:36 +02:00
Michael Niedermayer
438ea0eaf3
avcodec/escape124: Fix some return codes
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 98df605f7a)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2023-10-28 19:04:36 +02:00
Michael Niedermayer
f081f94601
avcodec/escape124: fix signdness of end of input check
Fixes: Timeout
Fixes: 56561/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_ESCAPE124_fuzzer-5560363635834880

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 87ad0a5dd7)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2023-10-28 19:04:36 +02:00
Timo Rothenpieler
c5039e158d avcodec/nvdec_hevc: fail to initialize on unsupported profiles 2023-10-12 21:03:03 +02:00
Marton Balint
3d5edb89e7 avcodec/on2avc: use correct fft sizes
Also rename the contexts and the functions so their names will reflect their
intended size.

With the earlier patch this fixes the audio corruption regression caused by
6ba0aa1770.

Fixes ticket #10029.

Signed-off-by: Marton Balint <cus@passwd.hu>
(cherry picked from commit 2eb7df1cbd)
2023-08-08 19:41:20 +02:00
Marton Balint
d2efb80027 avcodec/on2avc: use the matching AVTX context for the 512 sized iMDCT
Improves the audio corruption regression caused by
6ba0aa1770 reported in ticket #10029.

There is still however a noticable audio glitch, so the FFT conversion to AVTX
probably also needs some modifications.

Signed-off-by: Marton Balint <cus@passwd.hu>
(cherry picked from commit f296c76590)
2023-08-08 19:41:02 +02:00
Timo Rothenpieler
1ebb0e43f9 avcodec/nvenc: stop using deprecated rc modes with SDK 12.1 2023-07-23 12:58:13 +02:00
Rémi Denis-Courmont
cc703cf607 avcodec/x86/mathops: clip constants used with shift instructions within inline assembly
Fixes assembling with binutil as >= 2.41

Signed-off-by: James Almer <jamrial@gmail.com>
(cherry picked from commit effadce6c7)
2023-07-20 16:53:17 -03:00
James Almer
3f345ebf21 avcodec/aac_ac3_parser: add preprocessor checks for codec specific code
Should fix ticket #10421.

Signed-off-by: James Almer <jamrial@gmail.com>
(cherry picked from commit 35af1695a9)
2023-06-24 14:43:46 -03:00
Timo Rothenpieler
868af7d7a4 avcodec/nvenc: handle frame durations and AV_CODEC_CAP_ENCODER_REORDERED_OPAQUE 2023-06-16 22:45:24 +02:00
Timo Rothenpieler
d341895a08 Revert "lavc/nvenc: handle frame durations and AV_CODEC_CAP_ENCODER_REORDERED_OPAQUE"
The implementation is flawed in that the frame opaque data is not in
fact correctly reordered along with the packets, but is being output in
packet input order, just like the dts are.

This reverts commit 3553809703.
2023-06-16 22:43:32 +02:00
Timo Rothenpieler
cb3453eb25 Revert "avcodec/nvenc: fix b-frame DTS behavior with fractional framerates"
This reverts commit 9a245bdf5d.

This commit basically broke all samples with fractional framerates,
rather than fixing them.
I at this point do not understand the original issue anymore, and I'm
not sure how this slipped my initial testing.
All my test samples must have happened to have a simple timebase.

The actual dts values pretty much always are just a simple chain of
1,2,3,4,5,... Or maybe slightly bigger steps. Each increase by one means
an advance in time by one unit of the timebase.
So a fractional framerate/timebase is already not an issue.

So with this patch applied, the calculation might end up substracting
huge values (1001 is a common one) from the dts, which would be an
offset of that many frames, not of that many fractions of a second.
This broke at least muxing into mp4, if the sample happened to have a
fractional framerate.

I do not thing the original issue this patch tried to fix existed in the
first place, so it can be reverted without further consequences.
2023-06-16 22:35:24 +02:00
Hendrik Leppkes
549430e14d
avcodec/vdpau_mpeg4: fix order of quant matrix coefficients
The matrix coefficients are stored permutated for the IDCT,
rather then in plain raster order, and need to be un-permutated
for the hardware.
2023-05-15 18:41:30 +02:00
Hendrik Leppkes
bfa8a4a44b
avcodec/vdpau_mpeg12: fix order of quant matrix coefficients
The matrix coefficients are stored permutated for the IDCT,
rather then in plain raster order, and need to be un-permutated
for the hardware.
2023-05-15 18:41:30 +02:00
Hendrik Leppkes
d7c0991cfc
avcodec/nvdec_mpeg4: fix order of quant matrix coefficients
The matrix coefficients are stored permutated for the IDCT,
rather then in plain raster order, and need to be un-permutated
for the hardware.
2023-05-15 18:41:30 +02:00
Hendrik Leppkes
8ea302832f
avcodec/nvdec_mpeg2: fix order of quant matrix coefficients
The matrix coefficients are stored permutated for the IDCT,
rather then in plain raster order, and need to be un-permutated
for the hardware.
2023-05-15 18:41:30 +02:00
James Almer
2c11164707 avcodec/mp_cmp: reject invalid comparison function values
Fixes tickets #10306 and #10318.

Signed-off-by: James Almer <jamrial@gmail.com>
(cherry picked from commit 7c6e26a184)
2023-04-16 11:38:26 -03:00
James Almer
5606d27006 avcodec/aacpsy: clip global_quality within the psy_vbr_map array boundaries
Fixes ticket #10317.

Signed-off-by: James Almer <jamrial@gmail.com>
(cherry picked from commit 5cda6b94f4)
2023-04-16 11:38:26 -03:00
Kyle Manning
a6dc92968a avcodec/nvenc: fix b-frame DTS behavior with fractional framerates
When using fractional framerates (or any fraction with a numerator != 1),
DTS values for packets would be calculated incorrectly.

Signed-off-by: Kyle Manning <tt2468@irltoolkit.com>
Signed-off-by: Timo Rothenpieler <timo@rothenpieler.org>
2023-04-03 12:03:56 +02:00
James Almer
3980415627 avcodec/vorbisdec: export skip_samples instead of dropping frames
pts may not be set on input packets, which could result in the entire stream
being discarded.
This reverts commit 8fc2dedfe6, reintroducing the
behavior it replaced but now allowing the caller to manually drop the preroll
samples by looking at the skip_samples side data at the start while ignoring it
on seek, by setting the skip_manual avctx flag.

Fixes ticket #10251.

Suggested-by: Anton Khirnov <anton@khirnov.net>
Signed-off-by: James Almer <jamrial@gmail.com>
(cherry picked from commit cbcc817353)
2023-03-22 12:51:17 -03:00
Anton Khirnov
adb4688bfb lavc/decode: stop mangling last_pkt_props->opaque
It is currently abused to store packet size, which breaks
AV_CODEC_FLAG_COPY_OPAQUE.

Use stream_index instead, which is unused in libavcodec and has the
same type as size.

Found-by: Martin Storsjö
(cherry picked from commit a3bf63c26e)
Signed-off-by: Anton Khirnov <anton@khirnov.net>
2023-03-11 11:41:45 +01:00
Timo Rothenpieler
916bdc0f19 avcodec/nvenc: avoid failing b_ref_mode check when unset 2023-03-10 13:17:52 +01:00
Zhao Zhili
46970dd155 avcodec/aacps_tablegen: fix build error after avutil bump
Fix tickets #10225

DECLARE_ALIGNED has been moved to mem_internal.h.

Signed-off-by: Zhao Zhili <zhilizhao@tencent.com>
Reviewed-by: Anton Khirnov <anton@khirnov.net>
(cherry picked from commit 814178f926)
2023-03-02 14:43:24 -03:00
Zhao Zhili
0981053c29 avcodec/nvenc: fix potential NULL pointer dereference
Signed-off-by: Timo Rothenpieler <timo@rothenpieler.org>
2023-02-28 19:20:02 +01:00
Michael Niedermayer
ff5cce2eaf
avcodec/motionpixels: Mask pixels to valid values
Fixes: out of array access
Fixes: 48567/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_MOTIONPIXELS_fuzzer-6724203352555520

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit ac6eec1fc2)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2023-02-25 22:50:36 +01:00
Michael Niedermayer
1eb1acb62b
avcodec/xpmdec: Check size before allocation to avoid truncation
Fixes:OOM
Fixes:out of array access (no testcase)
Fixes: 48567/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_XPM_fuzzer-6573323838685184

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 95f0f84dae)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2023-02-25 22:50:10 +01:00
Michael Niedermayer
718d6c71ef
avcodec/bink: Avoid undefined out of array end pointers in binkb_decode_plane()
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit ea9deafd3b)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2023-02-25 22:49:27 +01:00
Michael Niedermayer
8ab01c8b0d
avcodec/bink: Fix off by 1 error in ref end
Fixes: out of array access
Fixes: 48567/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_BINK_fuzzer-6657932926517248

Alterantivly to this it is possibly to allocate a bigger array

Note: oss-fuzz assigned this issue to a unrelated theora bug so the bug number matches that

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 49487045dd)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2023-02-25 22:48:29 +01:00