Commit graph

37035 commits

Author SHA1 Message Date
Michael Niedermayer
cbd622be99 avcodec/h264_ps: Check delta scale for validity
Fixes: signed integer overflow: 5 + 2147483646 cannot be represented in type 'int'
Fixes: 634/clusterfuzz-testcase-5285420445204480

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/targets/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2017-02-21 15:31:06 +01:00
Michael Niedermayer
28dc6e7291 avcodec/simple_idct: Fix runtime error: left shift of negative value -6395
Fixes: 633/clusterfuzz-testcase-4553133554401280

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/targets/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2017-02-21 15:31:06 +01:00
Michael Niedermayer
956472a323 avcodec/rv40: Fix runtime error: left shift of negative value
Fixes: 630/clusterfuzz-testcase-6608718928019456

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/targets/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2017-02-21 15:31:06 +01:00
Paul B Mahol
f4777d1b89 avcodec/qdrw: add support for decoding rgb555
Signed-off-by: Paul B Mahol <onemda@gmail.com>
2017-02-21 12:17:21 +01:00
Michael Niedermayer
e8a3498f24 avcodec/dca_xll: Fix runtime error: signed integer overflow: -1073741824 * 32768 cannot be represented in type 'int'
Fixes: 629/clusterfuzz-testcase-6697457381539840

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/targets/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2017-02-21 00:40:20 +01:00
Michael Niedermayer
0a65dae9d0 avcodec/flacdec: reduce limit for golomb so that the max value does not overflow
Fixes: runtime error: left shift of 32 by 26 places cannot be represented in type 'int'

Fixes: 628/clusterfuzz-testcase-6187747641393152
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/targets/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2017-02-21 00:40:20 +01:00
Michael Niedermayer
e04108dfa6 avcodec/dca_xll: signed integer overflow: 255251 * 32768 cannot be represented in type 'int'
Fixes: 627/clusterfuzz-testcase-5020897033322496
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/targets/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2017-02-21 00:40:20 +01:00
Michael Niedermayer
4614bf2caf Factorize CHECK/SUINT code
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2017-02-21 00:40:20 +01:00
Steinar H. Gunderson
e3c14eaa54 speedhq: fix decoding artifacts
The quantization table is stored in the natural order, but when we
access it, we use an index that's in zigzag order, causing us to read
the wrong value. This causes artifacts, especially in areas with
horizontal or vertical edges. The artifacts look a lot like the
DCT ringing artifacts you'd expect to see from a low-bitrate file,
but when comparing to NewTek's own decoder, it's obvious they're not
supposed to be there.

Fix by simply storing the scaled quantization table in zigzag order.
Performance is unchanged.

Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2017-02-21 00:40:20 +01:00
Timo Rothenpieler
f747e1533a avcodec/nvenc: allow forcing keyframes by default 2017-02-20 22:38:39 +01:00
Michael Niedermayer
15ccaa344c avcodec/mpegaudiodec: Eliminate many undefined operations
Fixes: 625/clusterfuzz-testcase-4574924406521856
Fixes: 626/clusterfuzz-testcase-4738718621499392

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/targets/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2017-02-20 17:14:54 +01:00
Michael Niedermayer
01d196a67d avcodec/pictordec: Do not read more than nb_planes
Fixes undefined behavior
Fixes: 622/clusterfuzz-testcase-5745722022428672

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/targets/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2017-02-20 17:14:54 +01:00
Michael Niedermayer
b7d9b4a1f1 avcodec/ituh263dec: Check cbpy in ff_h263_decode_mb()
Fixes: 618/clusterfuzz-testcase-6594990333493248

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/targets/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2017-02-20 00:49:44 +01:00
Michael Niedermayer
c11d3634b0 avcodec/srtdec: Fix signed integer overflow: 1811992524 * 384 cannot be represented in type 'int'
Fixes: 617/clusterfuzz-testcase-6413875723370496

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/targets/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2017-02-19 23:38:46 +01:00
Michael Niedermayer
59e5b05ef6 avcodec/h264_ps: Fix runtime error: signed integer overflow: -1094995528 * 2 cannot be represented in type 'int'
Fixes: 615/clusterfuzz-testcase-5488002644049920
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/targets/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2017-02-19 22:55:04 +01:00
Michael Niedermayer
258763ad0e avcodec/h264_cabac: runtime error: signed integer overflow: 2147483647 + 14 cannot be represented in type 'int'
Fixes: 614/clusterfuzz-testcase-4931860079575040

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/targets/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2017-02-19 22:41:40 +01:00
Michael Niedermayer
aa2b75263e avcodec/mpeg4videodec: Fix runtime error: shift exponent -2 is negative
Fixes: 612/clusterfuzz-testcase-4707817137111040

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/targets/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2017-02-19 21:49:03 +01:00
Michael Niedermayer
c91bdd4524 avcodec/mjpegdec: Fix runtime error: left shift of negative value -507
Fixes: 611/clusterfuzz-testcase-5613455820193792

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/targets/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2017-02-19 21:49:03 +01:00
Michael Niedermayer
067485b673 avcodec/eac3dec: Fix runtime error: left shift of negative value
Fixes: 610/clusterfuzz-testcase-4831030085156864

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/targets/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2017-02-19 21:49:03 +01:00
Rostislav Pehlivanov
67fa02ed79 opus_pvq: fix PVQ search for K < 5 and low Ns
If the PVQ search picked a place to increment/decrement on the y[]
vector which had no pulse then it would cause a desync since it would
change the sum in the wrong direction. Fix this by not considering
places without pulses as viable.

This makes the PVQ search slightly worse at K < 5 which isn't all that
common. Still, this is a workaround to prevent making broken files until
I can think of a better way of fixing it.

Also add an assertion, which can be removed or moved to assert1/2 once
the PVQ search is stable.

Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>
2017-02-19 19:13:14 +00:00
Michael Niedermayer
4279613a26 avcodec/pngdec: Check bit depth for validity
Fixes: runtime error: shift exponent 132 is too large for 32-bit type 'int'
Fixes: 609/clusterfuzz-testcase-4825202619842560

See 11.2.2 IHDR Image header

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/targets/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2017-02-19 19:48:57 +01:00
Michael Niedermayer
706757d26d avcodec/mpeg12dec: Fix runtime error: left shift of negative value
Fixes: 608/clusterfuzz-testcase-603978286392934

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/targets/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2017-02-19 19:48:57 +01:00
James Almer
b921b3b42b avcodec/huffyuvencdsp: use an actual unsigned long constant
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: James Almer <jamrial@gmail.com>
2017-02-19 12:27:12 -03:00
James Almer
21d25da180 avcodec/huffyuvdsp: use an actual unsigned long constant
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: James Almer <jamrial@gmail.com>
2017-02-19 12:27:06 -03:00
Michael Niedermayer
12eebb845a avcodec/wavpacl: Fix runtime error: left shift of negative value -1
Fixes: 607/clusterfuzz-testcase-5108792465293312

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/targets/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2017-02-19 15:58:30 +01:00
Michael Niedermayer
3e1028c625 avcodec/flac: Fix several integer overflows
Fixes: 686513-media
Found-by: Matt Wolenetz <wolenetz@google.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2017-02-19 15:58:30 +01:00
Michael Niedermayer
e8d4eacc07 Factor CHECKED out of DEBUG so it can be set seperatly
Suggested-by: Matt Wolenetz <wolenetz@google.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2017-02-19 15:39:36 +01:00
Michael Niedermayer
2ac6eedac5 avcodec/pngdec: Store metadata directly into AVFrame
Fixes memleak
Fixes: 500/clusterfuzz-testcase-6315221727576064

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/targets/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2017-02-19 15:39:36 +01:00
Michael Niedermayer
3f28caf720 avcodec/mpeg12dec: Provide debug level log on skiped P/B frames
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2017-02-19 02:07:12 +01:00
James Darnley
5336887867 avcodec/h264: sse2, avx h luma mbaff deblock/loop filter
x86-64 only

Yorkfield:
- sse2: ~2.17x (434 vs. 200 cycles)

Nehalem:
- sse2: ~2.94x (409 vs. 139 cycles)

Skylake:
- sse2: ~3.10x (370 vs. 119 cycles)
- avx:  ~3.29x (370 vs. 112 cycles)
2017-02-18 20:26:52 +01:00
James Darnley
e18bc2114f avcodec/h264: add named parameters to x86 function 2017-02-18 20:26:50 +01:00
James Darnley
9d815b7424 avcodec/x86: deduplicate PASS8ROWS macro 2017-02-18 20:26:49 +01:00
Miroslav Slugeň
2a2f6b2887 avcodec/cuvid: add drop_second_field as input option
Signed-off-by: Timo Rothenpieler <timo@rothenpieler.org>
2017-02-18 12:44:08 +01:00
Miroslav Slugeň
4cb8872eb7 avcodec/cuvid: don't overwrite deinterlace at progressive input
If there is progressive input it will disable deinterlacing in cuvid for
all future frames even those interlaced.

Signed-off-by: Timo Rothenpieler <timo@rothenpieler.org>
2017-02-18 12:44:08 +01:00
Rostislav Pehlivanov
b71f62a9c8 opusenc: initialize the emphasis coefficients on init
Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>
2017-02-18 01:13:13 +00:00
Michael Niedermayer
04e611474b avcodec/opus: Check count of ambisonic channels
https://tools.ietf.org/html/draft-ietf-codec-ambisonics-01#section-3.1
specifies the maximum as 227

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2017-02-17 11:18:30 +01:00
Felicia
fcf3e06fe4 libopus: decode ambisonics with non-diegetic sources
Channel mapping 2 additionally supports a non-diegetic stereo track
appended to the end of a full-order ambisonics signal, such that the
total channel count is either
  (n + 1) ^ 2, or
  (n + 1) ^ 2 + 2
where n is the ambisonics order

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2017-02-17 11:05:15 +01:00
Paul B Mahol
1a71df9bac avcodec/fmvc: fix decoding of odd size videos
Signed-off-by: Paul B Mahol <onemda@gmail.com>
2017-02-17 10:48:04 +01:00
Carl Eugen Hoyos
1d54be2153 lavc/avpacket: Initialize a variable in error path.
Fixes ticket #6153.

Tested-by: Tyson Smith
2017-02-17 10:31:29 +01:00
Andreas Cadhalpun
9ccc6cecd2 wmaprodec: fix leaking fdsp on init failure
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
2017-02-17 01:45:19 +01:00
Andreas Cadhalpun
783b350b2e mpegaudiodec_template: fix leaking fdsp for mp3on4float
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
2017-02-17 01:45:19 +01:00
Mark Thompson
a1e83a2f90 vaapi_vp8: Use VP8_MAX_QUANT instead of magic number 2017-02-17 00:12:40 +00:00
Michael Niedermayer
1fa8884aeb avcodec/opusenc: Add () protecting macro arguments
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2017-02-16 13:24:11 +01:00
Michael Niedermayer
8365c1a15c avcodec/error_resilience: Fix "assignment from incompatible pointer type" warning
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2017-02-16 13:20:27 +01:00
Pavel Koshevoy
e3f1b99334 lavc/vda_h264_dec.c Fix NULL pointer dereference
ps.sps_list entries may be NULL, so check before dereferencing

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2017-02-16 13:20:27 +01:00
Rostislav Pehlivanov
3fc86f0d69 opusenc: fix coarse energy quantization with 2 bits left
Fixes CID1400584

Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>
2017-02-15 22:51:37 +00:00
Rostislav Pehlivanov
1b90e2414d opus_pvq: fix uninitialized variable usage
Fixes CID1400586

Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>
2017-02-15 21:11:41 +00:00
Paul B Mahol
ee4aa388b2 adpcm: fix clipping for yamaha
According to specification max value allowed is 0x6000.
Fixes #5862.

Signed-off-by: Paul B Mahol <onemda@gmail.com>
2017-02-15 12:46:01 +01:00
Michael Niedermayer
6a37abc59a avcodec/h264_sei: Check actual presence of SEI picture timing instead of implying it
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2017-02-15 00:19:50 +01:00
Michael Niedermayer
db3507a670 avcodec/hevc_parser: export framerate, remove one dependency on the decoder
Fixes Ticket6090

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2017-02-14 17:00:30 +01:00