Commit graph

32508 commits

Author SHA1 Message Date
Michael Niedermayer
3300005351 avcodec/jpeg2000dec: Check for duplicate SIZ marker
Fixes: 0231a17345734228011c6f35a64e4594/asan_heap-oob_1d92a72_3218_1213809a9e3affec77e4c191fdfdc0a9.mov

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 44a7f17d0b)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-11-14 01:11:00 +01:00
Andreas Cadhalpun
132bd4bc4f apng: use correct size for output buffer
The buffer needs s->bpp bytes, at maximum currently 10.
Assert that s->bpp is not larger.

This fixes a stack buffer overflow.

Reviewed-by: wm4 <nfxjfg@googlemail.com>
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 3e8e1a660e)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-11-12 02:55:47 +01:00
Michael Niedermayer
d332aa6ec6 avcodec/hevc_ps: Check chroma_format_idc
Fixes out of array access
Fixes: 24d05e8b84676799c735c9e27d97895e/asan_heap-oob_1b70f6a_2955_7c3652a7f370f9f3ef40642bc2c99bb2.bit

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 93f30f825c)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-11-12 02:55:47 +01:00
Michael Niedermayer
169a02da1e avcodec/jpeg2000dec: Clip all tile coordinates
Fixes out of array access
Fixes: b877a6b788a25c70e8b1d014f8628549/asan_heap-oob_1da2c3f_2324_5a1b329b0b3c4bb6b1d775660ac56717.r3d

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 43492ff3ab)

Conflicts:

	libavcodec/jpeg2000dec.c
2015-11-12 02:55:47 +01:00
Michael Niedermayer
af3a608e7e avcodec/microdvddec: Check for string end in 'P' case
Fixes out of array read
Fixes: a9502b60f4cecc19475382aee255f73c/asan_heap-oob_1e87fba_2548_a8ad47f6dde36644fe9cdc444d4632d0.sub

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit c719cd6cf7)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-11-12 02:55:47 +01:00
Michael Niedermayer
327dd70690 avcodec/dirac_parser: Fix undefined memcpy() use
Fixes: 9d375e415486edd1a0c826f2307d89a4/asan_generic_4a5159_1577_faa333e83dacdd9e4dd322380aeed537.iss

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit daefd8ab2f)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-11-12 02:55:47 +01:00
Michael Niedermayer
b0b1b208f5 avcodec/mpeg12dec: Do not call show_bits() with invalid bits
Fixes assertion failure
Fixes: 63e50545709a6440d3d59f6426d58db9/signal_sigabrt_7ffff6ae7cc9_8189_3272a3010fd98ddf947c662bbde1ac13.ts

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 973c3dba27)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-11-12 02:55:47 +01:00
Michael Niedermayer
e98f3ba221 avcodec/ffv1dec: Check for 0 quant tables
Fixes assertion failure
Fixes: 07ec1fc3c1cbf2d3edcd7d9b52ca156c/asan_heap-oob_13624c5_491_ecd4720a03e697ba750b235690656c8f.avi

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 5745cf799a)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-11-12 02:55:47 +01:00
Michael Niedermayer
32808460cc avcodec/mjpegdec: Reinitialize IDCT on BPP changes
Fixes misaligned access
Fixes: dc9262a469f6f315f74c087a7b3a7f35/signal_sigsegv_2e95bcd_9_9c0f9f4a9ba82aa9b3ab2b91ce4d5277.jpg

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit cc35f6f476)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-11-12 02:55:47 +01:00
Michael Niedermayer
edf06a0562 avcodec/mjpegdec: Check index in ljpeg_decode_yuv_scan() before using it
Fixes: 04715144ba237443010554be0d05343f/asan_heap-oob_1eafc76_1737_c685b48041a563461839e4e7ab97abb8.jpg
Fixes out of array access

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit d24888ef19)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-11-12 02:55:47 +01:00
Michael Niedermayer
ab6f4bc064 avcodec/h264_slice: Disable slice threads if there are multiple access units in a packet
Fixes null pointer dereference
Fixes part of Ticket4977

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 9368d2da3d33cac845f2fdf663df500b53625c5e)

Conflicts:

	libavcodec/h264_slice.c
2015-11-12 02:55:46 +01:00
Kieran Kunhya
c3ce508135 opusdec: Don't run vector_fmul_scalar on zero length arrays
Fixes crashes on fuzzed files
Fixes Ticket4969 part2

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit b3e5f15b95)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-11-12 02:55:46 +01:00
Michael Niedermayer
c6cb24ffbf avcodec/ffv1: Initialize vlc_state on allocation
This ensures that they are always set to valid values
Fixes Ticket4939

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit a878dfa4f5)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-11-12 02:55:46 +01:00
Michael Niedermayer
06c87da410 avcodec/ffv1dec: update progress in case of broken pointer chains
Fixes deadlock
Fixes Ticket4932

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 5063a18f56)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-11-12 02:55:46 +01:00
Michael Niedermayer
f7ef265387 avcodec/ffv1dec: Clear slice coordinates if they are invalid or slice header decoding fails for other reasons
Fixes Ticket4931

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 4c2d4e8700)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-11-12 02:55:46 +01:00
Michael Niedermayer
f43dac169e avcodec/x86/sbrdsp: Fix using uninitialized upper 32bit of noise
Fixes crash
Fixes: flicker-1.scout3d21443372922.28.m4a

Found-by: Dale Curtis <dalecurtis@google.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 1b82b934a1)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-11-12 02:55:46 +01:00
Michael Niedermayer
f10ba15d73 avcodec/ffv1dec: Fix off by 1 error in quant_table_count check
Fixes: invalid_read.nut
Found-by: Paul B Mahol <onemda@gmail.com>

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 2d221d9e06)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-11-12 02:55:46 +01:00
Michael Niedermayer
54661cf0b2 avcodec/ffv1dec: Explicitly check read_quant_table() return value
Forwards the error code, avoids potential integer overflow

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 10bbf6cf62)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-11-12 02:55:46 +01:00
Michael Niedermayer
50f62fef19 avcodec/rangecoder: Check e
Fixes hang.nut

Found-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit b2955b6c5a)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-11-12 02:55:46 +01:00
Michael Niedermayer
04ad22d77a avcodec/mjpegdec: Fix decoding RGBA RCT LJPEG
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 055e56e9f7)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-11-12 02:55:46 +01:00
Michael Niedermayer
484eb2b0e3 avcodec/g2meet: Also clear tile dimensions on header_fail
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit fb04666995)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-11-12 02:55:46 +01:00
Michael Niedermayer
746a65640f avcodec/g2meet: Fix potential overflow in tile dimensions check
Fixes CID1322351

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 71ec8e1ed6)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-11-12 02:55:46 +01:00
Michael Niedermayer
50e4d38ce4 avcodec/svq1dec: Check init_get_bits8() for failure
Fixes: CID1322313

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit a51d4246d8)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-11-12 02:55:46 +01:00
Michael Niedermayer
1e7ff902f2 avcodec/tta: Check init_get_bits8() for failure
Fixes: CID1322319

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit f1593e4ca5)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-11-12 02:55:46 +01:00
Michael Niedermayer
dceec28eac avcodec/vp3: Check init_get_bits8() for failure
Fixes CID1322316

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit cbd3cd8eb2)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-11-12 02:55:46 +01:00
Michael Niedermayer
19a29c92db avcodec/truemotion1: Check for even width
Fixes out of array access
Fixes: 87196d8bbc633629fc9dd851fce73e70/asan_heap-oob_26f6853_862_cov_585961513_sonic3dblast_intro-partial.avi

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 63fb5a6aef)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-11-12 02:55:46 +01:00
Michael Niedermayer
ffcd26d25b avcodec/mpeg12dec: Set dimensions in mpeg1_decode_sequence() only in absence of errors
Fixes assertion failure
Fixes: 56dcafde14a8397161bb61a16c511179/signal_sigabrt_7ffff6ac8cc9_686_cov_1897408623_microsoft_new_way_to_shove_mpeg2_in_asf.dvr_ms

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit b54e03c9dc)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-11-12 02:55:46 +01:00
Michael Niedermayer
42fd94b514 avcodec/libopusenc: Fix infinite loop on flushing after 0 input
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 6701c92fa4)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-11-12 02:55:46 +01:00
Michael Niedermayer
b87fc3526f avcodec/flashsvenc: Correct max dimension in error message
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit b1f59bb660)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-11-12 02:55:45 +01:00
Michael Niedermayer
d165ec179e avcodec/svq1enc: Check dimensions
Fixes assertion failure

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 88fe45e0fe)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-11-12 02:55:45 +01:00
Michael Niedermayer
94541f5f76 avcodec/dcaenc: clear bitstream end
This avoids leaving uninitialized bits in the output

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit e322b7061f)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-11-12 02:55:45 +01:00
Emanuel Czirai
5e71f4aa6a libavcodec/aacdec_template: Use init_get_bits8() in aac_decode_frame()
related to ticket4749

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 7ab1c57a64)

Conflicts:

	libavcodec/aacdec_template.c
2015-11-12 02:55:45 +01:00
Ronald S. Bultje
131f2c366e videodsp: don't overread edges in vfix3 emu_edge.
Fixes trac ticket 3226. Also see Andreas' analysis in
https://bugs.debian.org/801745, which was very helpful.
(cherry picked from commit 52f84d82bd)
2015-10-25 01:09:12 +02:00
Michael Niedermayer
8f00ff5ae0 avcodec/h264_mp4toannexb_bsf: Reorder operations in nal_size check
Fixes Ticket4778

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 2bb54b82b5)
2015-10-09 21:36:11 +02:00
Hendrik Leppkes
98fdd0baf3 hevc: properly handle no_rasl_output_flag when removing pictures from the DPB
Fixes ticket #4185.

Reviewed-By: Mickael Raulet <Mickael.Raulet@insa-rennes.fr>
Signed-off-by: Hendrik Leppkes <h.leppkes@gmail.com>
(cherry picked from commit 0118158efa)
2015-10-09 21:33:56 +02:00
Ronald S. Bultje
f7cb979766 hevc: fix wpp threading deadlock.
Fixes ticket 4258.
(cherry picked from commit 74e4948235)
2015-10-09 21:33:35 +02:00
Michael Niedermayer
0fb4a85603 avcodec/ffv1: seperate slice_count from max_slice_count
Fix segfault with too large slice_count
Fixes Ticket4879

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit aa6c43f3fd)

Conflicts:
	libavcodec/ffv1enc.c
2015-10-09 21:32:52 +02:00
wm4
114c0f002a avcodec/mp3: fix skipping zeros
Commits 43bc5cf9 and c5371f77 add code for skipping initial zeros in mp3
packets. This code forgot to report to the user that data was skipped at
all.

Since audio codecs allow partial packet decoding, the user application
has to rely on the return value. It will remove the data reported as
consumed by the decoder, and feed it to the decoder again. This resulted
in the mp3 frame after the zero region to be decoded over and over
again, until the zero region was finally skipped by the application.

Fix this by including the amount of skipped bytes to the number of
consumed bytes returned by the decode call.

Fixes trac ticket #4890.
(cherry picked from commit cb1da9fb8d)
2015-10-09 21:29:03 +02:00
Michael Niedermayer
b1863e3d94 avcodec/ffv1enc: fix assertion failure with unset bits per raw sample
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 749f85496c)

Fixes ticket #4751.
2015-08-04 12:56:52 +02:00
James Almer
a3d45dbcc1 mips/hevcdsp: fix string concatenation on macros
Needed for old compilers like GCC 4.2

Tested by trac user brad. Fixes ticket #4745

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: James Almer <jamrial@gmail.com>
(cherry picked from commit a0092cea46)
2015-07-30 21:01:32 -03:00
Andreas Cadhalpun
27816fb9ef imc: use correct position for flcoeffs2 calculation
flcoeffs2[pos] should be the log2 of flcoeffs1[pos].
flcoeffs1[0] can be 0 here, thus flcoeffs2[pos] gets set to -inf,
causing problems further down.

This seems to have been copied from imc_decode_level_coefficients in
commit 4eb4bb3 without updating the position.

Reviewed-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 75fd5ce4c1)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
2015-07-19 10:14:31 +02:00
Andreas Cadhalpun
f06d9dced4 hevc: check slice address length
It is used as get_bits argument and reading 0 bits isn't supported.

Reviewed-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 05cc8c8e4b)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
2015-07-19 10:01:30 +02:00
Andreas Cadhalpun
26cb351452 snow: remove an obsolete av_assert2
It asserts that the frame linesize is larger than 37, but it can be
smaller and decoding such frames works.

Before commit cc884a35 src_stride > 7*MB_SIZE was necessary, because the
blocks were interleaved in the tmp buffer and the last block was added
with an offset of 6*MB_SIZE.
It was changed for src_stride <= 7*MB_SIZE to write the blocks
sequentially, hence the larger tmp_step.
After that the assert was only necessary to make sure that the buffer
remained large enough.
Since commit bd2b6b33 s->scratchbuf is used as tmp buffer.
As part of commit 86e107a7 the minimal scratchbuf size was increased to
256*7*MB_SIZE, which is enough for any src_stride <= 7*MB_SIZE.

Also add a comment explaining the tmp_step calculation.

Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 3526a120f9)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
2015-07-19 10:01:18 +02:00
Andreas Cadhalpun
762a5878a6 webp: fix infinite loop in webp_decode_frame
The loop always needs at least 8 bytes for chunk_type and chunk_size.
If fewer are left, bytestream2_get_le32 just returns 0 without
reading any bytes, leading to an infinite loop.

Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 0762152f7a)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
2015-07-19 10:00:07 +02:00
Andreas Cadhalpun
f0af6e705f wavpack: limit extra_bits to 32 and use get_bits_long
More than 32 bits can't be stored in an integer and get_bits should not
be used with more than 25 bits.

Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit d0eff8857c)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
2015-07-19 09:43:20 +02:00
Andreas Cadhalpun
acfad331ad pthread_frame: forward error codes when flushing
This is the first part of the fix for ticket #4370.

Reviewed-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 32a5b63126)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
2015-07-19 09:37:42 +02:00
Andreas Cadhalpun
43f8a422b3 huffyuvdec: validate image size
Reviewed-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 9a345802ed)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
2015-07-19 09:37:19 +02:00
Andreas Cadhalpun
95bd0f3a4b wavpack: use get_bits_long to read up to 32 bits
get_bits should not be used for more than 25 bits.

Reviewed-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit f9883a669c)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
2015-07-19 09:37:03 +02:00
Andreas Cadhalpun
7293372959 s302m: fix arithmetic exception
If nb_samples is zero, the bit_rate calculation results in a division by
zero.

Since ff_get_buffer fails if frame->nb_samples is zero, this can be
fixed by moving the bit_rate calculation after that function call.

That also makes it possible to reuse the already calculated
frame->nb_samples value.

Reviewed-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 04dfbc9441)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
2015-07-19 09:36:13 +02:00
Andreas Cadhalpun
2e1226a695 vc1dec: use get_bits_long and limit the read bits to 32
get_bits should not be used with more than 25 bits.

Reviewed-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 1f1e0a2971)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
2015-07-19 09:35:56 +02:00