Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 732c3ebffa)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Several encoders were multiplying the buffer size by 8, in order to get
a bit size. However, the buffer_size argument is for the byte size of
the buffer. We had experienced crashes encoding prores (Anatoliy) at
size 4096x4096.
(cherry picked from commit 50833c9f7b)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Change register constraint on the v variable from = to +. This was causing GCC
to think that the v variable was never read and therefore not initialize it.
This fixes about 20 fate failures on mips64el.
Signed-off-by: James Cowgill <james410@cowgill.org.uk>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit b9de1303a6)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
log2 values which imply numeric overflow are not supported
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 205b2ba3d6)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Fixes out of array read
Fixes: asan_heap-oob_4d4eb0_3994_cov_3169972261_zmbv_15bit.avi
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 1f5c7781e6)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Fixes integer overflow and out of array read.
Fixes: signal_sigsegv_24169e6_3445_cov_3778346427_snow_chroma_bug.avi
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 8f4cbf9402)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
This also fixes a memleak
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 4da351ff0c)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Averaging over 2 pixels doesn't work correctly for the last pixel, because the
rest of the buffer is not initialized.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 87513d6545)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
This fixes the use of uninitialized values in avpriv_do_elbg.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit ab759f8f4a)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
This fixes invalid reads of the packet buffer in av_dup_packet
Based on patch by Andreas Cadhalpun <andreas.cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit d96142e9af)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
This fixes freeing the frame buffer twice on cleanup leading to a crash.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 39e4ed7c1d)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Based on patch by Francisco Blas Izquierdo Riera
Commit message partly taken from carl
fixes a compilation
error in mlpdsp_init.c with -fstack-check and some gcc compilers (I
reproduced the issue with gcc 4.7.3) by simplifying the code.
See also https://bugs.gentoo.org/show_bug.cgi?id=471756
$ make libavcodec/x86/mlpdsp_init.o
libavcodec/x86/mlpdsp_init.c: In function ‘mlp_filter_channel_x86’:
libavcodec/x86/mlpdsp_init.c:142:5: error: can’t find a register in
class ‘GENERAL_REGS’ while reloading ‘asm’
libavcodec/x86/mlpdsp_init.c:142:5: error: ‘asm’ operand has impossible
constraints
4551 -> 4509 dezicycles
Reviewed-by: Ramiro Polla <ramiro.polla@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 03f39fbb2a)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
According to the WebP Lossless Bitstream Specification
"each transform is allowed to be used only once".
If a transform is more than once this can lead to memory
corruption.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit c089e720c1)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Fixes out of array read
Fixes: signal_sigsegv_35bcf26_471_cov_2806540268_CAINIT_A_SHARP_4.bit
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 57e5812198)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
It's the output_window that is applied to the output frame, not
pic_conf_win
(cherry picked from commit 5127c00b97)
Signed-off-by: Anton Khirnov <anton@khirnov.net>
According to the WebP Lossless Bitstream Specification the highest
allowed value for a prefix code is 39.
If prefix_code is too large, the calculated extra_bits has an invalid
value and triggers an assertion in get_bits.
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 5de2dab12b)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
If it doesn't fit into 12 bits it triggers an assertion.
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 2578a54618)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
* commit '77eb3d9a60':
tiff: Check that there is no aliasing in pixel format selection
Conflicts:
libavcodec/tiff.c
See: e1c0cfaa41
Merged-by: Michael Niedermayer <michaelni@gmx.at>
* commit '7136a0bf88':
vorbis: Check the vlc value in setup_classifs
Conflicts:
libavcodec/vorbisdec.c
See: ae038c0914
See: 709cae2bcb
Merged-by: Michael Niedermayer <michaelni@gmx.at>
According to the WebP Lossless Bitstream Specification the highest
allowed value for a prefix code is 39.
If prefix_code is too large, the calculated extra_bits has an invalid
value and triggers an assertion in get_bits.
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Anton Khirnov <anton@khirnov.net>
If it doesn't fit into 12 bits it triggers an assertion.
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Anton Khirnov <anton@khirnov.net>
Normally the aic decoder finds the proper slice combination (multiple of
some number less than 32) but in case of odd width, it resorts to the
default values, which were actually swapped.
The number of slices is modified to account for such odd width cases.
CC: libav-stable@libav.org
(cherry picked from commit e878ec0d47)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
The valid returned values are always at most 11bit.
Remove the previous check that assumed larger values plausible and
use a signed integer to check get_vlc2 return values.
CC: libav-stable@libav.org
(cherry picked from commit 0025f7408a)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
Some files produced by the official encoder have up to 16bit of
padding instead of the expected padding to the byte.
Use a self-explanatory macro instead of a simple number.
CC: libav-stable@libav.org
(cherry picked from commit dbc1163b20)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
* commit '06d433366c':
h264: Do not share rbsp_buffer across threads
Conflicts:
libavcodec/h264.c
See: ecbf838c7d
Merged-by: Michael Niedermayer <michaelni@gmx.at>
* commit '1dbfaa34e6':
h264: only ref cur_pic in update_thread_context if it is initialized
Conflicts:
libavcodec/h264_slice.c
See: 0fc01ae33c
Merged-by: Michael Niedermayer <michaelni@gmx.at>
This fixes out of array reads and/or infinite loops.
30 is the maximum number of bits that can be read into
coeff_abs below.
CC: libav-stable@libav.org
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Martin Storsjö <martin@martin.st>
This prevents using a wrong (first thread's) AVCodecContext if decoding
a frame in the first pass over all threads fails.
(cherry picked from commit a06b0b1295)
Signed-off-by: Anton Khirnov <anton@khirnov.net>
It may be empty if the previous thread's decode call did not contain a
valid frame.
(cherry picked from commit 0dea4c77cc)
Signed-off-by: Anton Khirnov <anton@khirnov.net>
