* commit '1dcb68c02a':
alac: Reject rice_limit 0 if compression is used
Conflicts:
libavcodec/alac.c
See: 4b657a1b1e
Merged-by: Michael Niedermayer <michaelni@gmx.at>
If rice_limit is 0, k can be 0 in decode_scalar, which calls show_bits(gb, k).
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 4b657a1b1e)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
s->decoded_buffer is allocated with a min_size of:
2 * FFALIGN(blockstodecode, 8) * sizeof(*s->decoded_buffer)
Then it is assigned to s->decoded[0] (and s->decoded_buffer + FFALIGN(blockstodecode, 8)
to s->decoded[1]) and passed as out buffer to decode_array_0000.
In this function 64 elements of the out buffer are written
unconditionally and outside the array if blockstodecode is too small.
This causes memory corruption, leading to segmentation faults or other
crashes.
Thus change decode_array_0000 to write at most blockstodecode elements
of the out buffer.
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 699341d647)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Can be tested with:
$ ./configure --cc='cc -m32' --disable-optimizations --enable-pic
(cherry picked from commit b38910c979)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Simplifies the code and makes it build on certain compilers
running out of registers on x86.
CC: libav-stable@libav.org
Reported-By: mudler
(cherry picked from commit e4610300de)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
(cherry picked from commit 4dc0fbb13c)
Signed-off-by: Reinhard Tartler <siretart@tauware.de>
The format uses full range for the gray formats.
CC: libav-stable@libav.org
(cherry picked from commit 0f50c53cfb)
Signed-off-by: Reinhard Tartler <siretart@tauware.de>
If in compression mode rice_limit = 0 leads to call
`show_bits(gb, k)` in `decode_scalar` with k = 0.
Request a sample in case it is valid and it should be accepted.
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
CC: libav-stable@libav.org
(cherry picked from commit cb5324200c)
Signed-off-by: Reinhard Tartler <siretart@tauware.de>
The decode_array_0000 assumed that 64 is the minimal block size
while it is not.
CC: libav-stable@libav.org
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
(cherry picked from commit ac1660509e)
Signed-off-by: Reinhard Tartler <siretart@tauware.de>
It uses 6 registers, unbreaks building on hardened x86 system.
Bug-Id: gentoo/541930
CC: libav-stable@libav.org
(cherry picked from commit 2af720fe5f)
Signed-off-by: Reinhard Tartler <siretart@tauware.de>
So far it is only set in roq_encode_frame, but it is used in
roq_encode_end to free the coded_frame. This currently segfaults if
roq_encode_frame is not called between roq_encode_init and
roq_encode_end.
CC:libav-stable@libav.org
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Anton Khirnov <anton@khirnov.net>
(cherry picked from commit 9f6c36d961)
Signed-off-by: Reinhard Tartler <siretart@tauware.de>
This buffer is resized when vpx_codec_get_cx_data() returns a
VPX_CODEC_STATS_PKT packet.
CC: libav-stable@libav.org
Signed-off-by: Vittorio Giovara <vittorio.giovara@gmail.com>
(cherry picked from commit 7244cefd6e)
Signed-off-by: Reinhard Tartler <siretart@tauware.de>
For max_order = 0 the clipping range is invalid. (amin = 2, amax = 1)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 58d605ee9b)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Also change the type of begin, end and smp to ptrdiff_t to make the
comparison well-defined.
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Reviewed-by: Thilo Borgmann <thilo.borgmann@mail.de>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit afc7748d1f)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
If begin is smaller than t, the subtraction 'begin -= t' wraps around,
because begin is unsigned. The same applies for end < t.
This causes segmentation faults.
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit faf9fe2c22)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
If the same idx is used for more than one i, at least one entry in
sconf->chan_pos remains uninitialized.
This can cause segmentation faults.
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit ef16501aeb)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
avctx->bits_per_raw_sample is used in get_sbits_long, which only
supports up to 32 bits.
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 4c2b88678b)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
For max_order = 0 the clipping range is invalid. (amin = 2, amax = 1)
CC: libav-stable@libav.org
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Anton Khirnov <anton@khirnov.net>
(cherry picked from commit 60f1cc4a1f)
Signed-off-by: Anton Khirnov <anton@khirnov.net>
(cherry picked from commit 378ee3bad5)
Signed-off-by: Anton Khirnov <anton@khirnov.net>
Also change the type of begin, end and smp to ptrdiff_t to make the
comparison well-defined.
CC: libav-stable@libav.org
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Anton Khirnov <anton@khirnov.net>
(cherry picked from commit 94bb1ce882)
Signed-off-by: Anton Khirnov <anton@khirnov.net>
(cherry picked from commit 41a89cba60)
Signed-off-by: Anton Khirnov <anton@khirnov.net>
The minimum of the ath(x, ATH_ADD) function depends on ATH_ADD.
This patch uses the first order approximation to determine it.
For ATH_ADD = 4 this results in the value at 3407.06812 (-5.24241638)
not the one at 3410 (-5.24237967).
CC: libav-stabl@libav.org
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Anton Khirnov <anton@khirnov.net>
(cherry picked from commit 110f7f35fb)
Signed-off-by: Anton Khirnov <anton@khirnov.net>
(cherry picked from commit 7b66cf5ce7)
Signed-off-by: Anton Khirnov <anton@khirnov.net>
avctx->bits_per_raw_sample is used in get_sbits_long, which only
supports up to 32 bits.
CC: libav-stable@libav.org
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Anton Khirnov <anton@khirnov.net>
(cherry picked from commit e191aaca44)
Signed-off-by: Anton Khirnov <anton@khirnov.net>
(cherry picked from commit 97010c74cb)
Signed-off-by: Anton Khirnov <anton@khirnov.net>
CC: libav-stable@libav.org
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Anton Khirnov <anton@khirnov.net>
(cherry picked from commit 8fc8024ea5)
Signed-off-by: Anton Khirnov <anton@khirnov.net>
(cherry picked from commit 0d3a7dd264)
Signed-off-by: Anton Khirnov <anton@khirnov.net>
This fixes an invalid read if end is 0:
band_end = ff_ac3_bin_to_band_tab[end-1] + 1;
Depending on what is before the array, this can cause stack smashing,
when band_end becomes too large.
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit bc4fee7f2a)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
If band->thr is 0.0f, the division is undefined, making norm_fac not a
number or infinity, which causes psy_band->threshold to become NaN.
This is passed on to other variables until it finally reaches
sce->sf_idx and is converted to an integer (-2147483648).
This causes a segmentation fault when it is used as array index.
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Reviewed-by: Claudio Freire <klaussfreire@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit e224aa4191)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 0be54ad280)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
ac may be NULL and then accessing ac->avctx results in a segmentation fault.
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 5b75689b98)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
This fixes segmentation faults, when pic->linesize[0] is negative.
In that case 'line * pic->linesize[0] + pixel_ptr' is treated as
unsigned and wraps around.
This reverts commit 7d78a964.
The problem was introduced in commit f7e1367f, which should obsolete
that commit.
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit ae6fd7300b)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Fixes Ticket4460
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit e88b3852ae)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Fixes non deterministic crash in ticket4408/fuzz2.264
Likely fixes other samples as well
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 43b434210e)
Conflicts:
libavcodec/h264.h
libavcodec/h264_slice.c
(cherry picked from commit dbbc42858e87cdd04e6c3b7694f8b394d4bfcdc6)
Conflicts:
libavcodec/h264_slice.c
Fixes Ticket4408
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit fc58d5c43b)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Fixes out of array access
Fixes: da14e86d8462be6493eab16bc2d40f88/asan_heap-oob_204cfd2_528_cov_340150052_COMPRESS.BMP
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit f7e1367f58)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Fixes out of array read
Fixes: signal_sigsegv_35bcf26_471_cov_2806540268_CAINIT_A_SHARP_4.bit
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 57e5812198)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Fixes Ticket4227
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 2c660e34cf)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Normally the aic decoder finds the proper slice combination (multiple of
some number less than 32) but in case of odd width, it resorts to the
default values, which were actually swapped.
The number of slices is modified to account for such odd width cases.
CC: libav-stable@libav.org
(cherry picked from commit e878ec0d47)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
This ensures that all current and future code-pathes get bpp checked
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit d5e9fc7821)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
