Commit graph

31648 commits

Author SHA1 Message Date
Michael Niedermayer
342b6d3718 avcodec/mjpegdec: Do not try to detect last scan but apply idct after all scans for progressive jpeg
Fixes: IMG-20160418-WA0002.jpg

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit deaf58abf2)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-05-02 23:59:14 +02:00
Michael Niedermayer
72d8ce1cf2 avcodec/ttaenc: Reallocate packet if its too small
Fixes assertion failure
Fixes Ticket5394

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 005c61c6b8)

Conflicts:

	libavcodec/ttaenc.c
2016-04-29 14:23:41 +02:00
Jan Ekström
19fcdd0c2f pgssubdec: fix subpicture output colorspace and range
Functionality used before didn't widen the values from limited to
full range. Additionally, now the decoder uses BT.709 where it
should be used according to the video resolution.

Default for not yet set colorimetry is BT.709 due to most observed
HDMV content being HD.

BT.709 coefficients were gathered from the first two parts of BT.709
to BT.2020 conversion guide in ARIB STD-B62 (Pt. 1, Chapter 6.2.2).
They were additionally confirmed by manually calculating values.

Fixes #4637
(cherry picked from commit 9779b62624)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-04-29 14:23:41 +02:00
Michael Niedermayer
542cc06c64 avcodec/ac3dec: Reset SPX when switching from EAC3 to AC3
Fixes Ticket5319

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 9ac154d1fa)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-04-29 14:23:41 +02:00
Paul B Mahol
17e7cb98be avcodec/takdec: add code that got somehow lost in process of REing
Signed-off-by: Paul B Mahol <onemda@gmail.com>
(cherry picked from commit 38797a8033)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-04-29 14:23:41 +02:00
Paul B Mahol
945899864a avcodec/apedec: fix decoding of stereo files with one channel full of silence
Signed-off-by: Paul B Mahol <onemda@gmail.com>
(cherry picked from commit 9149e9c0ba)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-04-29 14:23:41 +02:00
Michael Niedermayer
e49d142fc0 avcodec/avpacket: Fix off by 5 error
Fixes out of array read
Fixes: mozilla bug 1266129
Found-by: Tyson Smith
Tested-by: Tyson Smith
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 9f36ea57ae)

Conflicts:

	libavcodec/avpacket.c
2016-04-29 14:23:41 +02:00
Ivan
f3ec2a0859 avcodec/h264: Fix for H.264 configuration parsing
Sometimes video fails to decode if H.264 configuration changes mid stream.
The reason is that configuration parser assumes that nal_ref_idc is equal to 11b
while actually some codecs but 01b there. The H.264 spec is somewhat
vague about this but it looks like it allows any non-zero nal_ref_idc for sps/pps.

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 3a727606c4)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-04-29 14:23:41 +02:00
Michael Niedermayer
036d049fb1 avcodec/bmp_parser: Ensure remaining_size is not too small in startcode packet crossing corner case
Fixes Ticket 5438

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 8e26bdd59b)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-04-29 14:23:41 +02:00
Aaron Boxer
6ec9611971 avcodec/j2kenc: Add attribution to OpenJPEG project:
http://ghostscript.com/~tor/gs-browse/gs/openjpeg/libopenjpeg/t1.c

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit b6b4b0a65e)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-04-29 14:23:41 +02:00
Michael Niedermayer
cb7809cc4d avcodec/libutvideodec: copy frame so it has reference counters when refcounted_frames is set
Reviewed-by: maintainer
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 0cd9ff4e3a)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-04-29 14:23:41 +02:00
Michael Niedermayer
f2019a0db6 avcodec/mjpegenc_common: Store approximate aspect if exact cannot be stored
Fixes Ticket5244

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 068026b0f7)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-04-29 14:23:41 +02:00
Michael Niedermayer
e1fb0a1dbc avcodec/resample: Remove disabled and faulty code
Fixes Ticket5345

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 50ef7361cb)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-04-29 14:23:41 +02:00
Luca Barbato
b55b843cd9 indeo2: Fix banding artefacts
Rename luma table to delta table and change how it is used.

CC: libav-stable@libav.org
Signed-off-by: Vittorio Giovara <vittorio.giovara@gmail.com>
Signed-off-by: Diego Biurrun <diego@biurrun.de>
(cherry picked from commit f8c34f4b8d)
(cherry picked from commit 73f3c8f73e)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-04-29 14:23:41 +02:00
Luca Barbato
b3d36aec75 indeo2data: K&R formatting cosmetics
Signed-off-by: Vittorio Giovara <vittorio.giovara@gmail.com>
Signed-off-by: Diego Biurrun <diego@biurrun.de>
(cherry picked from commit d4066a7024)
(cherry picked from commit 522ab0b9a9)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-04-29 14:23:41 +02:00
Przemysław Sobala
1eb1c2fd5d avcodec/imgconvert: Support non-planar colorspaces while padding
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 0d097a869c)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-04-29 14:23:41 +02:00
Michael Niedermayer
778ee32990 avcodec/mjpegdec: Fix decoding slightly odd progressive jpeg
Fixes: ebd58db6-dc86-11e5-91c2-59daeddf50c7.jpg

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit c6f4720b86)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-04-29 14:23:40 +02:00
Michael Niedermayer
97da8f43a4 avcodec/avpacket: clear priv in av_init_packet()
This should fix leaving uninitialized pointers in priv which can confuse
user applications.
See: https://github.com/golang/go/issues/14426

Only or release branches

Reviewed-by: Derek Buitenhuis <derek.buitenhuis@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-04-29 14:23:40 +02:00
Michael Niedermayer
684e189eb3 avcodec/jpeg2000dec: More completely check cdef
Fixes out of array access
Fixes: j2k-poc.bin

Found-by: Lucas Leong <wmliang.tw@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 0aada30510)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-01-31 00:24:28 +01:00
Paul B Mahol
f71e0b798a avcodec/flacenc: fix calculation of bits required in case of custom sample rate
Sample rate of 11025 takes 16 bits but previous code would pick only 8.
Fixes assertion failure.

Reviewed-by: Rostislav Pehlivanov <atomnuker@gmail.com>
Signed-off-by: Paul B Mahol <onemda@gmail.com>
(cherry picked from commit 3e7d684912)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-01-31 00:24:28 +01:00
Michael Niedermayer
517c856d7f avcodec/mjpegdec: Check for end for both bytes in unescaping
Fixes assertion failure
Fixes: c40c779601b77dc6e19aaea0b04b9751/signal_sigabrt_7ffff6ae7cb7_5769_b94f6ec70caecb2d3d76b4771b109ac1.avi

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 509c9e74e5)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-01-31 00:24:28 +01:00
Michael Niedermayer
302a3269d6 avcodec/mpegvideo_enc: Check for integer overflow in ff_mpv_reallocate_putbitbuffer()
Fixes assertion failure
Fixes: 6568d187979ce17878b6fe5fbbb89142/signal_sigabrt_7ffff6ae7cb7_7176_564bbc6741bdcf907f5c4e685c9a77a2.mpg

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit b65efbc0f4)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-01-31 00:24:28 +01:00
Michael Niedermayer
0d312030d2 avcodec/wmaenc: Check ff_wma_init() for failure
Fixes null pointer dereference
Fixes: c4faf8280ba366bf00a79d425f2910a8/signal_sigsegv_1f96477_5177_1448ba7e4125faceb966f44ceb69abfa.qcp
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 19e456d48c)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-01-31 00:24:28 +01:00
Michael Niedermayer
ee5ba0a1ad avcodec/mpeg12enc: Move high resolution thread check to before initializing threads
Cleaner solution is welcome!

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit a53fbda9dc)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-01-31 00:24:28 +01:00
Michael Niedermayer
0b70b546a2 avcodec/ass_split: Fix null pointer dereference in ff_ass_style_get()
Fixes: 55d71971da50365d542ed14b65565fe1/signal_sigsegv_4765a4_8499_f146af090a94f591d6254515c7700ef5.mkv

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Reviewed-by: Andreas Cadhalpun <andreas.cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 158f0545d8)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-01-31 00:24:28 +01:00
Michael Niedermayer
2b7f125af7 avcodec/gif: Fix lzw buffer size
Fixes out of array access
Fixes: aaa479088e6fb40b04837b3119f47b04/asan_heap-oob_e38c68_8576_9d653078b2470700e2834636f12ff557.tga

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 03d83ba34b)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-01-31 00:24:27 +01:00
Michael Niedermayer
8394fa2696 avcodec/put_bits: Assert buf_ptr in flush_put_bits()
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 3ef5de0f19)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-01-31 00:24:27 +01:00
Michael Niedermayer
524ee42050 avcodec/tiff: Check subsample & rps values more completely
Fixes out of array access
Fixes: 83aedfb29af669c4d6e10f1bfad974d2/asan_heap-oob_1ab42fe_4984_9f6ec14462f8d8a00ea24b320572a963.tif

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 89f464e9c2)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-01-31 00:24:27 +01:00
Michael Niedermayer
9acbe5fa84 avcodec/aacenc: Check both channels for finiteness
Fixes null pointer dereference
Fixes: 10412fc52ecc6eab40ed67f82ca7b372/signal_sigsegv_2618c99_2129_f808373959e46afb165593332799ffbc.aif

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 057549a9cc)

Conflicts:

	libavcodec/aacenc.c
2016-01-31 00:24:27 +01:00
Andreas Cadhalpun
1d8a6a46a3 dca: fix misaligned access in avpriv_dca_convert_bitstream
src and dst are only 8-bit-aligned, so accessing them as uint16_t causes
SIGBUS crashes on architectures like sparc.

This fixes ubsan runtime error: load of misaligned address for type
'const uint16_t', which requires 2 byte alignment

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 44ac13eed4)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
2016-01-28 01:21:15 +01:00
Andreas Cadhalpun
b244e67f9c vorbisdec: reject rangebits 0 with non-0 partitions
This causes non-unique elements in floor_setup->data.t1.list, which
makes the stream undecodable according to the specification.

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit e7a7b3135a)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
2016-01-28 01:20:25 +01:00
Andreas Cadhalpun
9dd768dc1e vorbisdec: reject channel mapping with less than two channels
It causes the angle channel number to equal the magnitude channel
number, which makes the stream undecodable according to the
specification.

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit b4b13848de)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
2016-01-28 01:20:21 +01:00
Paul B Mahol
eadf932867 avformat/ipmovie: put video decoding_map_size into packet and use it in decoder
The size of decoding map can differ from one calculated
internally, producing artifacts while decoding video.

Signed-off-by: Paul B Mahol <onemda@gmail.com>
(cherry picked from commit c293ef258c)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
2016-01-28 01:20:12 +01:00
James Almer
b901d1bbc9 avcodec/wavpackenc: print channel count in av_log call
Fixes a warning with -Wformat-extra-args
(cherry picked from commit 17e7fdf61a)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-01-15 17:53:59 +01:00
Michael Niedermayer
9b8d1a183d avcodec/mpeg4video: Check time_incr
Fixes assertion failure
Fixes out of memory access

Fixes: test_casex.ivf

Found-by: Tyson Smith <twsmith@mozilla.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 7c97946d61)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-01-15 17:23:55 +01:00
Michael Niedermayer
6d48354321 avcodec/wavpackenc: Check the number of channels
They are stored in a byte, thus more than 255 is not possible

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 59c915a403)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-01-15 17:23:55 +01:00
Michael Niedermayer
60956a9dc8 avcodec/wavpackenc: Headers are per channel
Fixes: 1b8b83a53bfa751f01b1daa65a4758db/signal_sigabrt_7ffff6ae7cb7_7488_403f71d1a2565b598d01b6cb110fac8f.aiff
Fixes: assertion failure

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 26757b0279)

Conflicts:

	libavcodec/wavpackenc.c

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-01-15 17:23:55 +01:00
Michael Niedermayer
526b5facc5 avcodec/dvdec: Fix "left shift of negative value -254"
Fixes: dvdec_left_shift.avi

Found-by: Piotr Bandurski <ami_stuff@o2.pl>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 93ac72a98d)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-01-15 17:23:55 +01:00
Michael Niedermayer
bb1263185b avcodec/mjpegdec: Fix negative shift
Fixes: mjpeg_left_shift.avi

Found-by: Piotr Bandurski <ami_stuff@o2.pl>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit d86d7b2486)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-01-15 17:23:55 +01:00
Michael Niedermayer
aa5ae8b213 avcodec/mss2: Check for repeat overflow
Fixes: mss2_left_shift.wmv

Found-by: Piotr Bandurski <ami_stuff@o2.pl>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit e273dade78)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-01-15 17:23:55 +01:00
Michael Niedermayer
81bac54ab3 avcodec/mpegvideo_enc: Clip bits_per_raw_sample within valid range
Fixes out of array read
Fixes: test_case-mdc.264 (b47be15a120979f5a1a945c938cbef33)

Found-by: Tyson Smith <twsmith@mozilla.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 13f266b50c)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-01-15 17:23:54 +01:00
Michael Niedermayer
f3cf94fa47 avcodec/motion_est: Fix mv_penalty table size
Fixes out of array read

Found-by: Tyson Smith <twsmith@mozilla.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 5b4da8a38a)

Conflicts:

	libavcodec/motion_est.h
2016-01-15 17:23:54 +01:00
Michael Niedermayer
85732714ce avcodec/h264_slice: Fix integer overflow in implicit weight computation
Fixes mozilla bug 1230423

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 7cc01c2572)

Conflicts:

	libavcodec/h264_slice.c

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-01-15 17:23:54 +01:00
Michael Niedermayer
9c4d8343bb avcodec/put_bits: Always check buffer end before writing
This causes a overall slowdown of 0.1 % (tested with mpeg4 single thread encoding of matrixbench at QP=3)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit cccb0ffccc)

Conflicts:

	libavcodec/put_bits.h
2016-01-15 17:23:54 +01:00
Andreas Cadhalpun
a5fb5a18e3 mjpegdec: extend check for incompatible values of s->rgb and s->ls
This can happen if s->ls changes from 0 to 1, but picture allocation is
skipped due to s->interlaced.

In that case ff_jpegls_decode_picture could be called even though the
s->picture_ptr frame has the wrong pixel format and thus a wrong
linesize, which results in a too small zero buffer being allocated.

This fixes an out-of-bounds read in ls_decode_line.

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 7ea2db6eaf)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-01-15 17:23:54 +01:00
Andreas Cadhalpun
462afe5d00 rawdec: only exempt BIT0 with need_copy from buffer sanity check
Otherwise the too small buffer is directly used in the frame, causing
segmentation faults, when trying to use the frame.

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 699e68371e)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
2015-12-20 15:06:31 +01:00
Andreas Cadhalpun
c4b2985d9a xwddec: prevent overflow of lsize * avctx->height
This is used to check if the input buffer is large enough, so if this
overflows it can cause a false negative leading to a segmentation fault
in bytestream2_get_bufferu.

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 9d38f06d05)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
2015-12-20 15:06:19 +01:00
Andreas Cadhalpun
350f1f4919 exr: fix out of bounds read in get_code
This macro unconditionally used out[-1], which causes an out of bounds
read, if out is the very beginning of the buffer.

Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 90b99a8107)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
2015-12-20 15:06:10 +01:00
Andreas Cadhalpun
564e63a033 on2avc: limit number of bits to 30 in get_egolomb
More don't fit into the integer output.

Also use get_bits_long, since get_bits only supports reading up to 25
bits, while get_bits_long supports the full integer range.

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 4d5c3b02e9)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
2015-12-20 15:06:05 +01:00
Michael Niedermayer
c1b7e9fe52 avcodec/mpeg4videodec: also for empty partitioned slices
Fixes assertion failure
Fixes: id_acf3e47f864e1ee4c7b86c0653e0ff31e5bde56e.m4v

Found-by: Andreas Cadhalpun <andreas.cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 70f13abb4f)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-20 10:59:28 +01:00