Commit graph

14701 commits

Author SHA1 Message Date
Justin Ruggles
e9de2d98a9 twinvq: check output buffer size before decoding
(cherry picked from commit e53eecd0e7)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-04 03:18:52 +01:00
Alex Converse
93f1159af5 vp6: Fix illegal read.
(cherry picked from commit 2a6eb06254)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-04 01:54:13 +01:00
Justin Ruggles
b08001e00a shorten: check output buffer size before decoding
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-04 01:30:29 +01:00
Justin Ruggles
e1ea35fb52 shorten: check for realloc failure
(cherry picked from commit 9e5e2c2d01)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-04 01:24:03 +01:00
Matthew Einhorn
f531193690 Fixes avpicture_layout to not write past buffer end.
avpicture_get_size() returns the size of buffer required for avpicture_layout.
For pseudo-paletted formats (gray8...) this size does not include the palette.
However, avpicture_layout doesn't know this and still writes the palette. Consequently,
avpicture_layout writes passed the length of the buffer. This fixes it
by fixing avpicture_layout so that it doesn't write the palette for these formats.

Signed-off-by: Matthew Einhorn <moiein2000@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit e662b263d9)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-04 01:23:47 +01:00
Ronald S. Bultje
15a7fe106c pthread: copy coded frame dimensions in update_context_from_thread
Signed-off-by: Janne Grunau <janne-libav@jannau.net>
(cherry picked from commit feadcd1bdc)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-04 01:23:28 +01:00
Ronald S. Bultje
d32f509de1 vp8: prevent read from uninitialized memory in decode_mvs
Signed-off-by: Janne Grunau <janne-libav@jannau.net>
(cherry picked from commit 0f0b5d6434)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-04 01:22:59 +01:00
Ronald S. Bultje
5f5f36b52e vp8: force reallocation in update_thread_context after frame size change
Signed-off-by: Janne Grunau <janne-libav@jannau.net>
(cherry picked from commit 5653579381)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-04 01:22:52 +01:00
Ronald S. Bultje
d1166f03be vp8: fix return value if update_dimensions fails
Signed-off-by: Janne Grunau <janne-libav@jannau.net>
(cherry picked from commit f05c2fb6eb)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-04 01:22:45 +01:00
Justin Ruggles
62cf52c860 truespeech: check to make sure channels == 1
(cherry picked from commit 3e7a176759)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-04 01:09:22 +01:00
Justin Ruggles
7e95a12d51 mlpdec: validate that the reported channel count matches the actual output
channel count
(cherry picked from commit caa845851d)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-04 01:09:17 +01:00
Mans Rullgard
1c3d46a924 h264: fix HRD parameters parsing
The bit_rate_value_minus1 and cpb_size_value_minus1 elements
allow a wider range than get_ue_golomb() supports.  This
adds a get_ue_golomb_long() function supporting up to 31
leading zeros, which is the maximum for these syntax
elements, and uses it in decode_hrd_parameters().

Signed-off-by: Mans Rullgard <mans@mansr.com>
(cherry picked from commit fdba370f8a)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-04 01:08:09 +01:00
Justin Ruggles
800ab099e3 smacker: validate channels and sample format.
(cherry picked from commit ff1f89de2d)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-04 01:07:49 +01:00
Justin Ruggles
e6b2255329 smacker: check buffer size before reading output size
(cherry picked from commit cf044f8bff)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-04 01:07:44 +01:00
Justin Ruggles
7f7b2e89e2 smacker: validate number of channels
(cherry picked from commit e190e453bd)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-04 01:07:39 +01:00
Mans Rullgard
73f85eae68 sipr: fix get_bits(0) calls
Zero-length get_bits() is undefined, must check before calling.

Signed-off-by: Mans Rullgard <mans@mansr.com>
(cherry picked from commit c79d2a20ba)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-04 01:02:46 +01:00
Mans Rullgard
190807a56c 4xm: fix signed overflow
Signed-off-by: Mans Rullgard <mans@mansr.com>
(cherry picked from commit 84dda40762)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-04 01:02:17 +01:00
Mans Rullgard
33029d7353 wmavoice: fix a signed overflow
Signed-off-by: Mans Rullgard <mans@mansr.com>
(cherry picked from commit ba3f07d061)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-04 01:02:13 +01:00
Mans Rullgard
c41950099d mpegvideo_enc: fix a signed overflow
Signed-off-by: Mans Rullgard <mans@mansr.com>
(cherry picked from commit 05795f35be)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-04 01:02:08 +01:00
Mans Rullgard
115d88c4b2 h264pred: use unsigned types for pixel values, fix signed overflows
Signed-off-by: Mans Rullgard <mans@mansr.com>
(cherry picked from commit 60f10e0ad3)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-04 01:01:48 +01:00
Laurent Aimar
a65045915f qtrle: check for out of bound writes.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 7fb92be7e5)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-04 01:01:23 +01:00
Laurent Aimar
adb12c4deb xxan: check for out of bound accesses
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit a68a6a4fb1)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-04 01:01:15 +01:00
Laurent Aimar
ca58b215ab txd: check for out of bound reads.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit e182de9a98)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-04 01:01:11 +01:00
Laurent Aimar
67c46b9b30 qtrle: check for invalid line offset
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit a4ed7c3fe9)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-04 01:01:07 +01:00
Laurent Aimar
7ab0b6b7ed vqavideo: check for out of bound reads.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 6d45702f7f)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-04 01:01:04 +01:00
Laurent Aimar
2fdbc1d553 vqavideo: check for invalid/unsupported version
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit b226af3910)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-04 01:00:54 +01:00
Laurent Aimar
5415c488f9 eamad: release the reference frame on video size changes
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 6c1fb3e763)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-04 01:00:49 +01:00
Laurent Aimar
79bafbb0dd eamad: check for out of bound reads when doing MC
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit da35797359)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-04 01:00:43 +01:00
Laurent Aimar
7b3c851526 eamad: avoid NULL derefence when missing the reference frame.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 6e20554a6d)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-04 01:00:38 +01:00
Laurent Aimar
1b6e6439fa eatgv: fix pointer arithmetic overflows.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 6bfe0d4c3d)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-04 01:00:34 +01:00
Laurent Aimar
4474051370 eatgv: fix out of bound reads on corrupted motions vectors.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 09302a897d)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-04 01:00:29 +01:00
Laurent Aimar
1646d2d2ae eamad: clear FF_INPUT_BUFFER_PADDING_SIZE bytes at the end of the temporary buffer
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 74b9c59839)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-04 01:00:23 +01:00
Mans Rullgard
f7be632cbd vp8: fix signed overflows
In addition to avoiding undefined behaviour, an unsigned type
makes more sense for packing multiple 8-bit values.

Signed-off-by: Mans Rullgard <mans@mansr.com>
(cherry picked from commit bb59156606)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-04 01:00:12 +01:00
Mans Rullgard
4ba0e03759 motion_est: fix some signed overflows
Signed-off-by: Mans Rullgard <mans@mansr.com>
(cherry picked from commit e708afd3c0)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-04 01:00:09 +01:00
Mans Rullgard
37ce6ba425 dca: fix signed overflow in shift
Signed-off-by: Mans Rullgard <mans@mansr.com>
(cherry picked from commit 559c244d42)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-04 01:00:04 +01:00
Mans Rullgard
c2c83dcb32 aacdec: fix undefined shifts
Since nnz can be zero, this is needed to avoid a shift by 32.

Signed-off-by: Mans Rullgard <mans@mansr.com>
(cherry picked from commit d12294304a)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-04 00:59:58 +01:00
Mans Rullgard
4c5cdb493c put_bits: fix invalid shift by 32 in flush_put_bits()
If flush_put_bits() is called when the 32-bit buffer is empty,
e.g. after writing a multiple of 32 bits, and invalid shift by
32 is performed.  Since flush_put_bits() is called infrequently,
this additional check should have negligible performance impact.

Signed-off-by: Mans Rullgard <mans@mansr.com>
(cherry picked from commit ac6eab1496)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-04 00:59:53 +01:00
Laurent Aimar
06b15b3715 h264: fix the size of PPS::chroma_qp_table
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit e588a5c2d4)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-04 00:59:41 +01:00
Michael Niedermayer
614ef0dc0d h264: fix fill_colmap() to not store entries mbaff style when the reference is not mbaff at all
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit a3ba542af3)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-04 00:59:34 +01:00
Ronald S. Bultje
5d2b6006f0 mpegvideo: fix position of bottom edge.
It was wrong in colorspaces where horizontal and vertical chroma
subsampling are not the same, e.g. 422.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-04 00:58:35 +01:00
Laurent Aimar
b491c15c85 h254: explicitly initialize bit depth/chroma idc
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-04 00:53:56 +01:00
Justin Ruggles
2809f4ab93 qcelp: check output buffer size before decoding
(cherry picked from commit e43dd3d2a8)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-04 00:43:10 +01:00
Justin Ruggles
c2d017e88f sipr: fix the output data size check and only calculate it once.
(cherry picked from commit 1b5a189f06)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-04 00:42:59 +01:00
Michael Niedermayer
4f45967cf5 ff_dv_frame_profile2: Check input buffer size.
Based on code by DivX, Inc. / drffmpeg

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 51b0694bc0)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-04 00:42:46 +01:00
Justin Ruggles
78eab18740 qdm2: check output buffer size before decoding
(cherry picked from commit 7d49f79f1c)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-04 00:42:37 +01:00
Justin Ruggles
fc8c0ee09f mpc8: check output buffer size before decoding
(cherry picked from commit 5674d4b0a3)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-04 00:40:13 +01:00
Justin Ruggles
490617b6ff mpc7: return error if packet is too small.
(cherry picked from commit 8290d1f38b)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-04 00:40:03 +01:00
Justin Ruggles
b833859daa mpc7: check output buffer size before decoding
(cherry picked from commit c8b5c4d274)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2011-11-04 00:39:47 +01:00
Gwenole Beauchesne
7d52ed686b vaapi: fix VC-1 decoding (reconstruct bitstream TTFRM correctly).
(cherry picked from commit 825dd135d8)
2011-10-12 11:27:11 +02:00
Laurent Aimar
f74d1c6de7 h264: do not let invalid values in h->ref_count after a decoder reset.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 0333d234b0)
2011-10-11 21:34:15 +02:00