Commit graph

146 commits

Author SHA1 Message Date
TreeHugger Robot
6cb07e106a Merge "decoder: Remove skip picture support" into qt-dev 2019-05-04 21:16:35 +00:00
TreeHugger Robot
a09be99800 Merge "Decoder: Add missing error checks for parsing sps or pps" into qt-dev 2019-05-04 19:37:30 +00:00
Harish Mahendrakar
4f95fc0b9c decoder: Remove skip picture support
SKIP_P and SKIP_PB support was experimental and is not used
in libstagefright

Bug: 130541597
Test: poc in bug
Change-Id: If13b435e80d13d57de2fee0aa07ea7e6abca481b
2019-05-01 13:18:27 -07:00
Ritu Baldwa
b3427a4c49 Decoder: Add missing error checks for parsing sps or pps
Return error to application if valid sps or pps has not been decoded.
Also add check to prevent parsing of pps, if valid sps is not present.

Test: Tested a few clips with missing sps/pps
Bug: 129746168
Bug: 130575087

Change-Id: I1a6409d1cffc66c1b910d2102ab10a94e0227107
2019-05-01 13:01:23 -07:00
Harish Mahendrakar
54160a6d3e decoder: Treat unsupported feature errors as fatal errors
This will ensure clips with supported bitdepth or unsupport color
formats are not attempted to be decoded till end of the sequence.

Bug: 130578045
Test: adb shell stagefright -s -S /sdcard/crowd422_h264.mp4
Test: Ensure decoder doesn't attemp decoding all frames

Change-Id: I118ef2141f77986f6ec65c91dfeaa7ef17646c43
2019-04-30 17:14:01 -07:00
Isha Kulkarni
34769a5b08 Decoder: Fix Integer Sanitizer Issues
Bug: 118445723
Test: vendor
Change-Id: I719898beb93ca5d20d9822f685ac6d4920252cdf
2019-03-28 14:40:40 -07:00
Rakesh Kumar
590ff56ba5 Decoder: Memset allocated dynamic and static buffer to zero
Bug: 111407302
Bug: 111208713
Bug: 111128067
Bug: 111762807
Bug: 111762100
Bug: 111761624
Bug: 112328051
Bug: 112329563
Bug: 112326322
Bug: 112205113
Bug: 112204376
Test: manual
Change-Id: I35e6069a26ee5ef4c0fff111df255a5eddff78ae
2018-08-23 17:27:22 -07:00
Ray Essick
e31fa45761 Merge "Decoder: Adding Error Check for Output Buffer Size in Shared Display Mode." into mnc-dev am: afc729db25 am: 1fb89d8488 am: 56f836a29c am: 33675f68d6 am: 547804d647 am: 7adc4d57f1 am: cfe44b34c0 am: 17b987beec
am: bce27b7e66

Change-Id: I6755dedeb1a32a32e85627602783ca77cd966fc7
2018-01-08 22:49:54 +00:00
Ray Essick
7adc4d57f1 Merge "Decoder: Adding Error Check for Output Buffer Size in Shared Display Mode." into mnc-dev am: afc729db25 am: 1fb89d8488 am: 56f836a29c am: 33675f68d6
am: 547804d647

Change-Id: I22c555cc975abbfa80aa38810c9e7a0f59ce3450
2018-01-08 22:41:22 +00:00
Ray Essick
1fb89d8488 Merge "Decoder: Adding Error Check for Output Buffer Size in Shared Display Mode." into mnc-dev
am: afc729db25

Change-Id: Ic57cdca65a505c45bec9c6ff8dd9b5d9dab5fec7
2018-01-08 22:27:38 +00:00
Ritu Baldwa
3692aceb1b Decoder: Adding Error Check for Output Buffer Size in Shared Display Mode.
The output buffer size given by the application, needs to be checked
in every process call. This is required in the case of resolution
change in shared display mode.

Bug: 70294343
Bug: 70350193
Bug: 70526411
Bug: 70526485
Test: manual
Change-Id: I2c1e59425e84ac62a874e5ee180e1b98f0a4058f
2018-01-02 14:25:27 -08:00
Harish Mahendrakar
423b7db2fb Merge "Decoder: Handle dec_hdl memory allocation failure gracefully" into mnc-dr-dev am: ac0d9cecb5 am: f63836fed1 am: 5448bcea3e am: f87d4848af am: 18ae83b015 am: 04b8990195 am: 7fc07418ef
am: faf7af464e

Change-Id: I68e929751f26ccbbe9af1104f84bebc07530c6af
2017-11-03 19:25:07 +00:00
Harish Mahendrakar
18ae83b015 Merge "Decoder: Handle dec_hdl memory allocation failure gracefully" into mnc-dr-dev am: ac0d9cecb5 am: f63836fed1 am: 5448bcea3e
am: f87d4848af

Change-Id: Ia060e78fe7cd7b4233733910f08caadbea05ff3d
2017-11-03 19:07:04 +00:00
TreeHugger Robot
ac0d9cecb5 Merge "Decoder: Handle dec_hdl memory allocation failure gracefully" into mnc-dr-dev 2017-11-03 18:57:47 +00:00
Harish Mahendrakar
7720b3fe3d Decoder: Handle dec_hdl memory allocation failure gracefully
If memory allocation for dec_hdl fails, return gracefully
with an error code. All other allocation failures are
handled correctly.

Bug: 68300072
Test: ran poc before/after
Change-Id: I118ae71f4aded658441f1932bd4ede3536f5028b
2017-11-01 09:57:48 -07:00
Hamsalekha S
a1f8cef8c4 Decoder: Increased allocation and added checks in sei parsing. am: f2b70d3537 am: 8d32a711ac am: a0827a3bb1 am: cd831159ce am: 2a3efcfcde am: 013d98b763 am: 48d8b5bbc9
am: ccc5298ffb

Change-Id: I83547b11f6da1c33f0ac304690924eacc2ba2920
2017-10-31 18:23:49 +00:00
Hamsalekha S
2a3efcfcde Decoder: Increased allocation and added checks in sei parsing. am: f2b70d3537 am: 8d32a711ac am: a0827a3bb1
am: cd831159ce

Change-Id: I49ada0ebdcca121f739c753dbedaca8e80ea6da5
2017-10-31 18:10:47 +00:00
Ray Essick
27e6efc8ec DO NOT MERGE Decoder: Increased allocation and added checks in sei parsing.
This prevents heap overflow while parsing sei_message.

Bug: 63122634
Test: ran PoC on unpatched/patched
Change-Id: I4785927b68cb17a3ca51e23aeaf96aacacf116d3
Merged-In: I61c1ff4ac053a060be8c24da4671db985cac628c
2017-10-17 15:09:06 -07:00
Hamsalekha S
f2b70d3537 Decoder: Increased allocation and added checks in sei parsing.
This prevents heap overflow while parsing sei_message.

Bug: 63122634
Test: ran PoC on unpatched/patched
Change-Id: I61c1ff4ac053a060be8c24da4671db985cac628c
2017-10-17 15:03:19 -07:00
TreeHugger Robot
6a09a35355 Merge "DO NOT MERGE Decoder: Modified setting of error code in set flush api." into mnc-dev 2017-10-03 22:51:37 +00:00
Ritu Baldwa
1541dcc79c DO NOT MERGE Decoder: Modified setting of error code in set flush api.
Fixed incorrect use of ps_dec->pv_dec_out to set error code.

Bug: 66372937
Test: at vendor
Merged-In: Ib04e0b15573b2482c9d5b43c8bc7dd30d8f8efdd
Change-Id: I7b66ee010089399c050a75d6d67feb03da0b8b3e
2017-10-03 14:57:06 -07:00
Ritu Baldwa
ed7a63fbcc DO NOT MERGE Decoder: Increased memory allocation.
ps_dec->u1_recon_mb_grp is twice the width in case of mbaff,
increasing relevant allocations accordingly.

Increased allocation of intra-prediction buffer to
include padding.

Bug: 64964675
Test: POC from bug report
Change-Id: Ic4a6151bb12ac1122c228220b9150b2a372aae21
2017-09-29 11:49:50 -07:00
Harish Mahendrakar
31b397c813 Added an out of bound check on u4_num_bufs in input argument am: aa11ab9fdb am: 50ec420228 am: 0514a1ebfb am: fa8bb9d89c am: 96228ba3b9 am: 4a6a743d40 am: 4ed04c7bb1 am: b9027e8cc1
am: 859498bb08

Change-Id: I364cdcdd56d88f411892723935683d16ffcd0321
2017-09-11 22:39:38 +00:00
Hamsalekha S
1d6b06a73a Decoder: Conceal picture only if valid picture buffer is obtained. am: 5df744afde am: 7f78e3aa9a am: 8810e9db7f am: 1ab3e51baf am: 32c369cc3f am: fd327a47fb am: b10090d47a
am: 2825c578d3

Change-Id: I1bf0cb0c96c45b419c06bd9b1d202b1ff08193e4
2017-09-11 22:39:14 +00:00
Harish Mahendrakar
4a6a743d40 Added an out of bound check on u4_num_bufs in input argument am: aa11ab9fdb am: 50ec420228 am: 0514a1ebfb am: fa8bb9d89c
am: 96228ba3b9

Change-Id: Ic048f96697bdaedea3bbc0a8cf1acf78ed488e37
2017-09-11 22:30:10 +00:00
Hamsalekha S
32c369cc3f Decoder: Conceal picture only if valid picture buffer is obtained. am: 5df744afde am: 7f78e3aa9a am: 8810e9db7f
am: 1ab3e51baf

Change-Id: I964f1bd1fc0d9534d6c9498864cc7831d13de449
2017-09-11 22:28:11 +00:00
Harish Mahendrakar
50ec420228 Added an out of bound check on u4_num_bufs in input argument
am: aa11ab9fdb

Change-Id: I7dd6739821da87a96072dae422416e8037332a4f
2017-09-11 22:22:40 +00:00
Ray Essick
dc9396971d Merge "DO NOT MERGE: Decoder: Conceal picture only if valid picture buffer is obtained." into mnc-dev 2017-09-11 22:17:06 +00:00
Ray Essick
45aa4ac86d DO NOT MERGE: Decoder: Conceal picture only if valid picture buffer is obtained.
If all the slices in the current pic were invalid, then
the decoder would not have received a valid picture buffer
in the current call. In such cases there is no need to conceal or
deblock the picture.

Bug: 62896384
Test: run ASAN-enabled PoC before/after the patch
Change-Id: Ia7d979a78ae3f4fb443a1759c8e6cf8780565ad8
Merged-In: I3cf6e871592826f93b0dcd2b06fff80677bc8338
2017-09-06 23:46:21 +00:00
Hamsalekha S
5df744afde Decoder: Conceal picture only if valid picture buffer is obtained.
If all the slices in the current pic were invalid, then
the decoder would not have received a valid picture buffer
in the current call. In such cases there is no need to conceal or
deblock the picture.

Bug: 62896384
Test: run ASAN-enabled PoC before/after the patch
Change-Id: I3cf6e871592826f93b0dcd2b06fff80677bc8338
2017-08-18 20:45:17 +00:00
Harish Mahendrakar
aa11ab9fdb Added an out of bound check on u4_num_bufs in input argument
ps_dec_ip->s_out_buffer.u4_num_bufs was missing out of bound checks

Bug: 62688399
Change-Id: Ic5e5c002d29fcb18064550d5a5f9289bb68b448e
2017-08-10 10:15:16 -07:00
Hamsalekha S
2e10678938 Added error check for output buffer size. am: 3f6c941de5 am: f0d2caa214 am: 9a6312ec9f am: 83c397d1ab am: 462316f8d2 am: 564bdc7609 am: 3aa85a9cc5 am: 873d325720
am: 08d268a942

Change-Id: I04d24975487e9188d30ff1ebac6f76e563143ef4
2017-06-19 20:49:53 +00:00
Hamsalekha S
564bdc7609 Added error check for output buffer size. am: 3f6c941de5 am: f0d2caa214 am: 9a6312ec9f am: 83c397d1ab
am: 462316f8d2

Change-Id: Id9021bcb9534cd9b30063eb9be6de464014c67d7
2017-06-19 20:26:20 +00:00
Hamsalekha S
f0d2caa214 Added error check for output buffer size.
am: 3f6c941de5

Change-Id: I5b06314d7b3d0fb699160ba1326ecb31aedf3082
2017-06-19 20:02:48 +00:00
Hamsalekha S
3f6c941de5 Added error check for output buffer size.
The output buffer size given by the application, needs to be checked
in every process call. This is required in the case of resolution
change.

Bug: 36006815
Test: avcdec -i poc.bin
Change-Id: I16a92cdad23eb7b1e12c1a67c1b2599204f29249
2017-06-16 17:41:23 -07:00
Marco Nelissen
36ffbf2beb Merge "Initializing reference list for every P/B slice." into mnc-dev am: 3477b8e0d7 am: d98f1cf41b am: 18c5ca6984 am: 3cc8a48116 am: 93767dc556 am: 900303e2b0 am: 610062c062 am: 12d32dedae
am: 3b7dde2b72

Change-Id: Ib1a5ed2bf00305861149aa220ac54b65673da4b2
2017-06-06 21:09:49 +00:00
Marco Nelissen
900303e2b0 Merge "Initializing reference list for every P/B slice." into mnc-dev am: 3477b8e0d7 am: d98f1cf41b am: 18c5ca6984 am: 3cc8a48116
am: 93767dc556

Change-Id: I7a1787851643bd77bb509f65d378482bb4f1957c
2017-06-06 20:50:48 +00:00
Marco Nelissen
d98f1cf41b Merge "Initializing reference list for every P/B slice." into mnc-dev
am: 3477b8e0d7

Change-Id: I0ccb9c141c7e7e1b8be6e8e6ddc1c333c10a8662
2017-06-06 20:27:41 +00:00
Marco Nelissen
3477b8e0d7 Merge "Initializing reference list for every P/B slice." into mnc-dev 2017-06-06 20:14:42 +00:00
Hamsalekha S
a7b1d1c722 Merge "Fix resolution change within a decode call." into mnc-dev am: 7fc42b801c am: 252ca18add am: 4c6288e861 am: 90fc697c82 am: 8b3d043280 am: d603f906d2 am: e1b496babb am: 85b8159cf0
am: 6edcc25a5a

Change-Id: I72143bf248487f085dbf816f9b2795c837dbb575
2017-06-06 18:30:46 +00:00
Hamsalekha S
d603f906d2 Merge "Fix resolution change within a decode call." into mnc-dev am: 7fc42b801c am: 252ca18add am: 4c6288e861 am: 90fc697c82
am: 8b3d043280

Change-Id: I3812d5c6cd9ea03959c020ad12ceec69144a282c
2017-06-06 18:05:00 +00:00
Hamsalekha S
252ca18add Merge "Fix resolution change within a decode call." into mnc-dev
am: 7fc42b801c

Change-Id: Ib6f3680beb6e5d837f2140eed3107a23491f99f6
2017-06-06 17:39:59 +00:00
TreeHugger Robot
7fc42b801c Merge "Fix resolution change within a decode call." into mnc-dev 2017-06-06 17:32:41 +00:00
Hamsalekha S
fe18375850 Fix resolution change within a decode call.
If resolution changes within a decode call,due to multiple
sps, the decoder hangs as the the application will
give the same data again in the next decode call. This
results in a hang. Fixed this by flaging an error,
when sps/resoultion changes within a process call.

Bug: 38487564
Test: ran POC on patched O-based system w/o hanging
Change-Id: I30095b2e8bf573c1a58a316a23b1a5e6a4af589b
2017-06-05 13:28:44 -07:00
TreeHugger Robot
0a91df5eca Merge "DO NOT MERGE Decoder: Fixed allocation size of pred info buffer" into mnc-dev 2017-06-02 18:40:21 +00:00
Hamsalekha S
f9d3f9af8f Initializing reference list for every P/B slice.
Reference list needs to be initialized for every P/B
slice, to ensure colocated picture always points to a
valid picture buffer, even in the case of error.

Bug: 36279112

Change-Id: I051d7e725b0af209cc7bb333db8da3518adf78a0
2017-06-02 10:22:52 -07:00
Marco Nelissen
70593599f5 Merge "Decoder: Fix allocation for Mbaff weight matrix" into mnc-dr-dev am: db8f1ae73f am: dcbf996666 am: 8b4c87943a am: 672cb37cc6 am: ad69241f3a am: 7ba9ed2338 am: fed8ee727b
am: 5ed090d7d8

Change-Id: Ie8cd53e71eea8d68e6235f35611f388b415c8154
2017-05-16 15:49:53 +00:00
Marco Nelissen
ad69241f3a Merge "Decoder: Fix allocation for Mbaff weight matrix" into mnc-dr-dev am: db8f1ae73f am: dcbf996666 am: 8b4c87943a
am: 672cb37cc6

Change-Id: Ic3a2832c01bd74ab7a3c788f5e04a6161fed4c78
2017-05-16 15:36:52 +00:00
Marco Nelissen
db8f1ae73f Merge "Decoder: Fix allocation for Mbaff weight matrix" into mnc-dr-dev 2017-05-16 15:22:17 +00:00
Hamsalekha S
c0994c1fea Merge "Decoder: Fixed flag u1_top_bottom_decoded." into mnc-dev am: cc07eaa5d2 am: 9a5c62841e am: 514d08016a am: 4dd1490ec9 am: 122d0942a8 am: 917c7eee1c am: d189b84641 am: 9e191ce3a6
am: ae093b98bc

Change-Id: Ia1f3dd1694f82f24fb7fa006d722c7a9f38a1572
2017-05-12 23:10:02 +00:00