Commit graph

601 commits

Author SHA1 Message Date
Hamsalekha S
94a33985df Decoder: Increased allocation and added checks in sei parsing. am: f2b70d3537 am: 8d32a711ac am: a0827a3bb1 am: cd831159ce am: 2a3efcfcde am: 013d98b763 am: 48d8b5bbc9 am: ccc5298ffb am: a1f8cef8c4
am: b864fbc779

Change-Id: I57274b8d74468c09aa88325a9324c244d7d8b82e
2017-10-31 18:41:52 +00:00
Hamsalekha S
338efffb84 Decoder: Detect change of mbaff flag in SPS am: d524ba0310 am: c30baf4986 am: 16388ca052 am: 7a94552676 am: e6d5d77d9a am: 2e965b33b3 am: 8ffbcc3c8c am: 2b0c57b966
am: cea907a84c

Change-Id: I51fcae11bf485aa17efede32c2145a3dfe974b78
2017-10-31 18:24:03 +00:00
Hamsalekha S
a1f8cef8c4 Decoder: Increased allocation and added checks in sei parsing. am: f2b70d3537 am: 8d32a711ac am: a0827a3bb1 am: cd831159ce am: 2a3efcfcde am: 013d98b763 am: 48d8b5bbc9
am: ccc5298ffb

Change-Id: I83547b11f6da1c33f0ac304690924eacc2ba2920
2017-10-31 18:23:49 +00:00
Hamsalekha S
2e965b33b3 Decoder: Detect change of mbaff flag in SPS am: d524ba0310 am: c30baf4986 am: 16388ca052 am: 7a94552676
am: e6d5d77d9a

Change-Id: I6048ae548b5ad5fe272df8e1853fce6e05215555
2017-10-31 18:11:00 +00:00
Hamsalekha S
2a3efcfcde Decoder: Increased allocation and added checks in sei parsing. am: f2b70d3537 am: 8d32a711ac am: a0827a3bb1
am: cd831159ce

Change-Id: I49ada0ebdcca121f739c753dbedaca8e80ea6da5
2017-10-31 18:10:47 +00:00
Hamsalekha S
c30baf4986 Decoder: Detect change of mbaff flag in SPS
am: d524ba0310

Change-Id: I818230d7fb1763b54d4920f9000663ec72286b6d
2017-10-31 18:02:16 +00:00
Hamsalekha S
d524ba0310 Decoder: Detect change of mbaff flag in SPS
Change in Mbaff flag needs re-initialization of NMB group
and other variables in decoder context.

Bug: 64380237
Test: ran poc on ASAN before/after
Change-Id: I0fc65e4dfc3cc2c15528ec52da1782ecec61feab
2017-10-30 14:19:41 -07:00
Ray Essick
27e6efc8ec DO NOT MERGE Decoder: Increased allocation and added checks in sei parsing.
This prevents heap overflow while parsing sei_message.

Bug: 63122634
Test: ran PoC on unpatched/patched
Change-Id: I4785927b68cb17a3ca51e23aeaf96aacacf116d3
Merged-In: I61c1ff4ac053a060be8c24da4671db985cac628c
2017-10-17 15:09:06 -07:00
Hamsalekha S
f2b70d3537 Decoder: Increased allocation and added checks in sei parsing.
This prevents heap overflow while parsing sei_message.

Bug: 63122634
Test: ran PoC on unpatched/patched
Change-Id: I61c1ff4ac053a060be8c24da4671db985cac628c
2017-10-17 15:03:19 -07:00
Ritu Baldwa
3c70b9a190 Decoder: Fixed incorrect use of mmco parameters.
Added extra structure to read mmco values and copied only once per
picture.

Bug: 65735716

Change-Id: I25b08a37bc78342042c52957774b089abce1a54b
2017-10-12 13:41:43 -07:00
TreeHugger Robot
6a09a35355 Merge "DO NOT MERGE Decoder: Modified setting of error code in set flush api." into mnc-dev 2017-10-03 22:51:37 +00:00
Ritu Baldwa
1541dcc79c DO NOT MERGE Decoder: Modified setting of error code in set flush api.
Fixed incorrect use of ps_dec->pv_dec_out to set error code.

Bug: 66372937
Test: at vendor
Merged-In: Ib04e0b15573b2482c9d5b43c8bc7dd30d8f8efdd
Change-Id: I7b66ee010089399c050a75d6d67feb03da0b8b3e
2017-10-03 14:57:06 -07:00
Hamsalekha S
21aaa6024a Merge "Decoder: Fixed hang in the case of dangling field" into mnc-dev am: 2f1ca945b1 am: 1b1e6d3ec9 am: 8ec4061310 am: 40d1b833d8 am: 73763b165e am: e9ec948685 am: 2411d507d4 am: 5795e09124 am: a55b5dd44f am: 359a2ba846
am: e127b146ff

Change-Id: Ic163faee250695def214290f06c8b5ab432e98eb
2017-10-03 17:57:14 +00:00
Hamsalekha S
359a2ba846 Merge "Decoder: Fixed hang in the case of dangling field" into mnc-dev am: 2f1ca945b1 am: 1b1e6d3ec9 am: 8ec4061310 am: 40d1b833d8 am: 73763b165e am: e9ec948685 am: 2411d507d4 am: 5795e09124
am: a55b5dd44f

Change-Id: I4393ad6185c54c0cf6cd8534e03cbca1d81f5af5
2017-10-03 17:52:12 +00:00
Hamsalekha S
e9ec948685 Merge "Decoder: Fixed hang in the case of dangling field" into mnc-dev am: 2f1ca945b1 am: 1b1e6d3ec9 am: 8ec4061310 am: 40d1b833d8
am: 73763b165e

Change-Id: Ia96b46ff5c254047eeff4dad62a0429a6d75ddf9
2017-10-03 17:43:45 +00:00
Hamsalekha S
1b1e6d3ec9 Merge "Decoder: Fixed hang in the case of dangling field" into mnc-dev
am: 2f1ca945b1

Change-Id: I203d518ad9dc60ea41dfdc88c1903b47398246ba
2017-10-03 17:34:44 +00:00
TreeHugger Robot
2f1ca945b1 Merge "Decoder: Fixed hang in the case of dangling field" into mnc-dev 2017-10-03 17:29:09 +00:00
Ritu Baldwa
ed7a63fbcc DO NOT MERGE Decoder: Increased memory allocation.
ps_dec->u1_recon_mb_grp is twice the width in case of mbaff,
increasing relevant allocations accordingly.

Increased allocation of intra-prediction buffer to
include padding.

Bug: 64964675
Test: POC from bug report
Change-Id: Ic4a6151bb12ac1122c228220b9150b2a372aae21
2017-09-29 11:49:50 -07:00
Hamsalekha S
252628cffb Decoder: Fixed hang in the case of dangling field
The u1_top_bottom_decoded  flag in the decoder context has been fixed
to be updated correctly in the case of dangling field

Bug: 63315932
Test: ran POC after patching
Change-Id: I8db4ebeb94fba735ba45f365c37e52a202ea84cd
2017-09-29 10:25:21 -07:00
Marco Nelissen
a30d819481 Merge "Decoder: Updated error check while parsing num_ref_idx_lx_active." into mnc-dev am: 224824ba17 am: 628056c0ed am: becbdb15b7 am: 2f85a749c5 am: bd6a74b45b am: 828dc44ed5 am: a1d4d39a61 am: b4f8a983ad am: d3ea876a4d am: 957708e9b1
am: 0c909b99f0

Change-Id: Ibcbbc2bd0475ddbac9a743e4f9d3fd44b25c1c64
2017-09-14 01:10:16 +00:00
Marco Nelissen
957708e9b1 Merge "Decoder: Updated error check while parsing num_ref_idx_lx_active." into mnc-dev am: 224824ba17 am: 628056c0ed am: becbdb15b7 am: 2f85a749c5 am: bd6a74b45b am: 828dc44ed5 am: a1d4d39a61 am: b4f8a983ad
am: d3ea876a4d

Change-Id: I2c6eb56ae9a8e094c9c7513527e07e75a2e22c71
2017-09-13 23:21:34 +00:00
Marco Nelissen
828dc44ed5 Merge "Decoder: Updated error check while parsing num_ref_idx_lx_active." into mnc-dev am: 224824ba17 am: 628056c0ed am: becbdb15b7 am: 2f85a749c5
am: bd6a74b45b

Change-Id: Ia3024ff02b981807fc0437a9611de72c92ea8c7d
2017-09-13 20:06:27 +00:00
Marco Nelissen
628056c0ed Merge "Decoder: Updated error check while parsing num_ref_idx_lx_active." into mnc-dev
am: 224824ba17

Change-Id: Id5366922a8312c3fc99f5dc42c64e4bfdf11795b
2017-09-13 05:51:32 +00:00
Marco Nelissen
224824ba17 Merge "Decoder: Updated error check while parsing num_ref_idx_lx_active." into mnc-dev 2017-09-13 05:00:15 +00:00
Hamsalekha S
2463d42f6f Merge "Decoder: Corrected variable datatypes in ih264d_get_implicit_weights." into mnc-dev am: a2d8a1444d am: 8659d0e563 am: 0af8672874 am: 30468b264e am: dd338e085c am: dc56ebdb83 am: e42cbf31fd am: 0bd2cd5f1c am: 9dc4a57a87 am: 16b7ec0491
am: 305bac508f

Change-Id: Ibe60a4e8b1db28ddad94f7fb71cbe3fd7873b6a9
2017-09-12 01:25:06 +00:00
Hamsalekha S
16b7ec0491 Merge "Decoder: Corrected variable datatypes in ih264d_get_implicit_weights." into mnc-dev am: a2d8a1444d am: 8659d0e563 am: 0af8672874 am: 30468b264e am: dd338e085c am: dc56ebdb83 am: e42cbf31fd am: 0bd2cd5f1c
am: 9dc4a57a87

Change-Id: Ib073bbec0588fb1d985eda19e07f98b821b7393f
2017-09-12 00:51:53 +00:00
Hamsalekha S
dc56ebdb83 Merge "Decoder: Corrected variable datatypes in ih264d_get_implicit_weights." into mnc-dev am: a2d8a1444d am: 8659d0e563 am: 0af8672874 am: 30468b264e
am: dd338e085c

Change-Id: Ib77127995a96a95fcc67d1817af3dda211d6f10f
2017-09-12 00:24:23 +00:00
Hamsalekha S
8659d0e563 Merge "Decoder: Corrected variable datatypes in ih264d_get_implicit_weights." into mnc-dev
am: a2d8a1444d

Change-Id: I50d88e962f2435189cbb4720d8254729eb47829f
2017-09-12 00:14:49 +00:00
TreeHugger Robot
a2d8a1444d Merge "Decoder: Corrected variable datatypes in ih264d_get_implicit_weights." into mnc-dev 2017-09-12 00:11:33 +00:00
Harish Mahendrakar
e5c2c33295 Added an out of bound check on u4_num_bufs in input argument am: aa11ab9fdb am: 50ec420228 am: 0514a1ebfb am: fa8bb9d89c am: 96228ba3b9 am: 4a6a743d40 am: 4ed04c7bb1 am: b9027e8cc1 am: 859498bb08 am: 31b397c813
am: 8564f06db6

Change-Id: I8e572678c2d456b7458793449cbf7e72e6b63b9b
2017-09-11 22:43:26 +00:00
Hamsalekha S
ba77e93be7 Decoder: Conceal picture only if valid picture buffer is obtained. am: 5df744afde am: 7f78e3aa9a am: 8810e9db7f am: 1ab3e51baf am: 32c369cc3f am: fd327a47fb am: b10090d47a am: 2825c578d3 am: 1d6b06a73a
am: 7f049a7472

Change-Id: I5a44be13778eb16c0300d13e5dc1c32d7e0c506a
2017-09-11 22:43:13 +00:00
Harish Mahendrakar
31b397c813 Added an out of bound check on u4_num_bufs in input argument am: aa11ab9fdb am: 50ec420228 am: 0514a1ebfb am: fa8bb9d89c am: 96228ba3b9 am: 4a6a743d40 am: 4ed04c7bb1 am: b9027e8cc1
am: 859498bb08

Change-Id: I364cdcdd56d88f411892723935683d16ffcd0321
2017-09-11 22:39:38 +00:00
Hamsalekha S
1d6b06a73a Decoder: Conceal picture only if valid picture buffer is obtained. am: 5df744afde am: 7f78e3aa9a am: 8810e9db7f am: 1ab3e51baf am: 32c369cc3f am: fd327a47fb am: b10090d47a
am: 2825c578d3

Change-Id: I1bf0cb0c96c45b419c06bd9b1d202b1ff08193e4
2017-09-11 22:39:14 +00:00
Harish Mahendrakar
4a6a743d40 Added an out of bound check on u4_num_bufs in input argument am: aa11ab9fdb am: 50ec420228 am: 0514a1ebfb am: fa8bb9d89c
am: 96228ba3b9

Change-Id: Ic048f96697bdaedea3bbc0a8cf1acf78ed488e37
2017-09-11 22:30:10 +00:00
Hamsalekha S
32c369cc3f Decoder: Conceal picture only if valid picture buffer is obtained. am: 5df744afde am: 7f78e3aa9a am: 8810e9db7f
am: 1ab3e51baf

Change-Id: I964f1bd1fc0d9534d6c9498864cc7831d13de449
2017-09-11 22:28:11 +00:00
Harish Mahendrakar
50ec420228 Added an out of bound check on u4_num_bufs in input argument
am: aa11ab9fdb

Change-Id: I7dd6739821da87a96072dae422416e8037332a4f
2017-09-11 22:22:40 +00:00
Ray Essick
dc9396971d Merge "DO NOT MERGE: Decoder: Conceal picture only if valid picture buffer is obtained." into mnc-dev 2017-09-11 22:17:06 +00:00
Hamsalekha S
208c74d62a Decoder: Updated error check while parsing num_ref_idx_lx_active.
Added an error check on the lower limit of u1_num_ref_idx_lx_active,
while parsing slice header. The minimum possible value is 1.

Bug: 64836894

Change-Id: I57056851fc135ed00f7a10af5c81eb560e9e12de
2017-09-11 20:16:47 +00:00
Hamsalekha S
3eb692de91 Decoder: Corrected variable datatypes in ih264d_get_implicit_weights.
The difference between two 32 signed numbers was getting assigned
to 16 bits, leading to a divide by zero arithmetic execption.
Modified variable names to match their datatypes.

Bug: 65122447

Change-Id: I45ade1945f10b4d7660bd09fb564e60fd29d40dc
2017-09-08 13:21:14 -07:00
Lajos Molnar
09c8cb1b30 support 4080 max dimension
4096 is 256 macroblocks which does not fit on 8-bits, and 8-bits
are used for some variables that contain the size.

Bug: 65438670
Change-Id: Id977303054841c458cbd21c1538a9035d178fbef
2017-09-08 12:42:40 -07:00
Ray Essick
45aa4ac86d DO NOT MERGE: Decoder: Conceal picture only if valid picture buffer is obtained.
If all the slices in the current pic were invalid, then
the decoder would not have received a valid picture buffer
in the current call. In such cases there is no need to conceal or
deblock the picture.

Bug: 62896384
Test: run ASAN-enabled PoC before/after the patch
Change-Id: Ia7d979a78ae3f4fb443a1759c8e6cf8780565ad8
Merged-In: I3cf6e871592826f93b0dcd2b06fff80677bc8338
2017-09-06 23:46:21 +00:00
Lajos Molnar
8a7f15c3d4 support 4K max dimension
This is required for handling 4K thumbnails that may be rotated.

Bug: 64939294
Change-Id: Ia33d8e1ce7fb07b5ec6bf604d30cd8f75c3228ae
2017-08-24 19:07:58 -07:00
Hamsalekha S
5df744afde Decoder: Conceal picture only if valid picture buffer is obtained.
If all the slices in the current pic were invalid, then
the decoder would not have received a valid picture buffer
in the current call. In such cases there is no need to conceal or
deblock the picture.

Bug: 62896384
Test: run ASAN-enabled PoC before/after the patch
Change-Id: I3cf6e871592826f93b0dcd2b06fff80677bc8338
2017-08-18 20:45:17 +00:00
Harish Mahendrakar
aa11ab9fdb Added an out of bound check on u4_num_bufs in input argument
ps_dec_ip->s_out_buffer.u4_num_bufs was missing out of bound checks

Bug: 62688399
Change-Id: Ic5e5c002d29fcb18064550d5a5f9289bb68b448e
2017-08-10 10:15:16 -07:00
Hamsalekha S
f96761b52d Decoder: Fixed allocation of pv_map_ref_idx_to_poc_buf. am: a925a6b539 am: bf8b7083e9 am: ebae2ef7bd am: d5ad4f8c04 am: 6f82dff7b9 am: 0716eeeb7f am: 0b64261f4c am: d2e5daaede am: 61ddc9e202 am: 6d73354309
am: 79b56d80d5

Change-Id: Ic7cfdc087bf9c3de52c5d6cf9a25dfe25db4a936
2017-07-15 01:16:34 +00:00
Hamsalekha S
61ddc9e202 Decoder: Fixed allocation of pv_map_ref_idx_to_poc_buf. am: a925a6b539 am: bf8b7083e9 am: ebae2ef7bd am: d5ad4f8c04 am: 6f82dff7b9 am: 0716eeeb7f am: 0b64261f4c
am: d2e5daaede

Change-Id: Ic1a04ff426ec654e28d142ced625011b7ff9fbe1
2017-07-15 01:04:04 +00:00
Hamsalekha S
6f82dff7b9 Decoder: Fixed allocation of pv_map_ref_idx_to_poc_buf. am: a925a6b539 am: bf8b7083e9 am: ebae2ef7bd
am: d5ad4f8c04

Change-Id: I8c096a96d86379e40d02c445e95044055b124d73
2017-07-15 00:48:02 +00:00
Hamsalekha S
a925a6b539 Decoder: Fixed allocation of pv_map_ref_idx_to_poc_buf.
Increased allocation to include reference list1 also by
default. In the case of error, we could get B_SLICES
even in BASE_PROFILE. The initialization in the
dec_slice_struct_t slice structure has also been
modified accordingly.

Test: run poc with and without this patch
Bug: 38496660
Change-Id: I3451d79bbcd9f0d7a80981a9897f877b7f0812bd
2017-07-13 17:49:14 +00:00
Ray Essick
707287f248 Merge "Decoder : Fix memory leak when sps changes." 2017-07-11 04:03:40 +00:00
Hamsalekha S
674fb1aa06 Decoder: Fixed overflow in refernce list creation. am: cbcd2846fa am: a54dcc6338 am: 136293c373 am: 28702336ea am: d4d96c6267 am: eaf6bed710 am: 400685c446 am: 4192026584
am: d00e2547c9

Change-Id: I2939be93a829b4ede24a1285fd337109e2f388d2
2017-07-10 19:25:26 +00:00