Commit graph

100 commits

Author SHA1 Message Date
Ray Essick
7adc4d57f1 Merge "Decoder: Adding Error Check for Output Buffer Size in Shared Display Mode." into mnc-dev am: afc729db25 am: 1fb89d8488 am: 56f836a29c am: 33675f68d6
am: 547804d647

Change-Id: I22c555cc975abbfa80aa38810c9e7a0f59ce3450
2018-01-08 22:41:22 +00:00
Ray Essick
1fb89d8488 Merge "Decoder: Adding Error Check for Output Buffer Size in Shared Display Mode." into mnc-dev
am: afc729db25

Change-Id: Ic57cdca65a505c45bec9c6ff8dd9b5d9dab5fec7
2018-01-08 22:27:38 +00:00
Ritu Baldwa
3692aceb1b Decoder: Adding Error Check for Output Buffer Size in Shared Display Mode.
The output buffer size given by the application, needs to be checked
in every process call. This is required in the case of resolution
change in shared display mode.

Bug: 70294343
Bug: 70350193
Bug: 70526411
Bug: 70526485
Test: manual
Change-Id: I2c1e59425e84ac62a874e5ee180e1b98f0a4058f
2018-01-02 14:25:27 -08:00
Harish Mahendrakar
18ae83b015 Merge "Decoder: Handle dec_hdl memory allocation failure gracefully" into mnc-dr-dev am: ac0d9cecb5 am: f63836fed1 am: 5448bcea3e
am: f87d4848af

Change-Id: Ia060e78fe7cd7b4233733910f08caadbea05ff3d
2017-11-03 19:07:04 +00:00
TreeHugger Robot
ac0d9cecb5 Merge "Decoder: Handle dec_hdl memory allocation failure gracefully" into mnc-dr-dev 2017-11-03 18:57:47 +00:00
Harish Mahendrakar
7720b3fe3d Decoder: Handle dec_hdl memory allocation failure gracefully
If memory allocation for dec_hdl fails, return gracefully
with an error code. All other allocation failures are
handled correctly.

Bug: 68300072
Test: ran poc before/after
Change-Id: I118ae71f4aded658441f1932bd4ede3536f5028b
2017-11-01 09:57:48 -07:00
Hamsalekha S
2a3efcfcde Decoder: Increased allocation and added checks in sei parsing. am: f2b70d3537 am: 8d32a711ac am: a0827a3bb1
am: cd831159ce

Change-Id: I49ada0ebdcca121f739c753dbedaca8e80ea6da5
2017-10-31 18:10:47 +00:00
Ray Essick
27e6efc8ec DO NOT MERGE Decoder: Increased allocation and added checks in sei parsing.
This prevents heap overflow while parsing sei_message.

Bug: 63122634
Test: ran PoC on unpatched/patched
Change-Id: I4785927b68cb17a3ca51e23aeaf96aacacf116d3
Merged-In: I61c1ff4ac053a060be8c24da4671db985cac628c
2017-10-17 15:09:06 -07:00
Hamsalekha S
f2b70d3537 Decoder: Increased allocation and added checks in sei parsing.
This prevents heap overflow while parsing sei_message.

Bug: 63122634
Test: ran PoC on unpatched/patched
Change-Id: I61c1ff4ac053a060be8c24da4671db985cac628c
2017-10-17 15:03:19 -07:00
TreeHugger Robot
6a09a35355 Merge "DO NOT MERGE Decoder: Modified setting of error code in set flush api." into mnc-dev 2017-10-03 22:51:37 +00:00
Ritu Baldwa
1541dcc79c DO NOT MERGE Decoder: Modified setting of error code in set flush api.
Fixed incorrect use of ps_dec->pv_dec_out to set error code.

Bug: 66372937
Test: at vendor
Merged-In: Ib04e0b15573b2482c9d5b43c8bc7dd30d8f8efdd
Change-Id: I7b66ee010089399c050a75d6d67feb03da0b8b3e
2017-10-03 14:57:06 -07:00
Ritu Baldwa
ed7a63fbcc DO NOT MERGE Decoder: Increased memory allocation.
ps_dec->u1_recon_mb_grp is twice the width in case of mbaff,
increasing relevant allocations accordingly.

Increased allocation of intra-prediction buffer to
include padding.

Bug: 64964675
Test: POC from bug report
Change-Id: Ic4a6151bb12ac1122c228220b9150b2a372aae21
2017-09-29 11:49:50 -07:00
Harish Mahendrakar
4a6a743d40 Added an out of bound check on u4_num_bufs in input argument am: aa11ab9fdb am: 50ec420228 am: 0514a1ebfb am: fa8bb9d89c
am: 96228ba3b9

Change-Id: Ic048f96697bdaedea3bbc0a8cf1acf78ed488e37
2017-09-11 22:30:10 +00:00
Hamsalekha S
32c369cc3f Decoder: Conceal picture only if valid picture buffer is obtained. am: 5df744afde am: 7f78e3aa9a am: 8810e9db7f
am: 1ab3e51baf

Change-Id: I964f1bd1fc0d9534d6c9498864cc7831d13de449
2017-09-11 22:28:11 +00:00
Harish Mahendrakar
50ec420228 Added an out of bound check on u4_num_bufs in input argument
am: aa11ab9fdb

Change-Id: I7dd6739821da87a96072dae422416e8037332a4f
2017-09-11 22:22:40 +00:00
Ray Essick
dc9396971d Merge "DO NOT MERGE: Decoder: Conceal picture only if valid picture buffer is obtained." into mnc-dev 2017-09-11 22:17:06 +00:00
Ray Essick
45aa4ac86d DO NOT MERGE: Decoder: Conceal picture only if valid picture buffer is obtained.
If all the slices in the current pic were invalid, then
the decoder would not have received a valid picture buffer
in the current call. In such cases there is no need to conceal or
deblock the picture.

Bug: 62896384
Test: run ASAN-enabled PoC before/after the patch
Change-Id: Ia7d979a78ae3f4fb443a1759c8e6cf8780565ad8
Merged-In: I3cf6e871592826f93b0dcd2b06fff80677bc8338
2017-09-06 23:46:21 +00:00
Hamsalekha S
5df744afde Decoder: Conceal picture only if valid picture buffer is obtained.
If all the slices in the current pic were invalid, then
the decoder would not have received a valid picture buffer
in the current call. In such cases there is no need to conceal or
deblock the picture.

Bug: 62896384
Test: run ASAN-enabled PoC before/after the patch
Change-Id: I3cf6e871592826f93b0dcd2b06fff80677bc8338
2017-08-18 20:45:17 +00:00
Harish Mahendrakar
aa11ab9fdb Added an out of bound check on u4_num_bufs in input argument
ps_dec_ip->s_out_buffer.u4_num_bufs was missing out of bound checks

Bug: 62688399
Change-Id: Ic5e5c002d29fcb18064550d5a5f9289bb68b448e
2017-08-10 10:15:16 -07:00
Hamsalekha S
564bdc7609 Added error check for output buffer size. am: 3f6c941de5 am: f0d2caa214 am: 9a6312ec9f am: 83c397d1ab
am: 462316f8d2

Change-Id: Id9021bcb9534cd9b30063eb9be6de464014c67d7
2017-06-19 20:26:20 +00:00
Hamsalekha S
f0d2caa214 Added error check for output buffer size.
am: 3f6c941de5

Change-Id: I5b06314d7b3d0fb699160ba1326ecb31aedf3082
2017-06-19 20:02:48 +00:00
Hamsalekha S
3f6c941de5 Added error check for output buffer size.
The output buffer size given by the application, needs to be checked
in every process call. This is required in the case of resolution
change.

Bug: 36006815
Test: avcdec -i poc.bin
Change-Id: I16a92cdad23eb7b1e12c1a67c1b2599204f29249
2017-06-16 17:41:23 -07:00
Marco Nelissen
900303e2b0 Merge "Initializing reference list for every P/B slice." into mnc-dev am: 3477b8e0d7 am: d98f1cf41b am: 18c5ca6984 am: 3cc8a48116
am: 93767dc556

Change-Id: I7a1787851643bd77bb509f65d378482bb4f1957c
2017-06-06 20:50:48 +00:00
Marco Nelissen
d98f1cf41b Merge "Initializing reference list for every P/B slice." into mnc-dev
am: 3477b8e0d7

Change-Id: I0ccb9c141c7e7e1b8be6e8e6ddc1c333c10a8662
2017-06-06 20:27:41 +00:00
Marco Nelissen
3477b8e0d7 Merge "Initializing reference list for every P/B slice." into mnc-dev 2017-06-06 20:14:42 +00:00
Hamsalekha S
d603f906d2 Merge "Fix resolution change within a decode call." into mnc-dev am: 7fc42b801c am: 252ca18add am: 4c6288e861 am: 90fc697c82
am: 8b3d043280

Change-Id: I3812d5c6cd9ea03959c020ad12ceec69144a282c
2017-06-06 18:05:00 +00:00
Hamsalekha S
252ca18add Merge "Fix resolution change within a decode call." into mnc-dev
am: 7fc42b801c

Change-Id: Ib6f3680beb6e5d837f2140eed3107a23491f99f6
2017-06-06 17:39:59 +00:00
TreeHugger Robot
7fc42b801c Merge "Fix resolution change within a decode call." into mnc-dev 2017-06-06 17:32:41 +00:00
Hamsalekha S
fe18375850 Fix resolution change within a decode call.
If resolution changes within a decode call,due to multiple
sps, the decoder hangs as the the application will
give the same data again in the next decode call. This
results in a hang. Fixed this by flaging an error,
when sps/resoultion changes within a process call.

Bug: 38487564
Test: ran POC on patched O-based system w/o hanging
Change-Id: I30095b2e8bf573c1a58a316a23b1a5e6a4af589b
2017-06-05 13:28:44 -07:00
TreeHugger Robot
0a91df5eca Merge "DO NOT MERGE Decoder: Fixed allocation size of pred info buffer" into mnc-dev 2017-06-02 18:40:21 +00:00
Hamsalekha S
f9d3f9af8f Initializing reference list for every P/B slice.
Reference list needs to be initialized for every P/B
slice, to ensure colocated picture always points to a
valid picture buffer, even in the case of error.

Bug: 36279112

Change-Id: I051d7e725b0af209cc7bb333db8da3518adf78a0
2017-06-02 10:22:52 -07:00
Marco Nelissen
ad69241f3a Merge "Decoder: Fix allocation for Mbaff weight matrix" into mnc-dr-dev am: db8f1ae73f am: dcbf996666 am: 8b4c87943a
am: 672cb37cc6

Change-Id: Ic3a2832c01bd74ab7a3c788f5e04a6161fed4c78
2017-05-16 15:36:52 +00:00
Marco Nelissen
db8f1ae73f Merge "Decoder: Fix allocation for Mbaff weight matrix" into mnc-dr-dev 2017-05-16 15:22:17 +00:00
Hamsalekha S
917c7eee1c Merge "Decoder: Fixed flag u1_top_bottom_decoded." into mnc-dev am: cc07eaa5d2 am: 9a5c62841e am: 514d08016a am: 4dd1490ec9
am: 122d0942a8

Change-Id: I53408f560fed7e2a5d88d5a3e3e5f77f01deb41d
2017-05-12 23:02:32 +00:00
Hamsalekha S
9a5c62841e Merge "Decoder: Fixed flag u1_top_bottom_decoded." into mnc-dev
am: cc07eaa5d2

Change-Id: Ie410da29e20aed95d674aaaeddd683e1f616c448
2017-05-12 22:51:00 +00:00
Hamsalekha S
07db35ad5a Decoder: Fix allocation for Mbaff weight matrix
Increased the allocation size for Mbaff weight
matrix buffer

Bug: 36996978

Change-Id: I21cf2cb1010abdc6346f743f5237ae1730c4bf41
2017-05-12 14:11:10 -07:00
Hamsalekha S
7703822731 Decoder: Fixed flag u1_top_bottom_decoded.
Fixed initialization of flag u1_top_bottom_decoded
in decoder context. This flag indicates if top
field and botton field is decoded.

Bug: 36993291
Test: avcdec --input poc.h264 --output /dev/null
Change-Id: I9f8a2620683abd8b15e4780d76d4849394710716
2017-05-12 20:14:46 +00:00
Hamsalekha S
6a5cb80068 DO NOT MERGE Decoder: Fixed allocation size of pred info buffer
Buffer allocation size for pred info was increased
in the case number reference frames equal to 1.

Bug: 36998372
Change-Id: I1f84a16703422109d40bed8436f35d0c2069c088
2017-05-11 15:09:57 -07:00
Harish Mahendrakar
94e17dc822 Decoder: Fixed initialization of first_slice_in_pic am: 0b23c81c3d am: 16fa990ce1 am: ab79eb3973 am: a37d120eb2
am: c011eeade7

Change-Id: I3f850497c864277c2b2524fab07d14a249e48afc
2017-02-14 17:11:38 +00:00
Harish Mahendrakar
7944003371 Decoder: Moved end of pic processing to end of decode call am: 494561291a am: ba7f9e2aed am: 26bb8b98ee am: 89fa016b5f
am: df2fa01c9e

Change-Id: Ifc329cbccad01c70ff1fea0d483cb155b15b3cc1
2017-02-14 17:08:45 +00:00
Harish Mahendrakar
16fa990ce1 Decoder: Fixed initialization of first_slice_in_pic
am: 0b23c81c3d

Change-Id: I55e49ca9616a151456342bad7673d0986098b448
2017-02-14 17:03:06 +00:00
Harish Mahendrakar
ba7f9e2aed Decoder: Moved end of pic processing to end of decode call
am: 494561291a

Change-Id: Id3867f5dd0788f12f6827ed311a309d1fa58c535
2017-02-14 17:00:16 +00:00
Harish Mahendrakar
0b23c81c3d Decoder: Fixed initialization of first_slice_in_pic
To handle some errors, first_slice_in_pic was being set to 2.
This is now cleaned up and first_slice_in_pic is set to 1 only once per pic.
This will ensure picture level initializations are done only once even in case
of error clips

Bug: 33717589
Bug: 33551775
Bug: 33716442
Bug: 33677995

Change-Id: If341436b3cbaa724017eedddd88c2e6fac36d8ba
2017-02-14 16:59:09 +00:00
Harish Mahendrakar
494561291a Decoder: Moved end of pic processing to end of decode call
ih264d_end_of_pic() was called after parsing slice of a new picture.
This is now being done at the end of decode of the current picture.
decode_gaps_in_frame_num which needs frame_num of new slice is now
done after decoding frame_num in new slice.

This helps in handling errors in picaff streams with gaps in frames

Bug: 33588051
Bug: 33641588
Bug: 34097231
Change-Id: I1a26e611aaa2c19e2043e05a210849bd21b22220
2017-02-14 16:58:07 +00:00
Harish Mahendrakar
1f0ab0b089 DO NOT MERGE: Decoder: Fixes an out of bound write in bitstream buffer
[this is for mnc-dev only, not any other mnc-*-dev flavors;
there is a different patch for mnc-dr-dev, nyc-* and going forward]

After emulation prevention, data is written as an int,
so at least 3 additional bytes should be available.
And since bitstream functions read 8 bytes ahead, 8 extra bytes
should be available in the bitstream buffer.

Bug: 33934721
Test: Ittiam testing, POC in the bug no longer fails
Change-Id: I444ec6f85d01b0bade9f827e15c4b476779d6c69
2017-02-13 14:44:11 -08:00
Harish Mahendrakar
89377f2acf Decoder: Fixes an out of bound write in bitstream buffer am: 33ef7de9dd am: 0aab4958fb am: 9f72b6eca1
am: 73a3e320d8

Change-Id: Id06cfbad9824a7921e4eb082417f14aa842bcfab
2017-02-13 18:52:16 +00:00
Harish Mahendrakar
33ef7de9dd Decoder: Fixes an out of bound write in bitstream buffer
[for mnc-dr-dev and later;  mnc-dev gets a different patch]

After emulation prevention, data is written as an int,
so at least 3 additional bytes should be available.
And since bitstream functions read 8 bytes ahead, 8 extra bytes
should be available in the bitstream buffer.

Bug: 33934721

Change-Id: I444ec6f85d01b0bade9f827e15c4b476779d6c69
2017-01-19 09:17:00 -08:00
Marco Nelissen
d5424f66e3 Merge "DO NOT MERGE Decoder: Increase memory allocation for weights & offsets for interlaced clips" into mnc-dev 2017-01-18 18:42:47 +00:00
Harish Mahendrakar
3f1af4a90c Decoder: Increase memory allocation for weights & offsets for interlaced clips am: bee9b9a54b am: bfc735cf17 am: 87c2d690fa
am: 826159785f

Change-Id: Iba990d241d8eb893356b748812342bab4ffecae2
2017-01-18 16:59:45 +00:00
Harish Mahendrakar
bfc735cf17 Decoder: Increase memory allocation for weights & offsets for interlaced clips
am: bee9b9a54b

Change-Id: I8c0bd3d77b87e6fb3f982ef70854b8ec73298636
2017-01-18 16:41:13 +00:00