Commit graph

78 commits

Author SHA1 Message Date
Ray Essick
1fb89d8488 Merge "Decoder: Adding Error Check for Output Buffer Size in Shared Display Mode." into mnc-dev
am: afc729db25

Change-Id: Ic57cdca65a505c45bec9c6ff8dd9b5d9dab5fec7
2018-01-08 22:27:38 +00:00
Ritu Baldwa
3692aceb1b Decoder: Adding Error Check for Output Buffer Size in Shared Display Mode.
The output buffer size given by the application, needs to be checked
in every process call. This is required in the case of resolution
change in shared display mode.

Bug: 70294343
Bug: 70350193
Bug: 70526411
Bug: 70526485
Test: manual
Change-Id: I2c1e59425e84ac62a874e5ee180e1b98f0a4058f
2018-01-02 14:25:27 -08:00
TreeHugger Robot
ac0d9cecb5 Merge "Decoder: Handle dec_hdl memory allocation failure gracefully" into mnc-dr-dev 2017-11-03 18:57:47 +00:00
Harish Mahendrakar
7720b3fe3d Decoder: Handle dec_hdl memory allocation failure gracefully
If memory allocation for dec_hdl fails, return gracefully
with an error code. All other allocation failures are
handled correctly.

Bug: 68300072
Test: ran poc before/after
Change-Id: I118ae71f4aded658441f1932bd4ede3536f5028b
2017-11-01 09:57:48 -07:00
Ray Essick
27e6efc8ec DO NOT MERGE Decoder: Increased allocation and added checks in sei parsing.
This prevents heap overflow while parsing sei_message.

Bug: 63122634
Test: ran PoC on unpatched/patched
Change-Id: I4785927b68cb17a3ca51e23aeaf96aacacf116d3
Merged-In: I61c1ff4ac053a060be8c24da4671db985cac628c
2017-10-17 15:09:06 -07:00
Hamsalekha S
f2b70d3537 Decoder: Increased allocation and added checks in sei parsing.
This prevents heap overflow while parsing sei_message.

Bug: 63122634
Test: ran PoC on unpatched/patched
Change-Id: I61c1ff4ac053a060be8c24da4671db985cac628c
2017-10-17 15:03:19 -07:00
TreeHugger Robot
6a09a35355 Merge "DO NOT MERGE Decoder: Modified setting of error code in set flush api." into mnc-dev 2017-10-03 22:51:37 +00:00
Ritu Baldwa
1541dcc79c DO NOT MERGE Decoder: Modified setting of error code in set flush api.
Fixed incorrect use of ps_dec->pv_dec_out to set error code.

Bug: 66372937
Test: at vendor
Merged-In: Ib04e0b15573b2482c9d5b43c8bc7dd30d8f8efdd
Change-Id: I7b66ee010089399c050a75d6d67feb03da0b8b3e
2017-10-03 14:57:06 -07:00
Ritu Baldwa
ed7a63fbcc DO NOT MERGE Decoder: Increased memory allocation.
ps_dec->u1_recon_mb_grp is twice the width in case of mbaff,
increasing relevant allocations accordingly.

Increased allocation of intra-prediction buffer to
include padding.

Bug: 64964675
Test: POC from bug report
Change-Id: Ic4a6151bb12ac1122c228220b9150b2a372aae21
2017-09-29 11:49:50 -07:00
Harish Mahendrakar
50ec420228 Added an out of bound check on u4_num_bufs in input argument
am: aa11ab9fdb

Change-Id: I7dd6739821da87a96072dae422416e8037332a4f
2017-09-11 22:22:40 +00:00
Ray Essick
dc9396971d Merge "DO NOT MERGE: Decoder: Conceal picture only if valid picture buffer is obtained." into mnc-dev 2017-09-11 22:17:06 +00:00
Ray Essick
45aa4ac86d DO NOT MERGE: Decoder: Conceal picture only if valid picture buffer is obtained.
If all the slices in the current pic were invalid, then
the decoder would not have received a valid picture buffer
in the current call. In such cases there is no need to conceal or
deblock the picture.

Bug: 62896384
Test: run ASAN-enabled PoC before/after the patch
Change-Id: Ia7d979a78ae3f4fb443a1759c8e6cf8780565ad8
Merged-In: I3cf6e871592826f93b0dcd2b06fff80677bc8338
2017-09-06 23:46:21 +00:00
Hamsalekha S
5df744afde Decoder: Conceal picture only if valid picture buffer is obtained.
If all the slices in the current pic were invalid, then
the decoder would not have received a valid picture buffer
in the current call. In such cases there is no need to conceal or
deblock the picture.

Bug: 62896384
Test: run ASAN-enabled PoC before/after the patch
Change-Id: I3cf6e871592826f93b0dcd2b06fff80677bc8338
2017-08-18 20:45:17 +00:00
Harish Mahendrakar
aa11ab9fdb Added an out of bound check on u4_num_bufs in input argument
ps_dec_ip->s_out_buffer.u4_num_bufs was missing out of bound checks

Bug: 62688399
Change-Id: Ic5e5c002d29fcb18064550d5a5f9289bb68b448e
2017-08-10 10:15:16 -07:00
Hamsalekha S
f0d2caa214 Added error check for output buffer size.
am: 3f6c941de5

Change-Id: I5b06314d7b3d0fb699160ba1326ecb31aedf3082
2017-06-19 20:02:48 +00:00
Hamsalekha S
3f6c941de5 Added error check for output buffer size.
The output buffer size given by the application, needs to be checked
in every process call. This is required in the case of resolution
change.

Bug: 36006815
Test: avcdec -i poc.bin
Change-Id: I16a92cdad23eb7b1e12c1a67c1b2599204f29249
2017-06-16 17:41:23 -07:00
Marco Nelissen
d98f1cf41b Merge "Initializing reference list for every P/B slice." into mnc-dev
am: 3477b8e0d7

Change-Id: I0ccb9c141c7e7e1b8be6e8e6ddc1c333c10a8662
2017-06-06 20:27:41 +00:00
Marco Nelissen
3477b8e0d7 Merge "Initializing reference list for every P/B slice." into mnc-dev 2017-06-06 20:14:42 +00:00
Hamsalekha S
252ca18add Merge "Fix resolution change within a decode call." into mnc-dev
am: 7fc42b801c

Change-Id: Ib6f3680beb6e5d837f2140eed3107a23491f99f6
2017-06-06 17:39:59 +00:00
TreeHugger Robot
7fc42b801c Merge "Fix resolution change within a decode call." into mnc-dev 2017-06-06 17:32:41 +00:00
Hamsalekha S
fe18375850 Fix resolution change within a decode call.
If resolution changes within a decode call,due to multiple
sps, the decoder hangs as the the application will
give the same data again in the next decode call. This
results in a hang. Fixed this by flaging an error,
when sps/resoultion changes within a process call.

Bug: 38487564
Test: ran POC on patched O-based system w/o hanging
Change-Id: I30095b2e8bf573c1a58a316a23b1a5e6a4af589b
2017-06-05 13:28:44 -07:00
TreeHugger Robot
0a91df5eca Merge "DO NOT MERGE Decoder: Fixed allocation size of pred info buffer" into mnc-dev 2017-06-02 18:40:21 +00:00
Hamsalekha S
f9d3f9af8f Initializing reference list for every P/B slice.
Reference list needs to be initialized for every P/B
slice, to ensure colocated picture always points to a
valid picture buffer, even in the case of error.

Bug: 36279112

Change-Id: I051d7e725b0af209cc7bb333db8da3518adf78a0
2017-06-02 10:22:52 -07:00
Marco Nelissen
db8f1ae73f Merge "Decoder: Fix allocation for Mbaff weight matrix" into mnc-dr-dev 2017-05-16 15:22:17 +00:00
Hamsalekha S
9a5c62841e Merge "Decoder: Fixed flag u1_top_bottom_decoded." into mnc-dev
am: cc07eaa5d2

Change-Id: Ie410da29e20aed95d674aaaeddd683e1f616c448
2017-05-12 22:51:00 +00:00
Hamsalekha S
07db35ad5a Decoder: Fix allocation for Mbaff weight matrix
Increased the allocation size for Mbaff weight
matrix buffer

Bug: 36996978

Change-Id: I21cf2cb1010abdc6346f743f5237ae1730c4bf41
2017-05-12 14:11:10 -07:00
Hamsalekha S
7703822731 Decoder: Fixed flag u1_top_bottom_decoded.
Fixed initialization of flag u1_top_bottom_decoded
in decoder context. This flag indicates if top
field and botton field is decoded.

Bug: 36993291
Test: avcdec --input poc.h264 --output /dev/null
Change-Id: I9f8a2620683abd8b15e4780d76d4849394710716
2017-05-12 20:14:46 +00:00
Hamsalekha S
6a5cb80068 DO NOT MERGE Decoder: Fixed allocation size of pred info buffer
Buffer allocation size for pred info was increased
in the case number reference frames equal to 1.

Bug: 36998372
Change-Id: I1f84a16703422109d40bed8436f35d0c2069c088
2017-05-11 15:09:57 -07:00
Harish Mahendrakar
16fa990ce1 Decoder: Fixed initialization of first_slice_in_pic
am: 0b23c81c3d

Change-Id: I55e49ca9616a151456342bad7673d0986098b448
2017-02-14 17:03:06 +00:00
Harish Mahendrakar
ba7f9e2aed Decoder: Moved end of pic processing to end of decode call
am: 494561291a

Change-Id: Id3867f5dd0788f12f6827ed311a309d1fa58c535
2017-02-14 17:00:16 +00:00
Harish Mahendrakar
0b23c81c3d Decoder: Fixed initialization of first_slice_in_pic
To handle some errors, first_slice_in_pic was being set to 2.
This is now cleaned up and first_slice_in_pic is set to 1 only once per pic.
This will ensure picture level initializations are done only once even in case
of error clips

Bug: 33717589
Bug: 33551775
Bug: 33716442
Bug: 33677995

Change-Id: If341436b3cbaa724017eedddd88c2e6fac36d8ba
2017-02-14 16:59:09 +00:00
Harish Mahendrakar
494561291a Decoder: Moved end of pic processing to end of decode call
ih264d_end_of_pic() was called after parsing slice of a new picture.
This is now being done at the end of decode of the current picture.
decode_gaps_in_frame_num which needs frame_num of new slice is now
done after decoding frame_num in new slice.

This helps in handling errors in picaff streams with gaps in frames

Bug: 33588051
Bug: 33641588
Bug: 34097231
Change-Id: I1a26e611aaa2c19e2043e05a210849bd21b22220
2017-02-14 16:58:07 +00:00
Harish Mahendrakar
1f0ab0b089 DO NOT MERGE: Decoder: Fixes an out of bound write in bitstream buffer
[this is for mnc-dev only, not any other mnc-*-dev flavors;
there is a different patch for mnc-dr-dev, nyc-* and going forward]

After emulation prevention, data is written as an int,
so at least 3 additional bytes should be available.
And since bitstream functions read 8 bytes ahead, 8 extra bytes
should be available in the bitstream buffer.

Bug: 33934721
Test: Ittiam testing, POC in the bug no longer fails
Change-Id: I444ec6f85d01b0bade9f827e15c4b476779d6c69
2017-02-13 14:44:11 -08:00
Harish Mahendrakar
33ef7de9dd Decoder: Fixes an out of bound write in bitstream buffer
[for mnc-dr-dev and later;  mnc-dev gets a different patch]

After emulation prevention, data is written as an int,
so at least 3 additional bytes should be available.
And since bitstream functions read 8 bytes ahead, 8 extra bytes
should be available in the bitstream buffer.

Bug: 33934721

Change-Id: I444ec6f85d01b0bade9f827e15c4b476779d6c69
2017-01-19 09:17:00 -08:00
Marco Nelissen
d5424f66e3 Merge "DO NOT MERGE Decoder: Increase memory allocation for weights & offsets for interlaced clips" into mnc-dev 2017-01-18 18:42:47 +00:00
Marco Nelissen
696b6f666c Merge "Decoder: Increase memory allocation for weights & offsets for interlaced clips" into mnc-dr-dev 2017-01-18 16:27:26 +00:00
Harish Mahendrakar
cec6503a20 Decoder: Fixed DoS in header decode when no PPS is present
am: 2cd2f7a335

Change-Id: If8a7c11ef215a84a350cd0a734bca9c69b91f1c6
2017-01-18 00:26:14 +00:00
TreeHugger Robot
e74ab7dba4 Merge "Decoder: Fixed DoS in header decode when no PPS is present" into mnc-dev 2017-01-18 00:23:33 +00:00
Harish Mahendrakar
bee9b9a54b Decoder: Increase memory allocation for weights & offsets for interlaced clips
Bug: 33816782
Change-Id: I65a5376f46902139f9fc49a2fff54b53b518d703
2017-01-17 15:25:56 -08:00
Harish Mahendrakar
2cd2f7a335 Decoder: Fixed DoS in header decode when no PPS is present
When the input does not contain PPS and decoder is in header decode
mode, decoder was entering an infinite loop.

Bug: 33621215
(cherry picked from commit 33e1b190d6db09bd72a9f0f51acef4b14eabd6ff)
2017-01-17 15:25:13 -08:00
Harish Mahendrakar
74f03b639e DO NOT MERGE Decoder: Increase memory allocation for weights & offsets for interlaced clips
Bug: 33816782
Change-Id: I10f8b2a7691aeb55365901c166e765ab22ec6106
2017-01-17 23:25:11 +00:00
Harish Mahendrakar
4e7779e0ef Decoder: Fixed number of MB calculation for interlaced error streams
am: e1cf7ea8ae

Change-Id: If723b1da6adf7a816c095f327023853bcaccd0c5
2017-01-17 20:34:28 +00:00
Marco Nelissen
c4f317b873 Merge "Decoder: Fixed number of MB calculation for interlaced error streams" into mnc-dev 2017-01-17 20:27:24 +00:00
Harish Mahendrakar
2c71fb1941 Call ih264d_deblock_display only for valid process calls
am: 318368435f

Change-Id: I210dfb1eaf04b3ddceffa041c43e25e52023090d
2017-01-09 17:03:46 +00:00
Harish Mahendrakar
e1cf7ea8ae Decoder: Fixed number of MB calculation for interlaced error streams
At the end of picture processing, if the current pic is partially
decoded, number of MBs to be processed was wrongly calculated for
interlaced cases.

Bug: 33129467
Change-Id: Ia81186c60d346f02663607f2dc14166781db6a69
2016-12-28 15:07:09 -08:00
Harish Mahendrakar
318368435f Call ih264d_deblock_display only for valid process calls
ih264d_deblock_display() should be called only if current
decoder call has got a picture buffer during start of pic

Bug: 33751193
Change-Id: I3bf2fdbb910bf7240484dae48b85d72833830d56
2016-12-28 15:03:10 -08:00
Harish Mahendrakar
ff7ccdab0c Fix in the case of invalid SPS PPS
am: c2e8ffe451

Change-Id: Iaf266c5ff2187ba980737b699884e3ebebe87bd0
2016-09-21 20:38:58 +00:00
Harish Mahendrakar
c2e8ffe451 Fix in the case of invalid SPS PPS
Bug: 31092462
Change-Id: I0e2ab5a1088717bbf36f6b0cf859b5a6e0f59c5d
2016-09-21 09:50:06 -07:00
Harish Mahendrakar
ff54ee6115 Fixed error concealment when no MBs are decoded in the current pic
am: b926905ee2

Change-Id: I8a0a6fc068b543a4431b085c9335301e89ef2e14
2016-07-14 01:22:44 +00:00
Harish Mahendrakar
b926905ee2 Fixed error concealment when no MBs are decoded in the current pic
Bug: 29493002
Change-Id: I3fae547ddb0616b4e6579580985232bd3d65881e
2016-07-13 17:32:27 -07:00