Commit graph

550 commits

Author SHA1 Message Date
Ray Essick
83c84781df Merge "Decoder: Update check for increment u2_cur_slice_num" into oc-mr1-dev am: 6b338befbc am: 6ddeb05426 am: a537f15bc9
Original change: 14460944

Change-Id: Iafcffc3094782c99dfe832f4bcfe91324e37a7fc
2021-06-08 04:57:54 +00:00
Ray Essick
a537f15bc9 Merge "Decoder: Update check for increment u2_cur_slice_num" into oc-mr1-dev am: 6b338befbc am: 6ddeb05426
Original change: 14460944

Change-Id: Id3844c9c5eb2c20ad2db75981bb19ff58151be47
2021-06-08 04:41:36 +00:00
Rakesh Kumar
a88e0683a4 Decoder: Update check for increment u2_cur_slice_num
Increment u2_cur_slice_num only if current slice had atleast
one MB of memory left.

Test: clusterfuzz generated poc in bug
Bug: b/182152757
Bug: b/179938345
Bug: b/185112718

Change-Id: Ic5eb07e961bccb7fde954bcfd791fd879804e335
2021-05-06 19:51:10 -07:00
Harish Mahendrakar
5f4f6e2ecf decoder: Update check for first mb in slice am: 7e06940dce am: 4fb26568a9 am: 2bf6790f63
Original change: 13171299

MUST ONLY BE SUBMITTED BY AUTOMERGER

Change-Id: I99b30e4b2ac5f7b0fdeac9adbce15e92d2b36c8f
2020-12-09 01:48:37 +00:00
Harish Mahendrakar
2bf6790f63 decoder: Update check for first mb in slice am: 7e06940dce am: 4fb26568a9
Original change: 13171299

MUST ONLY BE SUBMITTED BY AUTOMERGER

Change-Id: I062c47998e23df0fe4d9c18ad9622aa55eb7ec96
2020-12-09 01:33:06 +00:00
Harish Mahendrakar
7e06940dce decoder: Update check for first mb in slice
first_mb_in_slice shouldn't be >= mbs in the picture.

Test: poc in bugs
Bug: b/174238784
Bug: b/174507022
Bug: oss-fuzz:27856
Bug: oss-fuzz:28039
Change-Id: Id3a41c8c2ddf814910fc2d5dd4f57bdd84d28fec
2020-12-04 16:09:10 -08:00
TreeHugger Robot
280902e61f Merge "decoder: Fix lt_ref increment in ih264d_fix_error_in_dpb" into qt-qpr1-dev 2020-03-20 02:24:36 +00:00
Harish Mahendrakar
f80afe5b98 decoder: Fix lt_ref increment in ih264d_fix_error_in_dpb
When removing common nodes from short term list if it present in long
term reference list, number of long term buffers was wrongly
incremented. Only decrement of short term buffer count is needed in
such a case.

Bug: 140561484
Bug: 140566606
Bug: 140699853
Bug: 144980954
Bug: 145859214

Test: poc in bug
Change-Id: Iebf7ee7ed65e7b2097a899577982806b03fc6e5d
2020-02-11 16:29:24 -08:00
Harish Mahendrakar
6dd54b9e18 decoder: Fix in detecting dangling fields
When decoder is expecting to decode a field, ensure current slice
being decoded is a field and frame number matches with previous slice's
frame number.

Bug: 144687080
Test: poc in bug
Change-Id: I89d9d7b6fc26797de0420ecb0ca7e66b066aa908
2020-02-11 16:03:29 -08:00
Automerger Merge Worker
8c2ee3ea8b Merge "decoder: Fix minimum poc calculation check while adding to display" into oc-dev am: 9d7a036d7b am: cebf26ab4d am: f64194a96f am: ed2b83d989 am: 4b7a562681
Change-Id: I0966eed6f66382e3fa7ccc19558734cc202ed0f3
2020-01-09 04:51:33 +00:00
Automerger Merge Worker
4b7a562681 Merge "decoder: Fix minimum poc calculation check while adding to display" into oc-dev am: 9d7a036d7b am: cebf26ab4d am: f64194a96f am: ed2b83d989
Change-Id: I7feff45df303f49d95b4d5b07019929dead2810c
2020-01-09 04:38:27 +00:00
Automerger Merge Worker
f64194a96f Merge "decoder: Fix minimum poc calculation check while adding to display" into oc-dev am: 9d7a036d7b am: cebf26ab4d
Change-Id: Ib8872212d43d41ae25f28992b6c02ab15cb72b26
2020-01-09 04:06:39 +00:00
TreeHugger Robot
9d7a036d7b Merge "decoder: Fix minimum poc calculation check while adding to display" into oc-dev 2020-01-09 03:32:14 +00:00
Harish Mahendrakar
ffcf2a87d6 decoder: Fix minimum poc calculation check while adding to display
While adding the decoded buffer to display queue, allow buffer with
poc set to 0x7FFFFFFF

Bug: 145364230
Test: poc in bug
Change-Id: I2a15f73b8422cfa4fd3360bc21c0cea4542a3375
2020-01-02 10:22:38 -08:00
Harish Mahendrakar
474797af3c decoder: Move initialization of dbp_mgr entries to init_decoder() am: 01da7b5a52 am: 41fcd228ae am: cce42ea920 am: 6c9873af88 am: 662efee177
am: 873bac42ec

Change-Id: I6f5d7d5486926100a815ffcddd134cf998d35c79
2019-11-05 17:05:10 -08:00
Harish Mahendrakar
662efee177 decoder: Move initialization of dbp_mgr entries to init_decoder() am: 01da7b5a52 am: 41fcd228ae am: cce42ea920
am: 6c9873af88

Change-Id: Ie95d6b810f751847eed7bc2c17c120a891b88fef
2019-11-05 16:52:06 -08:00
Harish Mahendrakar
cce42ea920 decoder: Move initialization of dbp_mgr entries to init_decoder() am: 01da7b5a52
am: 41fcd228ae

Change-Id: Id8fa44c3401f7ac1d2a540fa35cf0f4346da2c42
2019-11-05 16:31:59 -08:00
Harish Mahendrakar
01da7b5a52 decoder: Move initialization of dbp_mgr entries to init_decoder()
Earlier these were only initialized during static buffer allocations.
Initializing them in init_decoder() will ensure that these get
initialized to default values during reset() as well. Without this,
in some error cases, there is a possibility of heap-use-after free,
when resolution changes and these pointers point to memory that is freed

Bug: 142602711
Test: poc in bug
Change-Id: Ie39fee0eca56bf32cdc558099bf167d05eb89620
2019-10-29 19:52:49 -07:00
Rakesh Kumar
fe5a6ada3e Decoder: Add support for returning reorder_depth
Added support for returning reorder_depth for frames in decode order.

Bug: 138627015
Test: Verify reorder_depth returned for few clips

Change-Id: I9b68a9544d7d011cf93dfccd308c754b8e16e39e
2019-09-11 22:29:39 +00:00
TreeHugger Robot
6db97fcc8d Merge "Decoder: Add error check for expected_poc in ih264d_decode_pic_order_cnt" into qt-dev 2019-05-30 17:31:28 +00:00
Rakesh Kumar
75cd59e45b [automerger] Decoder: Delete node from st if lt and st point to same am: f312a1d305 am: f82a1f8d7f am: ae2c8cfdc1 am: 8332d0bfa2 am: 1e6ef7fee5 am: b1734142ff am: 9460fbd521 am: 224bcf4592
am: d406f08816

Change-Id: I1351851c15384868d9ab579857e4fc46076f5942
2019-05-29 14:31:32 -07:00
Rakesh Kumar
224bcf4592 [automerger] Decoder: Delete node from st if lt and st point to same am: f312a1d305 am: f82a1f8d7f am: ae2c8cfdc1 am: 8332d0bfa2 am: 1e6ef7fee5 am: b1734142ff
am: 9460fbd521

Change-Id: I0c53bc30225153ff03d44670f6967a05e161c654
2019-05-29 14:16:23 -07:00
Isha Kulkarni
89daff526a Decoder: Add error check for expected_poc in ih264d_decode_pic_order_cnt
Bug: 131625739
Test: Validate with poc in bug

Change-Id: I3e1c132e28ad35541a0b9c4632b1f43611b90dea
2019-05-29 13:46:13 -07:00
Android Build Merger (Role)
b1734142ff [automerger] Decoder: Delete node from st if lt and st point to same am: f312a1d305 am: f82a1f8d7f am: ae2c8cfdc1 am: 8332d0bfa2 am: 1e6ef7fee5
Change-Id: Id6d3411d949ae069369d7bef1c50f904cf9c97dc
2019-05-08 18:45:13 +00:00
Android Build Merger (Role)
f82a1f8d7f [automerger] Decoder: Delete node from st if lt and st point to same am: f312a1d305
Change-Id: I90a0c3fca5d7542301994711448d7c79fe98e704
2019-05-08 18:45:05 +00:00
Rakesh Kumar
f312a1d305 Decoder: Delete node from st if lt and st point to same
If lt_list and st_list point to same node then delete it from st.
If there is error while adding a node in bottom field of lt_list (top is
already added) then this node will be pointed by st_list also. So we need
to remove it from st_list

bug: 73552574
Test: poc before/after on Android N security branch
Change-Id: I95304c242c5854b18c5c7220d114ce6215760124
2019-05-08 11:44:40 -07:00
TreeHugger Robot
6cb07e106a Merge "decoder: Remove skip picture support" into qt-dev 2019-05-04 21:16:35 +00:00
TreeHugger Robot
a09be99800 Merge "Decoder: Add missing error checks for parsing sps or pps" into qt-dev 2019-05-04 19:37:30 +00:00
Harish Mahendrakar
4f95fc0b9c decoder: Remove skip picture support
SKIP_P and SKIP_PB support was experimental and is not used
in libstagefright

Bug: 130541597
Test: poc in bug
Change-Id: If13b435e80d13d57de2fee0aa07ea7e6abca481b
2019-05-01 13:18:27 -07:00
Ritu Baldwa
b3427a4c49 Decoder: Add missing error checks for parsing sps or pps
Return error to application if valid sps or pps has not been decoded.
Also add check to prevent parsing of pps, if valid sps is not present.

Test: Tested a few clips with missing sps/pps
Bug: 129746168
Bug: 130575087

Change-Id: I1a6409d1cffc66c1b910d2102ab10a94e0227107
2019-05-01 13:01:23 -07:00
Harish Mahendrakar
54160a6d3e decoder: Treat unsupported feature errors as fatal errors
This will ensure clips with supported bitdepth or unsupport color
formats are not attempted to be decoded till end of the sequence.

Bug: 130578045
Test: adb shell stagefright -s -S /sdcard/crowd422_h264.mp4
Test: Ensure decoder doesn't attemp decoding all frames

Change-Id: I118ef2141f77986f6ec65c91dfeaa7ef17646c43
2019-04-30 17:14:01 -07:00
Isha Kulkarni
34769a5b08 Decoder: Fix Integer Sanitizer Issues
Bug: 118445723
Test: vendor
Change-Id: I719898beb93ca5d20d9822f685ac6d4920252cdf
2019-03-28 14:40:40 -07:00
Isha Kulkarni
43dc60e81e Decoder: Fix Integer Sanitizer Issues in ih264d_parse_cavlc
Bug:118445723
Test: vendor
Change-Id: I71b3c0f548b57c51c80de0bd4c640f56b2f31a55
2019-03-28 14:40:34 -07:00
Isha Shrikant Kulkarni
96f7193ba0 Decoder: Fix integer sanitizer issues in ih264d_parse_cabac
Bug: 118445723
Test: vendor
Change-Id: Icf98785fa474bd9c774ba4282b9c3d73360a69ea
2019-03-28 14:40:27 -07:00
Isha Shrikant Kulkarni
fc7139a53e Decoder: Fix Integer Sanitizer issues in ih264d_process_bslice
Bug: 118445723
Test: vendor
Change-Id: Ib5d8f72be016330beafa6e7996976eeff1b23c39
2019-03-28 14:40:17 -07:00
Ray Essick
fbc772b17b Merge "Decoder: Add null check for scaling matrix." 2019-02-12 21:54:59 +00:00
Sasha Smundak
51f1a220ec Explicitly include log/log.h instead of cutils/log.h
Eliminates the warning.
Test: treehugger

Bug: 123758136
Change-Id: Id8940e053c90c45ba39cdd27a44cc83f130487b7
2019-02-01 10:45:14 -08:00
S Hamsalekha
f28de0366c Decoder: Add null check for scaling matrix.
Also void->word32 return type so failure can be signaled to caller.

Bug: 115743419
Test: vendor + poc
Change-Id: I0a07ac11255c2c672803aab7085824c417bde282
2019-01-08 11:37:37 -08:00
Harish Mahendrakar
b13124444c [automerger] decoder: Signal IVD_RES_CHANGED error for change in crop params am: 442a01bf37 am: 1d7d7f9bf6 am: acacf063c2 am: f33c02ed73 am: c09d3f99ab am: 145ae0d049 am: 4402b34a4f am: 7a6b55da22 am: 48900d4e51
am: 2e5273847c

Change-Id: I7cf3e92affd9c8e9f90df336441ef1b4d5d33734
2018-11-27 12:23:55 -08:00
Harish Mahendrakar
7a6b55da22 [automerger] decoder: Signal IVD_RES_CHANGED error for change in crop params am: 442a01bf37 am: 1d7d7f9bf6 am: acacf063c2 am: f33c02ed73 am: c09d3f99ab am: 145ae0d049
am: 4402b34a4f

Change-Id: I16ce7cfc21dcb0ccf10b4ce8109a61e272765120
2018-11-27 12:10:36 -08:00
Android Build Merger (Role)
145ae0d049 [automerger] decoder: Signal IVD_RES_CHANGED error for change in crop params am: 442a01bf37 am: 1d7d7f9bf6 am: acacf063c2 am: f33c02ed73 am: c09d3f99ab
Change-Id: I0dda0584bd0bf112fade0827b0460a82f8c14437
2018-11-10 18:16:37 +00:00
Android Build Merger (Role)
1d7d7f9bf6 [automerger] decoder: Signal IVD_RES_CHANGED error for change in crop params am: 442a01bf37
Change-Id: I05d6d693a8e8d18cd3cf74d95271efd8a339e06f
2018-11-10 18:16:32 +00:00
Harish Mahendrakar
442a01bf37 decoder: Signal IVD_RES_CHANGED error for change in crop params
IVD_RES_CHANGED was not signaled when crop parameters changed, i.e.
display dimensions changed without change in decode dimensions.

In such cases, if output buffer was allocated as per the current
dimension being decoded, without IVD_RES_CHANGED signalled, there can be
an OOB write if the new buffer is smaller than the frame being returned
as output

Bug: 118399205
Test: vendor
Change-Id: Ia750a99cda08a3254a6f8ea8b55d07e655b34d05
2018-11-10 10:16:06 -08:00
Thomala Srinivas
2286dbb91f Decoder: Initialize some array and structure to zero
In intra_pred_ref_filtering function, initialize the values
of array au1_ngbr_pels to zero and in ih264d_parse_decode_slice
function, initialize the values of struct s_tmp_poc to zero.

Bug: 111762686
Test: vendor
Change-Id: Iac67f5e7d24f821627de64c8570f4be648c296c8
2018-11-01 14:35:05 -07:00
Rakesh Kumar
590ff56ba5 Decoder: Memset allocated dynamic and static buffer to zero
Bug: 111407302
Bug: 111208713
Bug: 111128067
Bug: 111762807
Bug: 111762100
Bug: 111761624
Bug: 112328051
Bug: 112329563
Bug: 112326322
Bug: 112205113
Bug: 112204376
Test: manual
Change-Id: I35e6069a26ee5ef4c0fff111df255a5eddff78ae
2018-08-23 17:27:22 -07:00
Rakesh Kumar
837aefa3a2 Fix Compiler warnings
Intrinsic _mm_set_epi8 require char input, converting everything to char.
Including log.h from android library instead of cutils.

Bug: 112316276
Test: before/after object comparison for no diff in generated code.
Change-Id: I82b8c48494619b64465eda2476c7c903652c6066
2018-08-10 15:17:08 -07:00
TreeHugger Robot
2c566a8712 Merge "Fix signed integer overflow in prev_max_display_seq" 2018-06-20 22:14:25 +00:00
Android Build Merger (Role)
e6d4971fdb [automerger] Bug fix for flush without valid frames am: 10c1176f1b am: 54fe392964 am: 6261ed297b am: 051d4fb2dc am: 1863b7cc03
Change-Id: Ibd65acb3016732e53b25cb4aee2ec58076f6340f
2018-05-21 23:36:25 +00:00
Android Build Merger (Role)
54fe392964 [automerger] Bug fix for flush without valid frames am: 10c1176f1b
Change-Id: Ie26a4e3f41ab437f9d41bddb0a1166274fc3a261
2018-05-21 23:36:19 +00:00
Hamsalekha S
10c1176f1b Bug fix for flush without valid frames
We now return fail and get out of flush mode to accept bitstream in the next call.

Bug: 35585952
Bug: 63521984
Test: test case does not hang
Change-Id: Id22cc98d4a47714475a67918990a181a805c4c9f
2018-05-21 15:36:09 -07:00