Commit graph

117 commits

Author SHA1 Message Date
Ritu Baldwa
ed7a63fbcc DO NOT MERGE Decoder: Increased memory allocation.
ps_dec->u1_recon_mb_grp is twice the width in case of mbaff,
increasing relevant allocations accordingly.

Increased allocation of intra-prediction buffer to
include padding.

Bug: 64964675
Test: POC from bug report
Change-Id: Ic4a6151bb12ac1122c228220b9150b2a372aae21
2017-09-29 11:49:50 -07:00
Marco Nelissen
224824ba17 Merge "Decoder: Updated error check while parsing num_ref_idx_lx_active." into mnc-dev 2017-09-13 05:00:15 +00:00
TreeHugger Robot
a2d8a1444d Merge "Decoder: Corrected variable datatypes in ih264d_get_implicit_weights." into mnc-dev 2017-09-12 00:11:33 +00:00
Ray Essick
dc9396971d Merge "DO NOT MERGE: Decoder: Conceal picture only if valid picture buffer is obtained." into mnc-dev 2017-09-11 22:17:06 +00:00
Hamsalekha S
208c74d62a Decoder: Updated error check while parsing num_ref_idx_lx_active.
Added an error check on the lower limit of u1_num_ref_idx_lx_active,
while parsing slice header. The minimum possible value is 1.

Bug: 64836894

Change-Id: I57056851fc135ed00f7a10af5c81eb560e9e12de
2017-09-11 20:16:47 +00:00
Hamsalekha S
3eb692de91 Decoder: Corrected variable datatypes in ih264d_get_implicit_weights.
The difference between two 32 signed numbers was getting assigned
to 16 bits, leading to a divide by zero arithmetic execption.
Modified variable names to match their datatypes.

Bug: 65122447

Change-Id: I45ade1945f10b4d7660bd09fb564e60fd29d40dc
2017-09-08 13:21:14 -07:00
Ray Essick
45aa4ac86d DO NOT MERGE: Decoder: Conceal picture only if valid picture buffer is obtained.
If all the slices in the current pic were invalid, then
the decoder would not have received a valid picture buffer
in the current call. In such cases there is no need to conceal or
deblock the picture.

Bug: 62896384
Test: run ASAN-enabled PoC before/after the patch
Change-Id: Ia7d979a78ae3f4fb443a1759c8e6cf8780565ad8
Merged-In: I3cf6e871592826f93b0dcd2b06fff80677bc8338
2017-09-06 23:46:21 +00:00
Harish Mahendrakar
aa11ab9fdb Added an out of bound check on u4_num_bufs in input argument
ps_dec_ip->s_out_buffer.u4_num_bufs was missing out of bound checks

Bug: 62688399
Change-Id: Ic5e5c002d29fcb18064550d5a5f9289bb68b448e
2017-08-10 10:15:16 -07:00
Hamsalekha S
cbcd2846fa Decoder: Fixed overflow in refernce list creation.
Since the maximum value of long term index is 255,
the loop control variable needs to be 32 bit.

Bug: 38448381
Test: ran POC before/after applying fix
Change-Id: Iae3ecff38d4a922bde10fde33f1cfcafd2ea2680
2017-07-07 10:34:44 -07:00
Hamsalekha S
b8d362561e Initialize DPB structures to valid values.
When the first frame is a B frame, the colocated picture
will now point to the current frame.

Test: run poc with and without this patch
Bug: 38115076
Change-Id: I48a8f128740551d6a9252931dafcf8c629ecad0d
2017-06-20 20:34:06 +00:00
Hamsalekha S
3f6c941de5 Added error check for output buffer size.
The output buffer size given by the application, needs to be checked
in every process call. This is required in the case of resolution
change.

Bug: 36006815
Test: avcdec -i poc.bin
Change-Id: I16a92cdad23eb7b1e12c1a67c1b2599204f29249
2017-06-16 17:41:23 -07:00
Ray Essick
16b019a620 Merge "Fixed hang in the case of multiple sps id." into mnc-dev 2017-06-06 23:48:10 +00:00
TreeHugger Robot
6b3d20c032 Merge "Decoder: Fix in the case of MMCO 6" into mnc-dev 2017-06-06 22:54:33 +00:00
Hamsalekha S
41489f9ece Decoder: Fix in the case of MMCO 6
Added an error check in the case of MMCO 6
(SET_LT_INDEX)

Bug: 38014992
Test: POC fails before / works after patch
Change-Id: I76e38a8e2ff0bab043b47f44f1f7b1d4fe60d416
(cherry picked from commit 9e4f0ce704)
2017-06-06 21:37:21 +00:00
Hamsalekha S
ec3f585000 Fixed hang in the case of multiple sps id.
The sps parameters used to detect change in
resolution/sps were incorrect. Made a fix to
use current sps from decoder context.

Bug: 38239864

Change-Id: I2d110e635ced32b3dc7f364e08a97d672fcbae37
(cherry picked from commit 8c6fe35f6d)
2017-06-06 21:22:17 +00:00
Hamsalekha S
4eb72f7c93 Decoder: Cleaned up parse sps function.
Postponed the initializations to decoder context
till the end of the parse sps function, after
all the error checks are done.

Bug: 37968755
Test: ran poc on ASAN-enabled build before/after
Change-Id: Ibee3383c28cede3edb68d2459565d6ce10683bbd
2017-06-06 21:05:07 +00:00
Marco Nelissen
3477b8e0d7 Merge "Initializing reference list for every P/B slice." into mnc-dev 2017-06-06 20:14:42 +00:00
TreeHugger Robot
7fc42b801c Merge "Fix resolution change within a decode call." into mnc-dev 2017-06-06 17:32:41 +00:00
Hamsalekha S
fe18375850 Fix resolution change within a decode call.
If resolution changes within a decode call,due to multiple
sps, the decoder hangs as the the application will
give the same data again in the next decode call. This
results in a hang. Fixed this by flaging an error,
when sps/resoultion changes within a process call.

Bug: 38487564
Test: ran POC on patched O-based system w/o hanging
Change-Id: I30095b2e8bf573c1a58a316a23b1a5e6a4af589b
2017-06-05 13:28:44 -07:00
TreeHugger Robot
0a91df5eca Merge "DO NOT MERGE Decoder: Fixed allocation size of pred info buffer" into mnc-dev 2017-06-02 18:40:21 +00:00
Hamsalekha S
f9d3f9af8f Initializing reference list for every P/B slice.
Reference list needs to be initialized for every P/B
slice, to ensure colocated picture always points to a
valid picture buffer, even in the case of error.

Bug: 36279112

Change-Id: I051d7e725b0af209cc7bb333db8da3518adf78a0
2017-06-02 10:22:52 -07:00
TreeHugger Robot
810356f4de Merge "Decoder: Fix end of bitstream error." into mnc-dev 2017-05-16 20:36:28 +00:00
Marco Nelissen
cb89747145 Merge "DO NOT MERGE Decoder: Fix allocation for Mbaff weight matrix" into mnc-dev 2017-05-16 15:22:10 +00:00
TreeHugger Robot
b0399be998 Merge "DO NOT MERGE Fixed bug during resolution change" into mnc-dev 2017-05-15 18:45:41 +00:00
Hamsalekha S
6b8184944b DO NOT MERGE Fixed bug during resolution change
Modifies how i4_header_decoded is decoder context is used, to ensure
that resolution change is detected even if PPS has not been decoded.

This retrofits parts of the below mentioned 'merged-in' commits
which solve this for mnc-dr and later versions of the library. The retrofit
required dropping portions that affected code introduced in a later baseline.

Bug: 35583675
Test: provided input file no longer hangs
Change-Id: I8fa30543759c5762b18e2380513e2ce616c858ef
Merged-In: I0d248212aaf6635f34a70ad36657416a0c623d32
Merged-In: Ifd21ebe1827d6d0c13018983c46c4301c2bb1669
2017-05-15 10:22:55 -07:00
TreeHugger Robot
cc07eaa5d2 Merge "Decoder: Fixed flag u1_top_bottom_decoded." into mnc-dev 2017-05-12 22:06:48 +00:00
Marco Nelissen
f7e747727c Merge "Decoder: Initialize MB info buffer to zero." into mnc-dev 2017-05-12 21:38:45 +00:00
Hamsalekha S
60be1fe459 DO NOT MERGE Decoder: Fix allocation for Mbaff weight matrix
Increased the allocation size for Mbaff weight
matrix buffer

Bug: 36996978

Change-Id: I66c8ab24ec4083f16a86ffc9c0e36f5fd7785f92
2017-05-12 14:08:44 -07:00
Hamsalekha S
62f98981ff Decoder: Added an error check while parsing PPS.
Added an error check while parsing PPS syntax element
second_chroma_qp_index_offset.

Bug: 37207120

Change-Id: Icba6b7bcf5940505717ee61134ed801c221b6e26
2017-05-12 20:46:28 +00:00
Hamsalekha S
7703822731 Decoder: Fixed flag u1_top_bottom_decoded.
Fixed initialization of flag u1_top_bottom_decoded
in decoder context. This flag indicates if top
field and botton field is decoded.

Bug: 36993291
Test: avcdec --input poc.h264 --output /dev/null
Change-Id: I9f8a2620683abd8b15e4780d76d4849394710716
2017-05-12 20:14:46 +00:00
Hamsalekha S
2e01924cd6 Decoder: Fix end of bitstream error.
The end of bistream error check was fixed for
odd number of macroblocks in Mbaff frames.

Bug: 37008096
Test: Ittiam-verified
Change-Id: I058d74a3c1d1511968c2b36802dfc5c102947919
2017-05-11 17:11:25 -07:00
Hamsalekha S
6a5cb80068 DO NOT MERGE Decoder: Fixed allocation size of pred info buffer
Buffer allocation size for pred info was increased
in the case number reference frames equal to 1.

Bug: 36998372
Change-Id: I1f84a16703422109d40bed8436f35d0c2069c088
2017-05-11 15:09:57 -07:00
Hamsalekha S
b421b4bc3b Decoder: Initialize MB info buffer to zero.
Initialize the buffer used to store inter mb info
(reference index, weights etc) to zero.

Bug: 36035683

Change-Id: Ie3ea4307de45813edd553b590df44323be46534f
2017-05-11 12:07:10 -07:00
Harish Mahendrakar
f69e34419b Fix stack buffer overflow in ih264d_process_intra_mb
Aligned the sizes of au1_ngbr_pels to ensure SSE42 functions do not
result in stack buffer overflow

Bug: 36490809

Change-Id: I0bfe493f94647046013759b3ec9db3c627ac471e
2017-05-10 15:58:24 +00:00
Marco Nelissen
0015860fdc Merge "Decoder: Fix in reference list initialization." into mnc-dev 2017-04-06 23:00:48 +00:00
Ray Essick
7ace2f5ca2 Merge "Decoder: Fixes in accessing mbaff flag in error cases" into mnc-dev 2017-04-05 23:29:45 +00:00
Ray Essick
64b617346c Merge "Fix in the case of MMCO 3 (long term reference idx)." into mnc-dev 2017-04-05 21:09:18 +00:00
Harish Mahendrakar
0f2f2b5fde Decoder: Fixes in accessing mbaff flag in error cases
ps_dec->ps_cur_slice->u1_mbaff_frame_flag is updated in ih264d_start_of_pic().
So updated value should be used after calling ih264d_start_of_pic()

Bug: 33974623
Test: ran POC from bug
Change-Id: I0f1ff5e01ed39767f493f197791e51b0da74952f
(cherry picked from commit 3f6937a003)
2017-03-23 23:46:33 +00:00
Hamsalekha S
93954f5e9a Decoder: Fix in reference list initialization.
In the case of error, initialize the new reference list1 with the first
picture in default list0 instead of default list1, as first picture in
list1 could still be invalid.

Bug: 36035074

Change-Id: I7ab493ee7a157cbefcd4da8389ff1ff899c16b7f
2017-03-22 16:55:01 +00:00
Ray Essick
78334b2a24 Merge "Decoder: Fixed error handling for dangling fields" into mnc-dev 2017-03-14 22:08:38 +00:00
Hamsalekha S
46e96d40db Fix in the case of MMCO 3 (long term reference idx).
Increment number of long term reference buffers only when both top field
and bottom field have been set as long term.

[backport for M/N from master]

Bug: 35584425
Test: ran POC - no hang, no segfault.

Change-Id: I94e3857944da675eda38f8e1a9bd887f48bff524
(cherry picked from commit 6fa5df8811)
2017-03-04 20:13:09 +00:00
Marco Nelissen
3654ad0da5 Merge changes Iec2941f1,I38be0e4c into mnc-dev
* changes:
  Decoder: Add supported width check for MBaff streams
  Decoder: Added a check for unsupported resolutions
2017-02-14 21:30:39 +00:00
Harish Mahendrakar
0b23c81c3d Decoder: Fixed initialization of first_slice_in_pic
To handle some errors, first_slice_in_pic was being set to 2.
This is now cleaned up and first_slice_in_pic is set to 1 only once per pic.
This will ensure picture level initializations are done only once even in case
of error clips

Bug: 33717589
Bug: 33551775
Bug: 33716442
Bug: 33677995

Change-Id: If341436b3cbaa724017eedddd88c2e6fac36d8ba
2017-02-14 16:59:09 +00:00
Harish Mahendrakar
494561291a Decoder: Moved end of pic processing to end of decode call
ih264d_end_of_pic() was called after parsing slice of a new picture.
This is now being done at the end of decode of the current picture.
decode_gaps_in_frame_num which needs frame_num of new slice is now
done after decoding frame_num in new slice.

This helps in handling errors in picaff streams with gaps in frames

Bug: 33588051
Bug: 33641588
Bug: 34097231
Change-Id: I1a26e611aaa2c19e2043e05a210849bd21b22220
2017-02-14 16:58:07 +00:00
Harish Mahendrakar
8b5fd8f24e Decoder: Treat first slice in a picture as part of new picture always
This is needed to decode streams with consecutive IDRs.

Bug: 34097231
Test: successful run of POC in security bug
Change-Id: Ib737a4ef4b8c5bb7a57c90292102dd28af0615fe
2017-02-14 04:41:02 +00:00
Harish Mahendrakar
69b5191865 Decoder: Add supported width check for MBaff streams
In case of MBAff streams, decoder processes two rows at a time,
this limits maximum supported width to 1920 for MBAff streams.

Bug: 33818508
Bug: 34013472
Change-Id: Iec2941f116cf3c36b63013a930319960023a3b42
2017-02-13 15:30:57 -08:00
Harish Mahendrakar
9d60a0c5da Decoder: Added a check for unsupported resolutions
Decoder now returns an error for resolutions greater than 3840x2176

Bug: 24542936

Change-Id: I38be0e4c5cf2a980bfd4c781f3b49171f73b5ccb
2017-02-13 15:30:46 -08:00
Harish Mahendrakar
1f0ab0b089 DO NOT MERGE: Decoder: Fixes an out of bound write in bitstream buffer
[this is for mnc-dev only, not any other mnc-*-dev flavors;
there is a different patch for mnc-dr-dev, nyc-* and going forward]

After emulation prevention, data is written as an int,
so at least 3 additional bytes should be available.
And since bitstream functions read 8 bytes ahead, 8 extra bytes
should be available in the bitstream buffer.

Bug: 33934721
Test: Ittiam testing, POC in the bug no longer fails
Change-Id: I444ec6f85d01b0bade9f827e15c4b476779d6c69
2017-02-13 14:44:11 -08:00
Ray Essick
51ecbcabfb Merge "Decoder: Return correct error code for slice header errors" into mnc-dev 2017-02-13 19:15:36 +00:00
Ray Essick
d61abc5195 Merge "Decoder: Initialize default reference buffers for all pictures" into mnc-dev 2017-02-13 19:15:31 +00:00