Commit graph

233 commits

Author SHA1 Message Date
Hamsalekha S
4f7fa508be Decoder: Added an error check while parsing PPS. am: 62f98981ff am: 0f6a1883e8 am: e0e549aad5 am: d9a55dded2
am: d5f3df9c33

Change-Id: I8c2bfdd9bc97e79747b62d62bd28a683d6d9d77f
2017-05-12 21:08:07 +00:00
Hamsalekha S
0f6a1883e8 Decoder: Added an error check while parsing PPS.
am: 62f98981ff

Change-Id: I2976869f34cf2a20078c3fe4504db9e6c5a3035d
2017-05-12 20:56:41 +00:00
Hamsalekha S
62f98981ff Decoder: Added an error check while parsing PPS.
Added an error check while parsing PPS syntax element
second_chroma_qp_index_offset.

Bug: 37207120

Change-Id: Icba6b7bcf5940505717ee61134ed801c221b6e26
2017-05-12 20:46:28 +00:00
Ray Essick
cba660f4c9 Merge "DO NOT MERGE Fixed bug in the case of resolution change." into nyc-dev
am: 3d01dc4bdd

Change-Id: Ia7bcdb836e2cde9925812ddef6736b89cd0e2f8f
2017-05-12 17:45:55 +00:00
Ray Essick
3d01dc4bdd Merge "DO NOT MERGE Fixed bug in the case of resolution change." into nyc-dev 2017-05-12 17:42:54 +00:00
Ray Essick
126058dcb2 Merge changes from topic 'b35583675-mnc-dr' into mnc-dr-dev
* changes:
  DO NOT MERGE Fixed bug in the case of resolution change.
  DO NOT MERGE Handle level/profile/num_ref_frames/num_reorder change at the same resolution
2017-05-12 17:42:26 +00:00
Harish Mahendrakar
70d50b4389 Fix stack buffer overflow in ih264d_process_intra_mb am: f69e34419b am: cbaa0dc3f9 am: 943e36e510 am: 6a897f0c3a
am: b308dc1318

Change-Id: I294bbd58ff6bc340c848783f7cbcfb9d05701e2e
2017-05-10 16:51:44 +00:00
Harish Mahendrakar
b308dc1318 Fix stack buffer overflow in ih264d_process_intra_mb am: f69e34419b am: cbaa0dc3f9 am: 943e36e510
am: 6a897f0c3a

Change-Id: I75f379b049ee2336b50d0fffe77a6a842896819c
2017-05-10 16:43:41 +00:00
Harish Mahendrakar
cbaa0dc3f9 Fix stack buffer overflow in ih264d_process_intra_mb
am: f69e34419b

Change-Id: I154d8dd94d11be2d34b8085c30e24f6bec62d9eb
2017-05-10 16:20:37 +00:00
Harish Mahendrakar
f69e34419b Fix stack buffer overflow in ih264d_process_intra_mb
Aligned the sizes of au1_ngbr_pels to ensure SSE42 functions do not
result in stack buffer overflow

Bug: 36490809

Change-Id: I0bfe493f94647046013759b3ec9db3c627ac471e
2017-05-10 15:58:24 +00:00
Hamsalekha S
1d06027c69 DO NOT MERGE Fixed bug in the case of resolution change.
Modified the way i4_header_decoded in decoder context
is used, to ensure that resolution change is detected
even if PPS has not been decoded.

Bug: 35583675
Test: ran POC, no longer hangs

Change-Id: Ibb3f8dfbeb66a999fd81720a7d2a02dd951a55c4
2017-04-12 12:58:13 -07:00
Hamsalekha S
8454dbfccd DO NOT MERGE Fixed bug in the case of resolution change.
Modified the way i4_header_decoded in decoder context
is used, to ensure that resolution change is detected
even if PPS has not been decoded.

Cherry pick of Change-Id: Ibb3f8dfbeb66a999fd81720a7d2a02dd951a55c4
which went into master post-N.

Bug: 35583675
Test: ran POC, no longer hangs
Change-Id: Ifd21ebe1827d6d0c13018983c46c4301c2bb1669
2017-04-12 10:26:46 -07:00
Wonsik Kim
142221a3f9 DO NOT MERGE Handle level/profile/num_ref_frames/num_reorder change at the same resolution
Cherry pick of Change-Id: Ifa78c3125ab207ce5e39166f4891cba0d3a4e39c
which went into master (post-n). This needed backporting to M so that
the final fix for 35583675 could be integrated.

Bug: 35583675
Test: ran POC without failure
Change-Id: I0d248212aaf6635f34a70ad36657416a0c623d32
2017-04-12 10:23:33 -07:00
Marco Nelissen
b581941c80 Merge "Decoder: Fix in reference list initialization." into mnc-dev am: 0015860fdc am: 9208f9461f am: c783f2efbe am: 68c59343e1
am: 0d7aed6e94

Change-Id: I4c6078819cfa010fbeae9843e842c2ab0580bf1e
2017-04-06 23:24:43 +00:00
Marco Nelissen
0d7aed6e94 Merge "Decoder: Fix in reference list initialization." into mnc-dev am: 0015860fdc am: 9208f9461f am: c783f2efbe
am: 68c59343e1

Change-Id: I1c765b8f5a0d82e673ecf5e64df2616ffb4ce444
2017-04-06 23:21:12 +00:00
Marco Nelissen
9208f9461f Merge "Decoder: Fix in reference list initialization." into mnc-dev
am: 0015860fdc

Change-Id: I731793974cd3607fed07cb7895234e5f9484a0ea
2017-04-06 23:10:11 +00:00
Marco Nelissen
0015860fdc Merge "Decoder: Fix in reference list initialization." into mnc-dev 2017-04-06 23:00:48 +00:00
Ray Essick
41bb6d6a66 Merge "Decoder: Fixes in accessing mbaff flag in error cases" into mnc-dev am: 7ace2f5ca2 am: ba33452066 am: e8faa24c74 am: 57ba056850
am: aff1a88f83

Change-Id: I361ec98b31db1d82106b1042c2170008fa661ec7
2017-04-05 23:47:35 +00:00
Ray Essick
aff1a88f83 Merge "Decoder: Fixes in accessing mbaff flag in error cases" into mnc-dev am: 7ace2f5ca2 am: ba33452066 am: e8faa24c74
am: 57ba056850

Change-Id: I17a61a91bafc8dc1c888b2d084f5b64e2adb01cf
2017-04-05 23:45:06 +00:00
Ray Essick
ba33452066 Merge "Decoder: Fixes in accessing mbaff flag in error cases" into mnc-dev
am: 7ace2f5ca2

Change-Id: I5dde0d284cb06fa2faddae483900df1102d9bc52
2017-04-05 23:36:35 +00:00
Ray Essick
7ace2f5ca2 Merge "Decoder: Fixes in accessing mbaff flag in error cases" into mnc-dev 2017-04-05 23:29:45 +00:00
Ray Essick
fa74fd8b50 Merge "Fix in the case of MMCO 3 (long term reference idx)." into mnc-dev am: 64b617346c am: bfb5f29471 am: 98a81bb017 am: 14276b25d1
am: 24c16eb2b5

Change-Id: I271a81f034a16b81f23389a5f91b2de1a384bc12
2017-04-05 21:24:57 +00:00
Ray Essick
24c16eb2b5 Merge "Fix in the case of MMCO 3 (long term reference idx)." into mnc-dev am: 64b617346c am: bfb5f29471 am: 98a81bb017
am: 14276b25d1

Change-Id: Idc8f7d3cc1c5869a11d0abc2bdf1aedf38fa8224
2017-04-05 21:22:26 +00:00
Ray Essick
bfb5f29471 Merge "Fix in the case of MMCO 3 (long term reference idx)." into mnc-dev
am: 64b617346c

Change-Id: Ia57a698ac23ee4abeefe194c8585301941984771
2017-04-05 21:13:56 +00:00
Ray Essick
64b617346c Merge "Fix in the case of MMCO 3 (long term reference idx)." into mnc-dev 2017-04-05 21:09:18 +00:00
Harish Mahendrakar
0f2f2b5fde Decoder: Fixes in accessing mbaff flag in error cases
ps_dec->ps_cur_slice->u1_mbaff_frame_flag is updated in ih264d_start_of_pic().
So updated value should be used after calling ih264d_start_of_pic()

Bug: 33974623
Test: ran POC from bug
Change-Id: I0f1ff5e01ed39767f493f197791e51b0da74952f
(cherry picked from commit 3f6937a003)
2017-03-23 23:46:33 +00:00
Hamsalekha S
93954f5e9a Decoder: Fix in reference list initialization.
In the case of error, initialize the new reference list1 with the first
picture in default list0 instead of default list1, as first picture in
list1 could still be invalid.

Bug: 36035074

Change-Id: I7ab493ee7a157cbefcd4da8389ff1ff899c16b7f
2017-03-22 16:55:01 +00:00
Ray Essick
691577d923 Merge "Decoder: Fixed error handling for dangling fields" into mnc-dev am: 78334b2a24 am: cd5ebec24a am: f48987d2fc am: 1cbcdd7f16
am: 3a669baf18

Change-Id: I828a75f93c0a49474f9be6d075ca3b75d27cf8fd
2017-03-14 22:35:50 +00:00
Ray Essick
3a669baf18 Merge "Decoder: Fixed error handling for dangling fields" into mnc-dev am: 78334b2a24 am: cd5ebec24a am: f48987d2fc
am: 1cbcdd7f16

Change-Id: I2adb58066b67a321c9b7275874a7ebb04e8e7c9d
2017-03-14 22:30:50 +00:00
Ray Essick
cd5ebec24a Merge "Decoder: Fixed error handling for dangling fields" into mnc-dev
am: 78334b2a24

Change-Id: Id5cb9826e72c59890526300ad9e8833b61691c20
2017-03-14 22:15:20 +00:00
Ray Essick
78334b2a24 Merge "Decoder: Fixed error handling for dangling fields" into mnc-dev 2017-03-14 22:08:38 +00:00
Hamsalekha S
46e96d40db Fix in the case of MMCO 3 (long term reference idx).
Increment number of long term reference buffers only when both top field
and bottom field have been set as long term.

[backport for M/N from master]

Bug: 35584425
Test: ran POC - no hang, no segfault.

Change-Id: I94e3857944da675eda38f8e1a9bd887f48bff524
(cherry picked from commit 6fa5df8811)
2017-03-04 20:13:09 +00:00
Marco Nelissen
7c50684ef0 resolve merge conflicts of 3654ad0 to mnc-dr-dev am: 37345554fe am: 33d9d00a3c am: 858542d83e
am: 2e4b53cfb4

Change-Id: I36252dfb66db49049efbc6fe9a8e59aaf90fae35
2017-02-14 23:03:32 +00:00
Marco Nelissen
2e4b53cfb4 resolve merge conflicts of 3654ad0 to mnc-dr-dev am: 37345554fe am: 33d9d00a3c
am: 858542d83e

Change-Id: I8376533d60ce908a6a08e260f57cb6163a7fa2dd
2017-02-14 23:01:52 +00:00
Marco Nelissen
37345554fe resolve merge conflicts of 3654ad0 to mnc-dr-dev
Bug: 33818508
Bug: 34013472
Change-Id: I2e99cbceba1c00555d624e8975522725e362362b
2017-02-14 13:49:16 -08:00
Marco Nelissen
3654ad0da5 Merge changes Iec2941f1,I38be0e4c into mnc-dev
* changes:
  Decoder: Add supported width check for MBaff streams
  Decoder: Added a check for unsupported resolutions
2017-02-14 21:30:39 +00:00
Harish Mahendrakar
94e17dc822 Decoder: Fixed initialization of first_slice_in_pic am: 0b23c81c3d am: 16fa990ce1 am: ab79eb3973 am: a37d120eb2
am: c011eeade7

Change-Id: I3f850497c864277c2b2524fab07d14a249e48afc
2017-02-14 17:11:38 +00:00
Harish Mahendrakar
7944003371 Decoder: Moved end of pic processing to end of decode call am: 494561291a am: ba7f9e2aed am: 26bb8b98ee am: 89fa016b5f
am: df2fa01c9e

Change-Id: Ifc329cbccad01c70ff1fea0d483cb155b15b3cc1
2017-02-14 17:08:45 +00:00
Harish Mahendrakar
c011eeade7 Decoder: Fixed initialization of first_slice_in_pic am: 0b23c81c3d am: 16fa990ce1 am: ab79eb3973
am: a37d120eb2

Change-Id: Ib3eb4da620fc964a15a35d8e862b91e55a611bd4
2017-02-14 17:08:36 +00:00
Harish Mahendrakar
b19aa035f6 Decoder: Treat first slice in a picture as part of new picture always am: 8b5fd8f24e am: 8b75cdae9b am: fb68ae8960 am: a8a3dc84b0
am: 3007985473

Change-Id: I2570766293b1d7fae490523607390873ed0016f0
2017-02-14 17:08:36 +00:00
Harish Mahendrakar
df2fa01c9e Decoder: Moved end of pic processing to end of decode call am: 494561291a am: ba7f9e2aed am: 26bb8b98ee
am: 89fa016b5f

Change-Id: Ib8de0752f0e90dfea59d9e8380d2006dd9643312
2017-02-14 17:07:16 +00:00
Harish Mahendrakar
3007985473 Decoder: Treat first slice in a picture as part of new picture always am: 8b5fd8f24e am: 8b75cdae9b am: fb68ae8960
am: a8a3dc84b0

Change-Id: I172945f3e9d84d77ae21d1f737258851529f94c7
2017-02-14 17:07:07 +00:00
Harish Mahendrakar
16fa990ce1 Decoder: Fixed initialization of first_slice_in_pic
am: 0b23c81c3d

Change-Id: I55e49ca9616a151456342bad7673d0986098b448
2017-02-14 17:03:06 +00:00
Harish Mahendrakar
ba7f9e2aed Decoder: Moved end of pic processing to end of decode call
am: 494561291a

Change-Id: Id3867f5dd0788f12f6827ed311a309d1fa58c535
2017-02-14 17:00:16 +00:00
Harish Mahendrakar
8b75cdae9b Decoder: Treat first slice in a picture as part of new picture always
am: 8b5fd8f24e

Change-Id: Ia14e9fae4a08143ff86439b5440ab9cf5e3052e3
2017-02-14 17:00:07 +00:00
Harish Mahendrakar
0b23c81c3d Decoder: Fixed initialization of first_slice_in_pic
To handle some errors, first_slice_in_pic was being set to 2.
This is now cleaned up and first_slice_in_pic is set to 1 only once per pic.
This will ensure picture level initializations are done only once even in case
of error clips

Bug: 33717589
Bug: 33551775
Bug: 33716442
Bug: 33677995

Change-Id: If341436b3cbaa724017eedddd88c2e6fac36d8ba
2017-02-14 16:59:09 +00:00
Harish Mahendrakar
494561291a Decoder: Moved end of pic processing to end of decode call
ih264d_end_of_pic() was called after parsing slice of a new picture.
This is now being done at the end of decode of the current picture.
decode_gaps_in_frame_num which needs frame_num of new slice is now
done after decoding frame_num in new slice.

This helps in handling errors in picaff streams with gaps in frames

Bug: 33588051
Bug: 33641588
Bug: 34097231
Change-Id: I1a26e611aaa2c19e2043e05a210849bd21b22220
2017-02-14 16:58:07 +00:00
Harish Mahendrakar
8b5fd8f24e Decoder: Treat first slice in a picture as part of new picture always
This is needed to decode streams with consecutive IDRs.

Bug: 34097231
Test: successful run of POC in security bug
Change-Id: Ib737a4ef4b8c5bb7a57c90292102dd28af0615fe
2017-02-14 04:41:02 +00:00
Harish Mahendrakar
69b5191865 Decoder: Add supported width check for MBaff streams
In case of MBAff streams, decoder processes two rows at a time,
this limits maximum supported width to 1920 for MBAff streams.

Bug: 33818508
Bug: 34013472
Change-Id: Iec2941f116cf3c36b63013a930319960023a3b42
2017-02-13 15:30:57 -08:00
Harish Mahendrakar
9d60a0c5da Decoder: Added a check for unsupported resolutions
Decoder now returns an error for resolutions greater than 3840x2176

Bug: 24542936

Change-Id: I38be0e4c5cf2a980bfd4c781f3b49171f73b5ccb
2017-02-13 15:30:46 -08:00