Commit graph

17 commits

Author SHA1 Message Date
Ashwin Natesan
63b8be0524 svcenc: Integer overflow in irc_ba_get_cur_frm_est_texture_bits
'isvce_svc_rc_params_validate' was not being invoked prior to
call to 'isvce_rc_init'. This resulted in an erroneous state
within RC's context wherein the instantaneous estimate for the
texture bits for the frame being processed exceeded INT_MAX.
'isvce_svc_rc_params_validate' has code that detects such a
state and is now being correctly invoked where apprpriate.

Bug = ossfuzz:63175
Test: svc_enc_fuzzer
2023-12-13 07:40:47 -08:00
Ashwin Natesan
51268b3a71 svcenc: Redundant code removed
The following lines of code have been removed to improve coverage -
  [x] Functions -
      - isvc_interleaved_copy
      - isvc_16bit_interleaved_copy
      - isvc_16bit_interleaved_memset
      - isvc_iquant_itrans_recon_chroma_4x4_neon
      - isvc_iquant_itrans_recon_chroma_4x4_sse42
      - isvc_iquant_itrans_recon_4x4_dc_with_res_output_neon
      - isvc_iquant_itrans_recon_res_dc_4x4_sse42
      - isvc_iquant_itrans_recon_4x4_dc_with_res_accumulate_neon
      - isvc_iquant_itrans_recon_res_dc_with_res_acc_4x4_sse42
      - isvce_wait_for_thread
  [x] Function pointer initialisations for the functions above

Test: svc_enc_fuzzer
2023-10-27 06:14:32 -07:00
Ashwin Natesan
ea694873e0 svcenc: Enabled and verified 420sp inputs
Test: svc_enc_fuzzer
2023-10-10 07:07:21 -07:00
Ashwin Natesan
b5a861581e svcenc: RC parameters verified before RC init
RC uses int32_t to store bitrates and other
parameters internally. For specific magnitudes of
bitrate, framerate, and GOP period, this can
result in signed integer overflow. This is now
detected before calls to RC init.

Note that calls to the 'ISVCE_CMD_CTL_SET_BITRATE' API can also
result in this behaviour but it will be appropriately handled
by 'isvce_svc_frame_params_validate'.

Bug = ossfuzz:63053
Test: svc_enc_fuzzer
2023-10-09 06:50:02 -07:00
Ashwin Natesan
56a598ca6f svcenc: Fixes for incorrect handling of entropy coding errors
There were cases where errors in a process thread
executing the last MB row would result in the entropy
job list without a terminnating entry, which would
result in a call to ih264_list_dequeue blocking infinitely.
The enum 'ISVCE_ENTROPY_THREAD_STATES_T' is now used to
monitor the eecution state of the entropy thread and handle
the situation described appropriately.

Bug = ossfuzz:62786
Test: svc_enc_fuzzer
2023-10-02 07:09:21 -07:00
Ram Mohan M
50fd862b8d libavcenc: Remove duplicate code and improve readability
This commit does not introduce any new functionality w.r.t previous
commit. But it fixes few things. They are listed below:
1. Guard Bands in header files are fixed
2. Header files contains function definition comments. These are same
as in source file. Maintaining same comment at two locations is
unnecessary. These are removed.
3. Improved consistency and code indentation
4. Removed comments that dont align with implementation
5. Grouped headers of a workspace together
2023-10-01 01:37:17 +05:30
Ashwin Natesan
c759935aad svcenc: Handling frames with few MB's in In-frame RC
A 'consumption ratio' is computed fr every MB when
in-frame RC is enabled. This computation can result in
divide-by-zeros in certain circumstances. Such cases
are now appropriately handled.

Bug: ossFuzz:60828
Test: svc_enc_fuzzer
2023-07-25 08:06:32 -07:00
Ashwin Natesan
5beaa86db4 svcenc: nullSan errors in ILPMv fixed
The ILP MV struct pointer will be set to NULL for I slices and
for spatial layer ID 0. A NULL check ought to be used in all places
that access this pointer. This was missing in 2 places and has been
added.

Test: svc_enc_fuzzer
2023-03-21 10:17:44 -07:00
Ashwin Natesan
c5f3b0c976 svcenc: RC bitrate overflows prevented
RC uses int32_t for storing multiple quantities related to
bits in a given period.
'isvce_svc_frame_params_validate' has been added, which queries
RC API for the relevant quantities and returns with and error
if computations involving those quantities exceed INT32_MAX.

Bug: 274221347
Bug: 274265498
Test: svc_enc_fuzzer
2023-03-19 23:02:06 -07:00
Ashwin Natesan
ab835f3240 svcenc: fixed incorrect EPB checks
If 'u4_strm_buf_offset < 4' when 'isvce_cabac_flush' is called,
then EPB could not have been inserted into the stream buffer.

BUG = ossfuzz:56816
Test: svc_enc_fuzzer
2023-03-09 06:51:21 -08:00
Ashwin Natesan
ed00dfc325 svcenc: fixed a bug in low bitrate encodes
Certain ILP buffers are reset when encoded bitrate exceeds
configured bitrate. The size used for one of them was incorrect.
2023-02-20 08:03:08 -08:00
Ashwin Natesan
c38af025ab svcenc: fixed incorrect streamBuf accesses
If 'u4_strm_buf_offset == 0' when 'isvce_cabac_flush' is called,
then 'carry' is implicitly 0.

'u4_strm_buf_offset == 0' implies stream buffer is empty.

Invocation of 'isvce_cabac_flush' implies all MB's in a slice
have been encoded.

BUG = ossfuzz:56137
Test: svc_enc_fuzzer
2023-02-20 08:02:26 -08:00
Ashwin Natesan
47125f49a4 svcenc: increased minimum outbuf size
For certain configurations of the fuzzer input, the minimum
output buffer size is insufficient.
The minimum value has been doubled in this commit.

BUG=oss-fuzz:55797
Test: svc_enc_fuzzer
2023-02-13 10:06:08 -08:00
Ashwin Natesan
43629a63d8 svcenc: fixed riscv4 soong build errors
riscv4 soong builds were failing due to incorrect usage of
'ARM' macro. This was fixed.
2023-02-07 07:09:18 -08:00
Ashwin Natesan
d6441bc469 Fixed OOB access of residue buf attributes in some cases
Fixed incorrect access of residue buf attributes when
'isvce_rc_post_enc' returns 'u1_is_post_enc_skip=1'.
Also, fixed incorrect memory initialisations for bitstream_t.

BUG=oss-fuzz:55640
Test: svc_enc_fuzzer
2023-02-05 08:57:41 -08:00
Ashwin Natesan
f89d401064 Fixed incorrect outbuf size assignment in 'isvce_get_buf_info'
BUG=oss-fuzz:55639
BUG=oss-fuzz:55643
Test: svc_enc_fuzzer
2023-02-05 08:52:38 -08:00
Ashwin Natesan
bb0f31cb6b Encoder: SVC encoding support added
Added support for encoding 'Scalable Baseline' profile, corresponding to
profile_idc of 83 in 'Rec. ITU-T H.264 (11/2007)'.

Bug: 248891908
Test: svcenc -c enc.cfg
Change-Id: Ib12ca4c4a8c0e674738ae2af01558a08cefe0929
2023-01-30 11:07:10 -08:00