When transquant bypass is enabled in HBD streams, decoded frames
mismatched the reference output due to two issues:
1. Intra boundary filtering:
In the 8-bit path, boundary filtering for horizontal (mode 10) and
vertical (mode 26) intra predictions is disabled when transquant bypass
is active with implicit RDPCM. However, the HBD reconstruction path was
passing raw u1_luma_pred_mode instead of disable_boundary_filter, and the
HBD filter functions ignored the mode parameter.
2. SAO Scratch Buffer Overflow:
During shifted CTB SAO processing, transquant bypass blocks are temporarily
backed up into pu1_tmp_buf_luma and restored afterwards to keep them lossless.
In HBD, each sample requires 2 bytes (UWORD16), but the static scratch buffer
was allocated for 8-bit samples (sizeof(UWORD8)).
Because the shifted CTB processing window spans beyond standard CTB boundaries
to handle delayed neighbor boundaries, the 16-bit luma backup exceeded the
allocated buffer capacity and overflowed into pu1_tmp_buf_chroma. Consequently,
chroma backup overwrote the luma backup data, resulting in corrupted samples
upon restoration.
This commit resolves the above issues with the following changes:
- common/ihevc_hbd_intra_pred_filters.c: Check 'disable_boundary_filter' to bypass
boundary filtering in ihevc_hbd_intra_pred_luma_horz and ihevc_hbd_intra_pred_luma_ver.
- decoder/ihevcd_iquant_itrans_recon_ctb.c: Pass disable_boundary_filter
to apf_hbd_intra_pred_luma for modes 10 and 26.
- decoder/ihevcd_api.c: Scale SAO temporary scratch buffer allocation
(pu1_tmp_buf_luma and pu1_tmp_buf_chroma) by sizeof(UWORD16) to prevent
buffer collision in HBD mode.
This commit implements HBD inverse transform residual functions, which is a
prerequisite to enable 10-bit decoding with Cross Component Prediction (CCP).
When CCP is enabled, the inverse transform and prediction addition cannot be
performed in a single pass (`itrans_recon`). Instead, the decoder must first
compute the unclipped transform residuals (`itrans_res`), scale them for CCP,
and then add the prediction. For 10-bit bitstreams, the decoder was missing
HBD implementations for `itrans_res` and improperly falling back to 8-bit
implementations. This resulted in the use of an incorrect second-stage inverse
transform shift (hardcoded to 12 instead of `20 - bit_depth`), significantly
corrupting the residuals.
Key Changes:
1. Created `common/ihevc_hbd_itrans_res.c` providing mathematically compliant
HBD implementations of `itrans_res` for all Transform Unit sizes (4x4, 8x8,
16x16, 32x32, and DC) using intermediate `CLIP_S16` logic and dynamic
bit-depth shifting (`20 - bit_depth`).
2. Implemented `ihevc_hbd_chroma_recon_nxn_ccp` to properly scale 10-bit residuals
with the cross-component alpha and add the prediction.
3. Modified `ihevcd_iquant_itrans_resi_recon_tu_plane` to dynamically branch to
`ps_codec->apf_hbd_itrans_res` and `ihevc_hbd_chroma_recon_nxn_ccp` when
`pixel_size > 1`. This cleanly handles 10-bit CCP while keeping 8-bit paths
untouched.
This commit introduces on-the-fly dynamic reallocation to safely decode
extreme edge-case bitstreams (e.g., random noise causing CABAC inflation)
that exceed standard MaxCPB size constraints.
It adds a dedicated `ihevcd_reallocate_dynamic_bitstream_buf` setup
function in `ihevcd_api.c`. If an incoming bitstream exceeds the currently
allocated internal buffer during parsing (`ihevcd_decode.c`), the decoder
will cleanly reallocate the buffer. This prevents truncation failures and
memory corruption for highly incompressible streams.
This commit adds the core math and CTB integration required for
inverse transform and reconstruction of 10-bit 4:4:4 chroma blocks.
Changes include:
- Added `ihevc_hbd_chroma_itrans_recon_32x32.c` to support the
maximum 32x32 chroma block size required by the 444 profile.
- Added 10-bit chroma recon functions in `ihevc_hbd_chroma_recon.c`.
- Updated CTB decoding in `ihevcd_iquant_itrans_recon_ctb.c` to
dynamically calculate 10-bit QP indices for 444 chroma blocks while
preserving the original 420 fallback behavior.
- Updated headers with new HBD function declarations.
Remove vestigial HEVC_DEC_PROFILE_T enum and e_profile member from codec_t, along with its hardcoded initialization in ihevcd_init and assignment block in ihevcd_parse_sps. All dynamic bit depth, pixel size, and function selection logic in HBD relies directly on bitstream syntax elements.
TAG=agy
CONV=af8f01f3-0862-4fb5-a5ce-b37c3025b633
- Rename pf_hbd_ prefixed function pointers in func_selector_t to use _fptr suffix consistently.
- Update usages of renamed function pointers.
- Clean up HBD function pointer tables (remove ihevcd_hbd_func_tables.c/h).
- Implement HBD function pointer arrays in codec_t.
TAG=agy
CONV=51cc5812-336f-4b5f-990d-34fbf4d5ef08
Remove the unused function pointer pv_parse_residual_coding from
codec_t and its initialization in ihevcd_init. The function
ihevcd_parse_residual_coding is already called directly.
TAG=agy
CONV=9b39e0bf-8a07-4122-9bb2-81ec77fbdc09
Remove the function pointer pv_iquant_itrans_recon_ctb from codec_t
and call ihevcd_iquant_itrans_recon_ctb directly. This removes
unnecessary indirection since there is only one implementation.
TAG=agy
CONV=9b39e0bf-8a07-4122-9bb2-81ec77fbdc09
- Update ihevcd_parse_sps to parse profile dynamically from clip SPS (profile_idc, chroma format, bit depth, intra and still constraint flags).
- Add support for Main, Main 10, Main 10 Still, Main 422 10, Main 444, Main 444 10, Main Intra, Main 10 Intra, Main 422 10 Intra, Main 444 Intra, Main 444 10 Intra, and Main 444 Still Picture profiles.
- Allow profile_idc == 4 (RExt) in SPS profile validation checks.
- Update pixel size determination in ihevcd_api.c for 8-bit vs 10-bit profiles.
- Add CLI argument parsing for new profiles in main.c.
This is the initial version and doesn't enable SIMD functions yet.
Has not been tested extensively yet.
This is not ready to be merged into main branch yet.
Corrected the static SAO buffer size calculation to match the
partitioned buffer bounds. Luma requires 4x and Chroma requires 8x
MAX_CTB_SIZE * MAX_CTB_SIZE, which is now correctly summed to allocate
12x MAX_CTB_SIZE * MAX_CTB_SIZE bytes per process thread.
Bug: 484436016
Test: ./hevc_dec_fuzzer
Change-Id: I40411c04ab00d7e23843eb1d033d4943e3ec76a9
- reduce unused memory allocations
- revert unrequired alignments done to widths and height in format
conversion leaf functions
- pass correct strides and base address to leaf functions
fixes oss-fuzz-520748344
Change-Id: I8b783d1dae8c53b68a6d3a32dbc3ffb16376851e
Reverted some of the unnecessary changes from commit: f6ef16b0f9
- Restored the format conversion structure prior to f6ef16b0f9
- No need to handle odd stride for 420 and 422 output.
- Dimensions for output format 444 and 400 format can be odd
Tested with few YUV444 and YUV400 odd dimension clips
To generate:
$ ffmpeg -f lavfi -i testsrc=s=511x511 -c:v libx265 -pix_fmt yuv444p -t 1 -tag:v hvc1 y444_511x511.hevc
$ ffmpeg -f lavfi -i testsrc=s=511x511 -c:v libx265 -pix_fmt gray -t 1 -tag:v hvc1 y400_511x511.hevc
To decode using hevcdec for YUV420 output format
$ hevcdec -i y444_511x511.hevc -o y444_511x511.yuv --save_output 1 --num_frames -1 --chroma_format YUV_420P --enable_yuv_format 31
$ hevcdec -i y400_511x511.hevc -o y400_511x511.yuv --save_output 1 --num_frames -1 --chroma_format YUV_420P --enable_yuv_format 31
To decode using hevcdec for YUV444 output format
$ hevcdec -i y444_511x511.hevc -o y444_511x511.yuv --save_output 1 --num_frames -1 --chroma_format YUV_444P --enable_yuv_format 31
To decode using hevcdec for YUV400 output format
$ hevcdec -i y400_511x511.hevc -o y400_511x511.yuv --save_output 1 --num_frames -1 --chroma_format GRAY --enable_yuv_format 31
Display resulting output using ffplay by passing `-f rawvideo -pixel_format yuv420p -video_size 511x511`
Change pixel_format to yuv420p/yuv444p/gray based on decoder chroma_format argument
Decoding streams with odd resolutions previously caused integer truncation
during right-shift or division operations, leading to incorrect buffer size
and stride calculations.
This is fixed by applying the ALIGN2 macro to width and height across buffer
metrics, dimension queries, and outargs to safely pad boundaries.
Test: ./hevcdec
Change-Id: I5c432aa63b0975b76baa5e59a816278087b65215
- updated parser for 422 specific syntax elements
- updated iqitrecon module to handle 422 subtu's. Verified functionality
for intra only slices
- add support 422 i/o
- restrict combinations of output chroma format and internal chroma
format allowed. That is, for a given chroma format idc not all output
color formats are supported. The output format can now be either same
as internal color format or 420p.
Test: ./hevcdec
Change-Id: I9dbf5c141aceb8ffcbe063aa35856eeed11e3d4f
while computing chroma residue, the same buffer is used for both cb and
cr. So remove allocation made for cr. Further guard the memory
allocation for rext toolsets under the macro ENABLE_MAIN_REXT_PROFILE
Test: ./hevcdec
Change-Id: I773c9413d262a4b84bcc7ec71ac11ceaadfd839d
These functions are similar to ihevc_itrans_recon_* with the exception
that instead of computing recon directly they store residue to the
destination buffer. This helps with additional transformation of the
residue coefficients (rotation, rdpcm, ccp) before adding to the
prediction data to generate the final recon.
Test: Build
Change-Id: I9f84e66f5e9595162335cb152a6329a0f8d23337
This change adds supports for handling color format yuv444 at input
output level. At the input level, it ensures the bitstream buffer
allocation is modified to cover 444 encoded streams and at the output
level it ensures the allocation is such that decoded output can be
written to 444 format.
Test: ./hevcdec
Change-Id: Icac7ca76a95c4140d7ef11189cdb6321961d88cc
These color formats are no longer supported. The decoder now returns
an error if configured for any of these output formats
Test: ./hevcdec
Change-Id: I944c539000f26898d5985cd6806f2968d4365f63
Updated library to allocate memory only for luma in case of monochrome
profile.
Bug:
Test: ./hevcdec
Change-Id: I5649bbd94e10d2c19fa7548b2380db598c9ff2b6
After flush, if the first slice NAL sent to decoder is a CRA
picture, then it may have associated RASL NALs that need to
be skipped as they may contain references to pictures that
are not present in the bitstream that is fed to the decoder.
Currently decoder was handling this case only at the start
of decode and after receiving an EOS NAL. Now it is updated
to ignore such RASL NALs associated with CRA after flush as
well.
This is needed to handle seek in clips, where CRA NALS are
marked as sync points.
Bug: 194768162
Test: poc in bug
Test: atest VtsHalMediaC2V1_0TargetVideoDecTest
Test: atest CtsMediaV2TestCases:CodecDecoderTest
Change-Id: If5f38b0689c98c4d2a2a720b81d7e28593be5732
Current design of HEVC Decoder creates new threads
to process each frame. This CL adds support to reuse
the threads across frames.
Test: manual
Bug: 167126301
Change-Id: Ic57133b25203dae9ed5d3d974e6137a733531da7
This will ensure clips with supported bitdepth or unsupport color
formats are not attempted to be decoded till end of the sequence.
Bug: 129796771
Bug: 130578045
Test: adb shell stagefright -s -S /sdcard/crowd422_hevc.mp4
Test: Ensure decoder doesn't attemp decoding all frames
Change-Id: I7c3cd58a0e231e7e64c278564d050086a079e12c
If memory allocation for ps_codec_obj fails, return gracefully
with an error code. All other allocation failures are
handled correctly.
Bug: 68299873
Test: before/after with always-failing malloc
Change-Id: I5e6c07b147b13df81e65476851662d4b55d33b83
