libhevc-mirror/common
Narayan Kalaburgi 0aa9a2566f libhevcdec: Fix HBD transquant bypass decoding mismatch
When transquant bypass is enabled in HBD streams, decoded frames
mismatched the reference output due to two issues:

1. Intra boundary filtering:
   In the 8-bit path, boundary filtering for horizontal (mode 10) and
   vertical (mode 26) intra predictions is disabled when transquant bypass
   is active with implicit RDPCM. However, the HBD reconstruction path was
   passing raw u1_luma_pred_mode instead of disable_boundary_filter, and the
   HBD filter functions ignored the mode parameter.

2. SAO Scratch Buffer Overflow:
   During shifted CTB SAO processing, transquant bypass blocks are temporarily
   backed up into pu1_tmp_buf_luma and restored afterwards to keep them lossless.
   In HBD, each sample requires 2 bytes (UWORD16), but the static scratch buffer
   was allocated for 8-bit samples (sizeof(UWORD8)).
   Because the shifted CTB processing window spans beyond standard CTB boundaries
   to handle delayed neighbor boundaries, the 16-bit luma backup exceeded the
   allocated buffer capacity and overflowed into pu1_tmp_buf_chroma. Consequently,
   chroma backup overwrote the luma backup data, resulting in corrupted samples
   upon restoration.

This commit resolves the above issues with the following changes:
- common/ihevc_hbd_intra_pred_filters.c: Check 'disable_boundary_filter' to bypass
  boundary filtering in ihevc_hbd_intra_pred_luma_horz and ihevc_hbd_intra_pred_luma_ver.
- decoder/ihevcd_iquant_itrans_recon_ctb.c: Pass disable_boundary_filter
  to apf_hbd_intra_pred_luma for modes 10 and 26.
- decoder/ihevcd_api.c: Scale SAO temporary scratch buffer allocation
  (pu1_tmp_buf_luma and pu1_tmp_buf_chroma) by sizeof(UWORD16) to prevent
  buffer collision in HBD mode.
2026-09-04 09:35:33 -07:00
..
arm libhevcdec: update assembly support for implicit RDPCM 2026-08-05 14:37:42 +00:00
arm64 libhevcdec: update assembly support for implicit RDPCM 2026-08-05 14:37:42 +00:00
riscv64 Remove redundant ihevc_func_selector.h 2026-07-29 15:59:08 -07:00
x86 libhevc: Update intrinsic for implicit RDPCM 2026-08-05 14:37:42 +00:00
Android.bp libhevcdec: Implement 10-bit decoding support for Cross Component Prediction 2026-08-24 05:35:06 +00:00
common.cmake libhevcdec: Implement 10-bit decoding support for Cross Component Prediction 2026-08-24 05:35:06 +00:00
ihevc_buf_mgr.c Remove redundant ihevc_func_selector.h 2026-07-29 15:59:08 -07:00
ihevc_buf_mgr.h Initial Version of HEVC decoder 2014-05-21 18:14:55 -07:00
ihevc_cabac_tables.c hevcdec: add support for chroma qp offset list 2026-06-18 06:42:01 -07:00
ihevc_cabac_tables.h hevcdec: add support for chroma qp offset list 2026-06-18 06:42:01 -07:00
ihevc_chroma_intra_pred.h Fix formatting in hbd functions 2026-08-19 18:08:51 +00:00
ihevc_chroma_intra_pred_filters.c Remove redundant ihevc_func_selector.h 2026-07-29 15:59:08 -07:00
ihevc_chroma_iquant_itrans_recon.c Remove redundant ihevc_func_selector.h 2026-07-29 15:59:08 -07:00
ihevc_chroma_iquant_itrans_recon.h Initial Version of HEVC decoder 2014-05-21 18:14:55 -07:00
ihevc_chroma_iquant_recon.c Remove redundant ihevc_func_selector.h 2026-07-29 15:59:08 -07:00
ihevc_chroma_iquant_recon.h Initial Version of HEVC decoder 2014-05-21 18:14:55 -07:00
ihevc_chroma_itrans_recon.c Remove redundant ihevc_func_selector.h 2026-07-29 15:59:08 -07:00
ihevc_chroma_itrans_recon.h Fix formatting in hbd functions 2026-08-19 18:08:51 +00:00
ihevc_chroma_itrans_recon_8x8.c Remove redundant ihevc_func_selector.h 2026-07-29 15:59:08 -07:00
ihevc_chroma_itrans_recon_16x16.c Remove redundant ihevc_func_selector.h 2026-07-29 15:59:08 -07:00
ihevc_chroma_itrans_recon_32x32.c Remove redundant ihevc_func_selector.h 2026-07-29 15:59:08 -07:00
ihevc_chroma_recon.c Remove redundant ihevc_func_selector.h 2026-07-29 15:59:08 -07:00
ihevc_chroma_recon.h libhevcdec: Implement 10-bit decoding support for Cross Component Prediction 2026-08-24 05:35:06 +00:00
ihevc_common_tables.c hevcdec: Add assemblies and intrinsics for intra prediction 2026-08-05 14:37:42 +00:00
ihevc_common_tables.h Initial Version of HEVC decoder 2014-05-21 18:14:55 -07:00
ihevc_deblk.h libhevcdec: add support for deblock filter for yuv 444 format 2026-05-13 07:22:18 -07:00
ihevc_deblk_edge_filter.c Remove redundant ihevc_func_selector.h 2026-07-29 15:59:08 -07:00
ihevc_deblk_tables.c Initial Version of HEVC decoder 2014-05-21 18:14:55 -07:00
ihevc_deblk_tables.h Initial Version of HEVC decoder 2014-05-21 18:14:55 -07:00
ihevc_debug.h Few bug fixes in loop filter 2015-07-31 01:59:55 +00:00
ihevc_defs.h hevcdec: fix incorrect error checks added in sps/pps parsing 2026-02-20 11:12:39 -08:00
ihevc_disp_mgr.c Remove redundant ihevc_func_selector.h 2026-07-29 15:59:08 -07:00
ihevc_disp_mgr.h Initial Version of HEVC decoder 2014-05-21 18:14:55 -07:00
ihevc_dpb_mgr.c Remove redundant ihevc_func_selector.h 2026-07-29 15:59:08 -07:00
ihevc_dpb_mgr.h Initial Version of HEVC decoder 2014-05-21 18:14:55 -07:00
ihevc_error.h Initial Version of HEVC decoder 2014-05-21 18:14:55 -07:00
ihevc_func_types.h Added SEI structures and reordered ihevc_structs.h contents 2015-10-09 11:44:26 -07:00
ihevc_function_selector.h Remove incorrect prefix from ihevcd_itrans_res_dc 2026-07-29 17:08:35 -07:00
ihevc_function_selector_generic.c Remove incorrect prefix from ihevcd_itrans_res_dc 2026-07-29 17:08:35 -07:00
ihevc_hbd_chroma_intra_pred_filters.c common: Remove redundant else if(nt == 32) in chroma intra ref substitution 2026-08-23 15:10:33 +00:00
ihevc_hbd_chroma_iquant_recon.c Further formatting updates in HBD functions 2026-08-19 18:59:53 +00:00
ihevc_hbd_chroma_itrans_recon.c Further formatting updates in HBD functions 2026-08-19 18:59:53 +00:00
ihevc_hbd_chroma_itrans_recon_8x8.c Further formatting updates in HBD functions 2026-08-19 18:59:53 +00:00
ihevc_hbd_chroma_itrans_recon_16x16.c Further formatting updates in HBD functions 2026-08-19 18:59:53 +00:00
ihevc_hbd_chroma_itrans_recon_32x32.c Fix formatting in hbd functions 2026-08-19 18:08:51 +00:00
ihevc_hbd_chroma_recon.c libhevcdec: Implement 10-bit decoding support for Cross Component Prediction 2026-08-24 05:35:06 +00:00
ihevc_hbd_deblk_edge_filter.c libhevcdec: Fix chroma deblocking QP derivation for YUV422 HBD 2026-08-13 01:17:59 +00:00
ihevc_hbd_inter_pred_filters.c Further formatting updates in HBD functions 2026-08-19 18:59:53 +00:00
ihevc_hbd_intra_pred_filters.c libhevcdec: Fix HBD transquant bypass decoding mismatch 2026-09-04 09:35:33 -07:00
ihevc_hbd_iquant_recon.c Further formatting updates in HBD functions 2026-08-19 18:59:53 +00:00
ihevc_hbd_itrans_recon.c Further formatting updates in HBD functions 2026-08-19 18:59:53 +00:00
ihevc_hbd_itrans_recon_8x8.c Further formatting updates in HBD functions 2026-08-19 18:59:53 +00:00
ihevc_hbd_itrans_recon_16x16.c Further formatting updates in HBD functions 2026-08-19 18:59:53 +00:00
ihevc_hbd_itrans_recon_32x32.c Further formatting updates in HBD functions 2026-08-19 18:59:53 +00:00
ihevc_hbd_itrans_res.c libhevcdec: Implement 10-bit decoding support for Cross Component Prediction 2026-08-24 05:35:06 +00:00
ihevc_hbd_padding.c Further formatting updates in HBD functions 2026-08-19 18:59:53 +00:00
ihevc_hbd_recon.c Further formatting updates in HBD functions 2026-08-19 18:59:53 +00:00
ihevc_hbd_sao.c libhevcdec: Update SAO support for 444 HBD 2026-08-31 09:39:45 -07:00
ihevc_hbd_tables_x86_intr.h [common]Add new files for HBD 2026-08-05 13:27:29 +00:00
ihevc_hbd_weighted_pred.c Further formatting updates in HBD functions 2026-08-19 18:59:53 +00:00
ihevc_inter_pred.h expose inter pred filter coefficients 2026-02-28 22:40:12 +05:30
ihevc_inter_pred_filters.c Remove redundant ihevc_func_selector.h 2026-07-29 15:59:08 -07:00
ihevc_intra_pred.h libhevcdec: Fix HBD transquant bypass decoding mismatch 2026-09-04 09:35:33 -07:00
ihevc_intra_pred_filters.c Remove redundant ihevc_func_selector.h 2026-07-29 15:59:08 -07:00
ihevc_iquant_itrans_recon.c Remove redundant ihevc_func_selector.h 2026-07-29 15:59:08 -07:00
ihevc_iquant_itrans_recon.h Initial Version of HEVC decoder 2014-05-21 18:14:55 -07:00
ihevc_iquant_recon.c Remove redundant ihevc_func_selector.h 2026-07-29 15:59:08 -07:00
ihevc_iquant_recon.h Initial Version of HEVC decoder 2014-05-21 18:14:55 -07:00
ihevc_itrans.c Remove redundant ihevc_func_selector.h 2026-07-29 15:59:08 -07:00
ihevc_itrans.h Initial Version of HEVC decoder 2014-05-21 18:14:55 -07:00
ihevc_itrans_recon.c Remove redundant ihevc_func_selector.h 2026-07-29 15:59:08 -07:00
ihevc_itrans_recon.h Initial Version of HEVC decoder 2014-05-21 18:14:55 -07:00
ihevc_itrans_recon_8x8.c Remove redundant ihevc_func_selector.h 2026-07-29 15:59:08 -07:00
ihevc_itrans_recon_16x16.c Remove redundant ihevc_func_selector.h 2026-07-29 15:59:08 -07:00
ihevc_itrans_recon_32x32.c Remove redundant ihevc_func_selector.h 2026-07-29 15:59:08 -07:00
ihevc_itrans_res.c Remove incorrect prefix from ihevcd_itrans_res_dc 2026-07-29 17:08:35 -07:00
ihevc_itrans_res.h libhevcdec: Implement 10-bit decoding support for Cross Component Prediction 2026-08-24 05:35:06 +00:00
ihevc_macros.h libhevcenc: Initial commit 2019-02-18 07:43:11 -08:00
ihevc_mem_fns.c Remove redundant ihevc_func_selector.h 2026-07-29 15:59:08 -07:00
ihevc_mem_fns.h libhevc: Direct memcpy and memset calls to enable _FORTIFY_SOURCE 2026-06-18 06:44:56 -07:00
ihevc_padding.c Remove redundant ihevc_func_selector.h 2026-07-29 15:59:08 -07:00
ihevc_padding.h Initial Version of HEVC decoder 2014-05-21 18:14:55 -07:00
ihevc_quant_iquant_ssd.c Remove redundant ihevc_func_selector.h 2026-07-29 15:59:08 -07:00
ihevc_quant_iquant_ssd.h libhevcenc: Initial commit 2019-02-18 07:43:11 -08:00
ihevc_quant_tables.c Remove redundant ihevc_func_selector.h 2026-07-29 15:59:08 -07:00
ihevc_quant_tables.h Initial Version of HEVC decoder 2014-05-21 18:14:55 -07:00
ihevc_recon.c Remove redundant ihevc_func_selector.h 2026-07-29 15:59:08 -07:00
ihevc_recon.h Initial Version of HEVC decoder 2014-05-21 18:14:55 -07:00
ihevc_resi_trans.c Remove redundant ihevc_func_selector.h 2026-07-29 15:59:08 -07:00
ihevc_resi_trans.h encoder: Update chroma modules to avoid reading an extra byte 2021-03-26 21:24:54 -07:00
ihevc_sao.c Remove redundant ihevc_func_selector.h 2026-07-29 15:59:08 -07:00
ihevc_sao.h Initial Version of HEVC decoder 2014-05-21 18:14:55 -07:00
ihevc_structs.h hevcdec: add support for chroma qp offset list 2026-06-18 06:42:01 -07:00
ihevc_tables_x86_intr.h Initial Version of HEVC decoder 2014-05-21 18:14:55 -07:00
ihevc_trans.h Initial Version of HEVC decoder 2014-05-21 18:14:55 -07:00
ihevc_trans_macros.h Initial Version of HEVC decoder 2014-05-21 18:14:55 -07:00
ihevc_trans_tables.c Remove redundant ihevc_func_selector.h 2026-07-29 15:59:08 -07:00
ihevc_trans_tables.h hevcdec: add support for transform_skip_rotation_enabled_flag 2026-03-04 12:32:03 -08:00
ihevc_typedefs.h Reduced memory requirements 2015-10-09 11:44:26 -07:00
ihevc_weighted_pred.c Remove redundant ihevc_func_selector.h 2026-07-29 15:59:08 -07:00
ihevc_weighted_pred.h Initial Version of HEVC decoder 2014-05-21 18:14:55 -07:00
ithread.c Decoder: add support for KEEP_THREADS_ACTIVE 2020-08-31 22:00:10 +05:30
ithread.h Decoder: add support for KEEP_THREADS_ACTIVE 2020-08-31 22:00:10 +05:30
iv.h Clean up HBD support before split 2026-08-07 17:25:44 +00:00
ivd.h Add support to return bit depth after decode header call 2026-08-22 02:22:06 +00:00