[rootvector2]

Description

QuantValsToDistance (lib/jpegli/quant.cc) looks up a reference matrix with kBaseQuantMatrixYCbCr[quant_tbl_no * DCTSIZE2], but that table only defines base matrices for the three standard YCbCr tables (indices 0..2). A component's quant_tbl_no is validated only to 0..NUM_QUANT_TBLS-1 (0..3), so a component that legally references quant table 3 during a YCbCr encode with adaptive quantization (the default) makes the lookup point one 64-float block past the end of the array, and the per-coefficient loop reads kBaseQuantMatrixYCbCr[192..255] out of bounds (CWE-125).

This is reachable through the public encoder API via jpegli_start_compress and InitQuantizer; the transcode (jpegli_write_coefficients) path skips InitQuantizer and is not affected. The fix clamps the base-matrix index to the three available matrices, leaving behavior for the usual tables 0..2 unchanged.

Verified with a standalone AddressSanitizer reproducer of the verbatim lookup: reading kBaseQuantMatrixYCbCr[192..255] is reported as a global-buffer-overflow before the change and is clean afterwards.

Pull Request Checklist

• CLA Signed: Have you signed the Contributor License Agreement (individual or corporate, as appropriate)? Only contributions from signed contributors can be accepted.
• Authors: Have you considered adding your name to the AUTHORS file?
• Code Style: Have you ensured your code adheres to the project's coding style guidelines? You can use ./ci.sh lint for automatic code formatting.

[gemini-code-assist]

Code Review

This pull request clamps the base-matrix index to prevent out-of-bounds reads when accessing kBaseQuantMatrixYCbCr for table index 3. However, the review points out that using std::min(quant_idx, 2) introduces a bug when add_two_chroma_tables is false, as it incorrectly maps quant_idx 1 to index 1 instead of index 2, leading to mismatched base matrices and incorrect distance calculations. A code suggestion is provided to handle this mapping correctly.

[gemini-code-assist]

When add_two_chroma_tables is false (which is the case when cinfo->num_components >= 3 and cinfo->comp_info[2].quant_tbl_no == 1), SetQuantMatrices uses index 2 of kBaseQuantMatrixYCbCr to generate the quantization table for index 1 (both Cb and Cr share table 1).

However, std::min(quant_idx, 2) will evaluate to 1 when quant_idx is 1, causing QuantValsToDistance to use index 1 of kBaseQuantMatrixYCbCr instead of index 2. This leads to an incorrect distance calculation because the base matrix used for reconstruction does not match the one used for generation.

To fix this, we should map quant_idx == 1 to index 2 when add_two_chroma_tables is false.

    int base_idx = quant_idx;
    if (quant_idx == 1 && cinfo->num_components >= 3 &&
        cinfo->comp_info[2].quant_tbl_no == 1) {
      base_idx = 2;
    }
    const float* base_qm =
        &kBaseQuantMatrixYCbCr[std::min(base_idx, 2) * DCTSIZE2];