Skip to content

feat(KB/SCANOSS): skip small files - #324

Merged
soimkim merged 3 commits into
mainfrom
feat/filesize
Oct 12, 2026
Merged

soimkim merged 3 commits into
mainfrom
feat/filesize

Conversation

@JustinWonjaePark

@JustinWonjaePark JustinWonjaePark commented Oct 8, 2026 •

Copy link
Copy Markdown
Contributor
  • Improvements
    • Source scanning now skips files that are 150 bytes or smaller when collecting knowledge-base hashes, reducing unnecessary processing.
    • Files with an existing download location are also excluded from knowledge-base hash collection.
    • Hash checks are cached after completion, including when no hash is found, avoiding repeated checks.

@JustinWonjaePark JustinWonjaePark self-assigned this Oct 8, 2026
@JustinWonjaePark JustinWonjaePark added the chore [PR/Issue] Refactoring, maintenance the code label Oct 8, 2026
@coderabbitai

coderabbitai Bot commented Oct 8, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration
  • Configuration used: Repository: fosslight/fosslight_source_scanner/.coderabbit.yaml
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 944a1b88-5452-4821-a9c7-992c34e9d244

📥 Commits

Reviewing files that changed from the base of the PR and between 4ee1bf2 and f2ad8e8.


📒 Files selected for processing (5)
  • src/fosslight_source/_exclude.py
  • src/fosslight_source/_help.py
  • src/fosslight_source/_scan_item.py
  • src/fosslight_source/cli.py
  • src/fosslight_source/run_scanoss.py

💤 Files with no reviewable changes (1)
  • src/fosslight_source/_help.py

Included review availability: This review used your included allowance. Your plan provides up to 1 included review per hour; 0 remain after this review.



📝 Walkthrough

Walkthrough

The source scanner adds a 150-byte threshold for small-file filtering. KB hash collection skips small files and files with download locations. SCANOSS is configured to skip files at or below the threshold.

Changes

Small-file filtering

Layer / File(s) Summary
Define the threshold and configure SCANOSS
src/fosslight_source/_exclude.py, src/fosslight_source/run_scanoss.py, src/fosslight_source/_help.py
Adds a 150-byte threshold and a file-size helper. SCANOSS receives a skip size of 151 bytes. The help text removes a blank line before the Examples section.
Skip ineligible files during KB processing
src/fosslight_source/_scan_item.py, src/fosslight_source/cli.py
KB hash collection skips files with download locations and files at or below 150 bytes. SourceItem caches an unchecked KB hash separately from an empty hash result.

Priority: ⬇️ Low

Estimated code review effort: 2 (Simple) | ~10 minutes

Change: Feature


Merge Risk | ⚪ Minimal · up to f2ad8

Merge Risk: ⚪ Minimal · up to f2ad8

No actionable merge-blocking risk is established. The only identified eligibility race depends on concurrent pathname changes outside the inspected scanner lifecycle.

Security Architecture Review

Security architecture risk: 🔵 Low · up to f2ad8

The change intentionally reduces evidence collection for very small files. Cache state remains per file, and the inspected changes do not introduce additional privileges or a new trust boundary. The external scanner’s exact cutoff behavior remains unverified.

Retained concerns
No architecture-level concerns identified.

Security review details

Security Blast Radius

  • inferred — The changed exposure is source-file eligibility and downstream origin attribution within scans using this package. Source bytes and size influence whether a KB candidate is submitted. The inspected changes do not add endpoint selection, credential authority, or tenant routing.

Trust Boundaries and Controls

  • observed — Hashing continues to resolve source paths and read files using the scanning process’s filesystem privileges. The unchanged hashing method accepts absolute paths; the new size predicate is an eligibility decision, not path confinement or authorization. A size-read OSError returns False and permits the existing hashing attempt.

Resilience and Maintainability Implications

  • inferred — The inspected transition is per-item and in memory, rather than a shared persistent-state migration. Empty hashes cannot select another file’s KB origin, and normal collection and attribution execute sequentially. Reusing or concurrently mutating a SourceItem outside that flow is not established as a supported lifecycle.

Pre-merge checks | Passed 4 | Failed 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage Warning Docstring coverage is 37.50% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 8 functions across 5 files. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check Passed The title clearly and concisely describes the main change: skipping small files during KB hashing and SCANOSS scans.
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.

  • Fix all pre-merge checks with AI
✨ Finishing Touches 💡 2
📝 Generate docstrings 💡
  • Commit to this branch
  • Create a new PR


🛠️ Fix failing CI checks 💡
  • Commit to this branch
  • Create a new PR

🧪 Generate unit tests (beta)
  • Commit to this branch
  • Create a new PR

  • Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Signed-off-by: Park Wonjae <wonjae.park@lge.com>
Signed-off-by: Park Wonjae <wonjae.park@lge.com>
Use SCANOSS skip_size instead of per-file size patterns. Skip known KB exclusions before checking file sizes and cache their ineligible state to avoid fallback hashing.

Signed-off-by: Soim Kim <soim.kim@lge.com>
@soimkim
soimkim merged commit 9452e0d into main Oct 12, 2026
7 of 8 checks passed
@JustinWonjaePark
JustinWonjaePark deleted the feat/filesize branch October 12, 2026 01:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

chore [PR/Issue] Refactoring, maintenance the code

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants