[xaleryb]

Summary

ragas has been the optional [ragas] extra since #126, but requirements.txt still listed it, so requirements.lock.txt (the CI and make dev lock, read by Dependabot alerts and Renovate) still carried ragas, the langchain packages and diskcache, with two advisories that pip-audit.yml and osv-scanner.toml had to ignore:

• PYSEC-2026-2447 / CVE-2025-69872 (diskcache 5.6.3)
• PYSEC-2026-3046 / CVE-2026-6587 (ragas 0.3.9)

This PR takes ragas and everything only it pulled in out of every lock:

• requirements.txt is main's file without the ragas/langchain block; a one-line comment points to the [ragas] extra in pyproject.toml.
• requirements-test.txt declares markdown-it-py>=3. tests/test_docs_hooks.py imports it, and it reached the lock only through ragas -> rich; without it the whole module skips.
• The three locks are regenerated with the command each header records. requirements.lock.txt loses 50 pins (49 packages; networkx has two), all of them reached only through ragas or langchain-*: ragas, langchain, langchain-anthropic, langchain-classic, langchain-community, langchain-core, langchain-openai, langchain-protocol, langchain-text-splitters, langgraph (+checkpoint, prebuilt, sdk), langsmith, anthropic, instructor, diskcache, gitpython, gitdb, smmap, pillow, networkx, scikit-network, rich, typer, shellingham, annotated-doc, appdirs, nest-asyncio, dataclasses-json, marshmallow, typing-inspect, mypy-extensions, docstring-parser, httpx-sse, httpx2, httpcore2, httpx2-jsfetch, truststore, jsonpatch, jsonpointer, orjson, ormsgpack, requests-toolbelt, sqlalchemy, tenacity, uuid-utils, websockets, zstandard. Nothing is added and no remaining pin changes version; markdown-it-py stays, now via requirements-test.txt. requirements-ci.lock.txt changes only in three # via comments. requirements-docs.lock.txt is unchanged.
• gitpython and pillow are gone from the main lock, so their --exclude-newer-package exemptions go too (certifi's stays), and tests/test_dependency_locks.py expects the new set.
• PYSEC-2026-2447 and PYSEC-2026-3046 come out of the --ignore-vuln list in pip-audit.yml and out of osv-scanner.toml. SECURITY.md keeps the reasoning in a short note: the [ragas] extra installs ragas 0.3.9 and diskcache 5.6.3, and neither affects Dokimi (DiskCacheBackend is never constructed; the vulnerable multi-modal module first ships in 0.4.2).

What stays

• eval ragas and answers generate --ragas work as before with pip install 'dokimi[ragas]' (the dev extra includes it) and exit with that install command when it is missing.
• The weekly/dispatch test-latest job now installs -e '.[ragas]' and sets DOKIMI_REQUIRE_RAGAS=1. tests/test_ragas_runtime.py::test_ragas_imports_and_evaluator_builds_from_installed_deps skips when ragas is not installed (the PR legs), and fails instead of skipping when that variable is set, so the job that installs the extra cannot lose the real-import check silently. The job installs unpinned, so it adds no manifest for Dependabot to read.
• tests/test_model_defaults.py::test_ragas_falls_back_to_the_default_judge_for_an_unsupported_provider now stubs langchain_core.language_models as well, so it runs without ragas installed instead of depending on the real package.
• .agent-eval/run.py deselects that one real-import test from regress_pass: the eval image is built from the lock, and the task workspaces start at commits older than the skip, which is why repo tasks check fix-degrade-pre failed on this PR.

Not addressed: filelock

GHSA-v4x8-3wr5-3wq8, GHSA-9vhv-jxww-xfm5 and GHSA-567q-5qvw-pf92 (filelock 3.32.7, fixed in 4.2.1) stay ignored: every litellm release requires filelock<4.0, so the fix cannot be locked. That is a separate track; SECURITY.md already explains why two of them do not apply and where the third does.

Evidence

Run on Linux in fresh venvs built from the locks (pip install --require-hashes -r requirements.lock.txt, as CI's tests job does):

• Full suite (python -m pytest -q -n 8 -p no:cacheprovider): this PR 9009 passed, 25 skipped; main 9010 passed, 24 skipped. The 9034 collected ids are the same per file; the only outcome change is the ragas real-import test, passed -> skipped.
• tests/test_docs_hooks.py: 24 passed, 20 skipped with the new lock, as on main (the 20 need python-markdown, a docs dependency). With the lock this PR had before (2d161e3) the whole module skipped.
• test-latest as the job runs it (fresh venv, pip install --upgrade -r requirements.txt -r requirements-test.txt -e '.[ragas]', DOKIMI_REQUIRE_RAGAS=1): 9010 passed, 24 skipped; it resolved ragas 0.3.9, diskcache 5.6.3, langchain-core 1.7.0.
• Ragas real-import guard: without ragas, skipped; without ragas and DOKIMI_REQUIRE_RAGAS=1, 1 failed ("DOKIMI_REQUIRE_RAGAS=1 but ragas is not installed"); with the extra installed, passed.
• pip-audit step from pip-audit.yml against the three locks: "No known vulnerabilities found, 3 ignored" (the filelock IDs). The same step on main's locks without the two removed ignores reports PYSEC-2026-2447 and PYSEC-2026-3046.
• repo tasks check .. --manifest .agent-eval/repo-eval.yaml --task fix-degrade-pre: BROKEN before the run.py change (regress_pass failed on the ragas import), 1/1 ok after.
• ruff, mypy (262 files), deptry, bandit -r . -ll, zizmor (--offline --min-severity medium), actionlint, pre-commit: clean. tests/test_dependency_locks.py and tests/test_advisory_lists.py: 23 passed. Re-running each lock's header command leaves the three locks unchanged.

Merge Danger

Door: two-way. Reverting the PR restores the ragas block and the locks as they were on main; nothing outside the repository changes.

Blast Radius: the environments built from requirements.lock.txt: CI's test, typecheck, agent-quality and docs jobs, the .agent-eval image and make dev. They no longer have ragas or langchain installed, so a source checkout set up from the lock needs pip install -e '.[ragas]' to run eval ragas. The real-import check on ragas moves from every PR to the weekly test-latest run, so a ragas or langchain release that breaks eval ragas shows up there, not on a PR. Dependabot alerts and Renovate no longer see ragas or langchain in the locks. pip install dokimi is unchanged.

[codecov-commenter]

⚠️ Please install the to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 92.75%. Comparing base (b69b15f) to head (bb93984).
⚠️ Report is 1 commits behind head on main.
❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #245      +/-   ##
==========================================
+ Coverage   92.69%   92.75%   +0.05%     
==========================================
  Files         256      256              
  Lines       32421    32414       -7     
  Branches     5422     5421       -1     
==========================================
+ Hits        30054    30066      +12     
+ Misses       1579     1562      -17     
+ Partials      788      786       -2     

see 3 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[vchamarthi]

Not approving yet: 1 item to fix before merge.

The dependency removal is sound, but CI is not green and one test looks broken by it: the tests job installs only requirements.lock.txt, which no longer contains ragas. DCO also fails (no Signed-off-by).

• Non-blocking: Add a Signed-off-by trailer (git commit --amend -s, then force-push the branch) so the DCO check passes.

  gh api repos/intel/dokimi/pulls/245/commits shows 515588c by Alexander Rybkin with no Signed-off-by in the message; the DCO check is red.

CI at send time: 4 failing, 0 pending.

[vchamarthi]

Should fix: The tests job installs requirements.lock.txt only, which no longer contains ragas, so test_ragas_imports_and_evaluator_builds_from_installed_deps will fail on the 3.11 and 3.14 legs; install the ragas extra in that job (or a hash-locked ragas lock) or gate the test on the extra.

.github/workflows/ci.yml (Install deps step in the tests job): pip install --require-hashes -r requirements.lock.txt. This PR removes ragas, langchain-core, langchain-community, langchain-anthropic and langchain-openai from that lock. dokimi/tests/test_ragas_runtime.py runs a subprocess probe from ragas.metrics import ... / RagasEvaluator(...) with no importorskip or skip marker, and assert proc.returncode == 0. The dev extra in pyproject.toml pulls dokimi[ragas], but CI does not install the dev extra. The Tests jobs for 3.11 and 3.14 had not reported when I collected, so I have not seen the failure, only read the install path.

[xaleryb]

Fixed at 4af9bc8. The lock stays without ragas, and the ragas tests no longer depend on it: the test_model_defaults fallback test now stubs langchain_core too and runs without the extra, and test_ragas_runtime skips when ragas is missing unless DOKIMI_REQUIRE_RAGAS=1, where it fails. The weekly test-latest job installs -e .[ragas] and sets that variable, so the real-import guard still runs on real packages. Fresh venv from the new lock: 9009 passed, 25 skipped (main: 9010 / 24; the one difference is that test). fix-degrade-pre deselects the test in .agent-eval/run.py, since its task base predates the skip.

[vchamarthi]

Non-blocking: Restore the documented regeneration headers and flags in requirements-docs.lock.txt and requirements-ci.lock.txt, and recompile with them, so the next regeneration does not change the files again.

On main, requirements-docs.lock.txt opens with a 7-line header giving --no-header --no-annotate and the install line; at the head it is the two-line uv autogenerated header with -c requirements.lock.txt and no --no-annotate, and the file gains about 115 # via annotation lines. On main, requirements-ci.lock.txt line 2 has --constraint=requirements.lock.txt ... --exclude-newer=P7D --exclude-newer-package=certifi=false ...; at the head it is -c requirements.lock.txt --universal --python-version=3.11 --generate-hashes with no --exclude-newer. Package pins in these two files are unchanged, so the effect is the lost 7-day cooldown on the next recompile and a header that misdescribes how the file is built.

[xaleryb]

Fixed at 4af9bc8: both locks are regenerated with the commands their own headers record. requirements-docs.lock.txt is identical to main again, and requirements-ci.lock.txt has main's header (--constraint=, --exclude-newer=P7D, its exemptions) with three # via lines changed. The four lock tests pass.

[antonwolfy]

LGTM. The lock loses exactly the ragas/langchain/diskcache closure, the ignore lists and SECURITY.md agree, and the ragas tests now run in the lock environment. One nit inline.

What I ran, on Linux at 4af9bc8:

• make ci-local: 28 passed, 0 failed, 7 skipped (test-latest and other jobs whose condition is false on a PR; matrix legs on 3.11/3.12/3.14 are not covered).
• A pin-by-pin comparison of the three locks against the base: requirements.lock.txt loses 50 pins, nothing is added, no version or hash changes; the CI and docs locks keep their pin sets. The only header change is the dropped gitpython and pillow exemptions.
• The ragas, model-defaults and lock tests in the lock environment (no ragas installed): the real-import test skips, the rest pass. DOKIMI_REQUIRE_RAGAS=1 makes it fail; any other value (true, 0, empty) skips.

Checked, no issues

• Lock closure: no remaining # via line in any lock names a removed package, pip check is clean, and no unguarded import of a removed package exists in dokimi/, scripts/ or .agent-eval.
• Advisory lists: pip-audit.yml and osv-scanner.toml carry only the three filelock IDs, and SECURITY.md documents each; test_advisory_lists.py and test_dependency_locks.py pass.
• Fallback test: test_ragas_falls_back_to_the_default_judge_for_an_unsupported_provider still reaches the production fallback branch without ragas; removing the new langchain_core.language_models stub makes it fail.