Skip to content

fix(indexing): partial settings rollback, clone cancellation & 409 lock guard, minified input halving retry, and pipeline concurrency (#183, #184, #185, #186) - #187

Merged
spelech merged 2 commits into
mainfrom
fix/deployment-issues-183-184-185-186
Nov 20, 2026
Merged

spelech merged 2 commits into
mainfrom
fix/deployment-issues-183-184-185-186

Conversation

@spelech

@spelech spelech commented Nov 20, 2026

Copy link
Copy Markdown
Owner

Summary

Resolves four issues discovered during deployment testing on v2.28.1:

  1. Partial Embedding Settings Update & Atomic Rollback on Failure (#183):

    • Changed EmbeddingSettingsRequest.provider default from "local" to None so partial payloads (e.g., {"api_concurrency": 8}) retain the saved provider and model.
    • Updated update_embedding_config and init_embeddings to initialize candidate models/clients before persisting to system_metadata and to snapshot/restore all module globals and active clients if initialization fails.
  2. Clone Subprocess Cancellation, DB Status Sync, and HTTP 409 on Overlapping Sync (#184):

    • Added _run_cancellable_git_cmd in app/services/git_manager.py so shallow_clone_repo polls progress_tracker.is_cancelled(repo_id) and terminates/kills the git clone or git fetch subprocess immediately on cancel.
    • Added _mark_repo_cancelled in app/services/indexing/git_syncer.py so cancelling a sync at any stage updates git_repositories.status = 'error' in the database to stay consistent with sync-status.
    • Added is_repo_sync_running(repo_id) and updated POST /admin/api/repos/{repo_id}/sync (api_sync_repo) to return HTTP 409 Conflict (status: "already_running") when the per-repository sync lock is already held.
  3. Dense/Minified Input Token Estimation & Single-Input HTTP 400 Halving Retry (#185):

    • Added _chars_per_token_for_text in app/services/embedding_batcher.py to use a conservative 2 chars/token ratio for dense/minified files (high punctuation ratio or long lines such as minified CSS/JS, HTML fixtures, or literal arrays) during pre-batch truncation.
    • When a single isolated input still fails with an HTTP 400 input-length error in execute_api_embeddings_batch, truncates the input in half and retries up to 3 times before skipping so the file chunk is embedded instead of dropped.
  4. Shared Concurrency Preservation Across Pipeline Slices (#186):

    • Made EmbeddingTelemetryAndPacer.get_effective_concurrency non-clamping on _current_concurrency (min(self._current_concurrency, max_c)) and added set_target_concurrency on init_embeddings.
    • Updated ChannelIngestionPipeline to track active_futs across batches and pass max_concurrency=max_conc to execute_api_embeddings_batch so api_concurrency requests run in parallel without clamping active_concurrency to 1.

Closes #183
Closes #184
Closes #185
Closes #186

Verification

  • Added tests for all four issues in tests/test_embedding_input_limits_and_pipeline_recovery.py and verified all existing embedding, git manager, sync progress, and 500-LOC guardrail test suites pass.

Antigravity Agent added 2 commits November 20, 2026 13:45
…ck guard, minified input halving retry, and pipeline concurrency (#183, #184, #185, #186)
@spelech
spelech merged commit 52fa9d5 into main Nov 20, 2026
15 checks passed
@spelech
spelech deleted the fix/deployment-issues-183-184-185-186 branch November 20, 2026 20:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment