- 🔭 Forward Deployed Engineer at Future Path: I embed with customer teams, find where AI actually fits, and ship it with evals and guardrails
- 🌱 Contributing to open-source LLM routing and inference infrastructure: semantic-router, AIBrix, llm-d, llmgateway, agent-router, LiteLLM, caveman
- 💬 Ask me about RAG, agents, LLM evaluation and guardrails, and LLM gateways
- 👯 Happy to collaborate on open-source LLM infrastructure
- ⚡ Fun fact: I turn coffee and prompts into production code
- 📫 Reach me at sharmaalok705@gmail.com
A selection of my merged pull requests in other projects (bug fixes with tests), including one I co-authored in LiteLLM. The last row is three fixes the caveman maintainer adopted into a batch PR with my authorship kept:
| Project | PR | What it fixed |
|---|---|---|
| BerriAI/litellm | #46617 (co-author) | Cache the public provider and agent field endpoints, so the bundled JSON is read and parsed once per process instead of on every request |
| llm-d/llm-d | #2410 | Align the modelexpress-p2p guide's PROVIDER_NAME default with optimized-baseline, so a default install no longer applies GKE-only router resources on a non-GKE cluster |
| vllm-project/semantic-router | #5167 | Reject a zero or negative max_tokens_limit or max_n in the request params config, so a bad limit no longer rewrites every client's max_tokens or n to zero and gets each request rejected upstream |
| vllm-project/semantic-router | #5152 | Reject a Set answer that omits a declared label's probability, so a missing label is no longer read as zero routing evidence |
| vllm-project/semantic-router | #4858 | Stop mirroring a request the router rejected (for example on an unresolved credential) to the shadow model |
| vllm-project/semantic-router | #4712 | Stop tool-observability debug headers from replacing a dispatch failure with a request continue, which let a rejected request go upstream |
| vllm-project/aibrix | #2975 | Report a missing latency in the pending-load estimate, so routers stop treating a pod with an incomplete profile as free |
| vllm-project/aibrix | #3116 | Count a panicking SlowStartBatch callback as a failure, so RoleSet pod create and delete batches stop hiding panics as successes |
| theagentrouter/agent-router | #2749 | Accept reasoning_effort: high for Gemini 3 Pro models, and map medium to the right thinking level per model |
| vllm-project/aibrix | #2846 | Enforce the prefix-hash table's maxContexts cap and evict expired pods (write-up: what the review taught me about activating dead code) |
| theopenco/llmgateway | #3960 | Clamp auto reasoning effort in the gateway |
| JuliusBrussee/caveman | #1315 (3 fixes adopted) | Stop uninstall deleting a settings.json.bak it did not create, read the schema of Anthropic input_schema tools in shrink, and keep integers above 2^53 off the numeric range track so elision markers stay correct |
Deeper dive: Fixing code that was never called: wiring up eviction in AIBrix's prefix-cache index
See all merged pull requests →
Languages
AI / LLMs
Backend & Web
Data & Infrastructure
