Skip to content
#

llmengineering

Here are 6 public repositories matching this topic...

Language: All
Filter by language

A RAG system that replaces standard BM25/FAISS retrieval with a fully learned neural retrieval stack - including a fine-tuned bi-encoder, a cross-encoder reranker, ColBERT-style late interaction scoring, and a locally hosted LLM generator. Built entirely with free and open-source tools.

  • Updated Apr 3, 2026
  • Python

This repository is a practical overview of inference engineering for Large Language Model deployment, with vLLM as the serving engine. It is meant to help you understand not only how to start a model server, but also how to reason about throughput, latency, batching, quantization, KV cache memory, and GPU VRAM requirements before deploying a model.

  • Updated Jul 14, 2026
  • Jupyter Notebook

Add this topic to your repo

To associate your repository with the llmengineering topic, visit your repo's landing page and select "manage topics."

Learn more