Awesome RAG

Study the resources. Inspect the pipeline. Build the application.

Awesome RAG is a learning workspace for understanding how retrieval-augmented generation works, one engineering decision at a time. It brings together curated reading, runnable Jupyter experiments on real documents, and planned project tracks for different frameworks and clouds.

The central idea is to follow a document from ingestion to a cited answer. Along the way, inspect what each component produces, change one decision, and measure the effect. The resources below support that journey: use them to understand a component, explore its implementation, and decide what to try yourself.

Project status: Four foundational notebooks are available alongside the resource directory. Advanced notebooks and complete cloud projects are planned. External resources are labeled separately from this repository's lessons.

Contents

• Architecture overview
• Start here
• Run the foundational notebooks
• Follow a document through RAG
• Papers and surveys
• Courses and tutorials
• Document ingestion and parsing
• Chunking and context preparation
• Embeddings
• Vector indexes and databases
• Retrieval and reranking
• Generation and grounded answers
• Frameworks
• Advanced RAG
• Evaluation and datasets
• Observability and production
• Cloud guides and example projects
• Notebook collections and implementations
• Our notebook and project roadmap
• Contributing resources

Architecture overview

This diagram maps the broader architecture the labs and projects explore. It includes planned capabilities; the lab catalog and roadmap identify what is currently implemented.

Open the full-size architecture diagram.

Start here

RAG retrieves information from an external collection and supplies that information as context for a model's response. A useful starting architecture has two flows:

Indexing: Documents -> Parse and clean -> Chunk -> Embed -> Store
Answering: Question -> Retrieve -> Rerank (optional) -> Build context -> Generate answer
Evaluation: Check retrieved evidence and generated answers throughout development

For a first pass, follow this reading and building sequence:

1. Understand the architecture: Read AWS's RAG introduction for an overview of the components.
2. Build a first pipeline: Follow a Haystack tutorial to connect retrieval and generation.
3. Study each stage: Work through the ingestion, chunking, embeddings, and retrieval resources below.
4. Measure quality: Use Ragas to explore retrieval and answer evaluation.
5. Experiment with improvements: Explore RAG Techniques and compare changes against a baseline.
6. Study a deployed application: Explore the Azure RAG sample, then the other cloud examples.

Run the foundational notebooks

These lessons use 5,183 real scientific abstracts, 809 development claims, and 300 separate test claims from SciFact. They search the complete corpus, use published relevance judgments, and expose the intermediate results. Each notebook runs independently from a fresh kernel after a one-time data download; no API keys or model downloads are required for the default exercises. See data provenance, licensing, and preparation.

Notebook	What you will build
01 - Documents and chunking	Corpus profiling, exact source spans, and chunking/storage comparisons
02 - TF-IDF and BM25	Full-corpus inverted indexes, scoring formulas, and ranking disagreements
03 - Context and generation	Evidence budgets, source references, evidence-removal experiments, and optional Ollama generation
04 - Retrieval evaluation	Recall, MRR, nDCG, a chunking ablation, bootstrap intervals, and actual retrieval misses

Use Python 3.11 or newer. From the repository root on Windows PowerShell:

python -m venv .venv
.\.venv\Scripts\python.exe -m pip install -r requirements.txt
.\.venv\Scripts\python.exe scripts/prepare_data.py
.\.venv\Scripts\python.exe -m notebook

On macOS or Linux:

python3 -m venv .venv
.venv/bin/python -m pip install -r requirements.txt
.venv/bin/python scripts/prepare_data.py
.venv/bin/python -m notebook

Open a notebook under notebooks/foundations/ and run its cells in order. In VS Code, select the .venv Python interpreter as the notebook kernel. Installation and initial data preparation require internet access; subsequent default runs use Python's standard library and the local corpus. The dataset download is checksum-pinned and cached, with no silent sample mode. For Windows certificate issues, see the documented download alternative.

TF-IDF and BM25 demonstrate lexical retrieval, not neural embeddings. SciFact evaluates evidence retrieval for scientific claims; it does not provide reference answers in this export. Lesson 03 defaults to evidence assembly. To enable generation, install Ollama and a model separately, then configure the optional cell. This model-dependent step is not part of the offline checks.

To validate notebook structure and execute all default cells in fresh kernels:

.\.venv\Scripts\python.exe scripts/check_notebooks.py
.\.venv\Scripts\python.exe -m unittest discover -s tests -v

On macOS or Linux, use .venv/bin/python scripts/check_notebooks.py.

The reference test report records both fixed baselines over all 300 test claims, including per-query rankings, configurations, dataset hashes, and source-code hashes. Use python scripts/benchmark.py --split train for development and --split test for final reporting after fixing your configuration.

Follow a document through RAG

Use this map to choose what to study next. The foundational notebooks above cover the first experiments; the broader experiments below guide future lessons.

Stage	Question to investigate	Experiment to build	Evidence to inspect
Ingestion	Did we preserve the information in the source?	Parse a document containing headings and a table	Extracted text, table structure, page references
Chunking	Does each passage contain enough context?	Split the same document three ways	Broken sentences, missing headings, passage lengths
Embedding	Can similar meanings be found with different words?	Compare literal and paraphrased queries	Nearest passages and relevance judgments
Indexing	What happens when a document changes?	Add, replace, and delete a source	Search results before and after each change
Retrieval	Why did the right passage miss the results?	Compare keyword, dense, and hybrid retrieval	Relevant passages found within a fixed result limit
Reranking	Can we move better evidence closer to the top?	Rerank the same candidate passages	Rank changes and added latency
Generation	Does the answer follow from the evidence?	Ask answerable and unanswerable questions	Supported claims, citations, unsupported answers
Evaluation	Did the change actually help?	Run the same question set against two configurations	Quality differences, failure examples, cost
Deployment	Can someone else reproduce the application?	Deploy a reference project and load the same corpus	Setup steps, request traces, cleanup results

The foundational notebooks share the full SciFact corpus and its development judgments to make these changes comparable. Each notebook shows its inputs, intermediate outputs, and an exercise to extend the experiment.

Papers and surveys

• Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks — Paper, 2020. Foundational work combining a retriever with a generative model.
• Dense Passage Retrieval for Open-Domain Question Answering — Paper, 2020. Learn how dense representations support passage retrieval.
• Retrieval-Augmented Generation for Large Language Models: A Survey — Survey, 2023. An overview of naive, advanced, and modular RAG approaches.
• Precise Zero-Shot Dense Retrieval without Relevance Labels — Paper, 2022. Introduces HyDE: using hypothetical documents to help retrieve real evidence.
• Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection — Paper, 2023. Studies learned retrieval and reflection behavior.
• Corrective Retrieval Augmented Generation — Paper, 2024. Explores evaluating retrieved evidence and correcting retrieval failures.

Years refer to the initial paper submissions. Read the linked versions for revisions and publication details.

Courses and tutorials

• LangChain: Chat with Your Data — Course. Study document loading, splitting, retrieval, and conversational question answering. Check the provider for current access terms.
• Haystack Tutorials — Tutorials. Guided examples for building search, question-answering, and RAG pipelines.
• LlamaIndex Documentation — Documentation and examples. Learn document ingestion, indexing, querying, and evaluation.
• Sentence Transformers: Retrieve and Re-Rank — Tutorial. Understand a two-stage retrieval pipeline using embeddings and a cross-encoder.

Document ingestion and parsing

• Docling — Library. Convert documents into structured representations for downstream retrieval workflows.
• Unstructured — Library. Partition document formats into elements that can be cleaned, enriched, and indexed.
• LlamaIndex — Framework documentation. Explore document connectors and ingestion pipelines.

Try investigating: Select a paragraph and table in the original document. Can you find both in the parsed output and trace them back to the correct page?

Chunking and context preparation

• Unstructured Chunking — Guide. Learn how document elements, size limits, and overlap affect chunks.
• Contextual Retrieval — Engineering article. Study an approach that adds explanatory context to chunks before indexing.
• RAG Techniques — Examples. Explore chunking and context-handling experiments alongside other retrieval techniques.

Try investigating: Find a question whose answer crosses a chunk boundary. Compare whether overlap or splitting by document structure makes the evidence easier to retrieve.

Embeddings

• Sentence Transformers — Documentation. Learn text embeddings, semantic similarity, and cross-encoder models.
• FlagEmbedding — Models and code. Explore the BGE embedding and reranking ecosystem.
• Massive Text Embedding Benchmark (MTEB) — Benchmark toolkit. Study embedding evaluation across tasks, languages, and modalities.

Try investigating: Write a literal query, a paraphrase, and a query containing an exact identifier. Compare what each embedding model retrieves and record its latency.

Vector indexes and databases

• FAISS — Library. Learn similarity search and indexing for dense vectors; useful for understanding the index itself.
• Chroma — Documentation. Explore storing embeddings and querying collections.
• Qdrant — Documentation. Study vector search, payload filtering, and retrieval workflows.
• Milvus — Documentation. Explore vector indexing, search, and database deployment.
• Weaviate — Documentation. Study vector and hybrid retrieval with structured object data.
• pgvector — PostgreSQL extension. Add vector similarity search alongside relational data.
• Pinecone — Documentation. Explore managed vector search and retrieval services.

Retrieval and reranking

• Sentence Transformers: Retrieve and Re-Rank — Tutorial. Retrieve candidates efficiently, then score them with a cross-encoder.
• FlagEmbedding — Implementation. Experiment with embedding models and rerankers.
• HyDE Paper — Research. Explore query expansion through hypothetical documents.
• Contextual Retrieval — Engineering article. Examine contextual embeddings, keyword retrieval, and reranking together.
• BEIR — Evaluation toolkit. Compare retrieval approaches across diverse datasets.

Try investigating: Collect one failed query for each retrieval method. Inspect whether the cause is vocabulary, missing context, filtering, or ranking before adding another component.

Generation and grounded answers

• Haystack Tutorials — Examples. Connect retrievers, prompt construction, and generators into a pipeline.
• LlamaIndex Documentation — Guides. Explore query engines and response synthesis over retrieved content.
• Ollama — Local model runtime. Explore running the generation component locally; model and hardware requirements vary.
• Azure RAG Sample — Application. Study how a document question-answering interface connects search results to generated responses.

Try investigating: Remove the passage containing the answer, then ask the same question again. Inspect whether the response acknowledges the missing evidence and whether its citations support its claims.

Frameworks

• LangChain Retrieval Documentation — Documentation. Explore retrieval components and how they fit into model-driven applications.
• LlamaIndex — Documentation. Study data ingestion, indexes, query engines, and document workflows.
• Haystack — Tutorials. Build component-based retrieval and generation pipelines.

Choose one framework for a first project. Use a shared dataset and evaluation set when comparing implementations across frameworks.

Advanced RAG

• Microsoft GraphRAG — Implementation. Explore graph-based indexing and retrieval over document collections.
• Self-RAG — Paper. Study retrieval and generation with learned self-reflection.
• Corrective RAG — Paper. Study how retrieval-quality assessment can guide corrective steps.
• ColPali — Models and code. Explore visual document retrieval over page images.
• RAG with Deep Agents — Guide. Explore agent-driven retrieval workflows.
• RAG Techniques — Examples. Explore additional approaches to query processing, retrieval, and context preparation.

Evaluation and datasets

• Ragas — Documentation. Explore context precision, context recall, faithfulness, and test-set generation.
• DeepEval — Evaluation framework. Build automated checks for LLM and RAG application behavior.
• BEIR — Datasets and toolkit. Evaluate retrieval across different domains and tasks.
• MTEB — Benchmark toolkit. Compare embedding models using task-specific evaluations.

Try investigating: Keep the questions fixed while changing one component. Compare retrieval quality separately from answer quality, and save examples where one improves while the other does not.

Observability and production

• Langfuse — Observability platform. Explore tracing, prompt management, and evaluation for LLM applications.
• Phoenix — Observability and evaluation. Inspect application traces and investigate retrieval or generation failures.
• AWS RAG Architecture Guidance — Architecture guide. Review the data processing and retrieval components of a cloud RAG system.
• Azure AI Search RAG Overview — Architecture guide. Study search-backed RAG patterns and their supporting components.

Study focus: Document updates and deletion, access-aware retrieval, caching, tracing, failure handling, deployment, and resource cleanup.

Cloud guides and example projects

AWS

• Amazon Bedrock Knowledge Bases — Official guide. Learn managed ingestion, retrieval, and generation with knowledge bases.
• Amazon Bedrock Samples — Official examples. Explore code for Bedrock capabilities, including RAG workflows.
• RAG Architecture Guidance — Official guidance. Understand the components involved in choosing an AWS architecture.

Microsoft Azure

• RAG with Azure AI Search — Official guide. Learn how search integrates with generation.
• Azure Search and OpenAI Demo — Official sample application. Explore a deployable document chat application with search-backed retrieval.

Google Cloud

• Google Cloud RAG Engine — Official guide. Explore managed RAG capabilities and data-to-retrieval workflows.
• Google Cloud Generative AI Examples — Official examples. Find notebooks and sample applications, including retrieval and grounding workflows.

Check each provider's prerequisites, supported regions, pricing, and teardown instructions before running cloud examples.

Notebook collections and implementations

• RAG Techniques — Detailed technique examples and notebook tutorials for hands-on experimentation.
• Haystack Tutorials — Guided pipeline implementations for search and RAG.
• Google Cloud Generative AI — Cloud notebooks and application examples.
• Amazon Bedrock Samples — Bedrock examples to study and adapt.
• Azure RAG Demo — An application reference for a complete cloud project.
• Microsoft GraphRAG — A graph-based implementation to explore after building a baseline pipeline.

Our notebook and project roadmap

The table distinguishes the available introductory lessons from the broader planned coverage.

Track	Planned coverage	Status
Foundations	Lexical retrieval, context assembly, and optional local generation	Available: lessons 01-03
Ingestion	Verified benchmark loading, corpus profiling, and source metadata; document parsing to follow	Available: 01
Chunking	Fixed-size word windows and overlap; document-aware strategies to follow	Basic lesson available: 01
Embeddings and indexing	Similarity, index construction, updates, and deletion	Planned
Retrieval	TF-IDF and BM25 over the full corpus; dense, hybrid, and reranked retrieval to follow	Available: 02
Generation	Context and source references with optional local LLM; robust answer validation to follow	Basic lesson available: 03
Evaluation	Published judgments, ranking metrics, ablations, and failure analysis; answer evaluation to follow	Available: 04
Advanced patterns	Query rewriting, graph, multimodal, and agentic RAG	Planned
Framework projects	Implement the same document assistant with LangChain, LlamaIndex, and Haystack	Planned
Cloud projects	Take a document assistant from local development to AWS, Azure, and Google Cloud	Planned

The notebooks explain one component at a time and expose intermediate results. Planned end-to-end projects will connect ingestion, retrieval, generation, evaluation, and deployment, with setup and cleanup instructions. Comparable implementations will use shared data and evaluation questions where practical.

The first project will be a document assistant that answers with source references and handles missing evidence. Framework variants will explore how the same pipeline is expressed in different libraries. Cloud variants will explore deployment, storage, configuration, and operations. Each project will document its choices and the results of running the shared evaluation questions.

Contributing resources

Contributions are welcome for resources, explanations, notebooks, datasets, evaluation tools, and complete projects. Start with the contribution guide for setup, quality standards, validation commands, and the pull-request workflow.

• Read the contributor agreement for authorship, licensing, attribution, and DCO sign-offs. The project-wide license is currently undecided; the agreement explains how this affects external contributions.
• Follow the code of conduct in discussions and reviews.
• Use the issue templates for bugs, resource suggestions, notebook/project proposals, and datasets. Pull requests include a review checklist.

For a resource suggestion, provide the original link, what readers will learn, the relevant RAG stage, and any access requirements or affiliation. Listings are learning references, not rankings or endorsements.