Skip to content

Executable tutorial submission: LLMOps prompt regression testing #3038

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
JingzeGuo wants to merge 1 commit into
base: 2026
Choose a base branch
from
+2 −0
Open

Executable tutorial submission: LLMOps prompt regression testing #3038

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions contributions/executable-tutorial/jiaxun-jingze/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,8 @@ Prompt regression testing for LLM applications (LLMOps)

## Description

**Tutorial:** [Google Colab](https://drive.google.com/file/d/1-0P0LdPLC0fstHJVhJ_B3xGATXP8OIYL/view?usp=sharing)

We want to write a Google Colab notebook that shows how to keep an LLM application stable when its prompt or its model changes. The reader writes a promptfoo configuration with test cases and assertions, runs it against a small instruction-tuned model that is loaded locally in the notebook. The reader then breaks the prompt on purpose, watches the assertions fail, and finally wires the evaluation into a pipeline, so that a failed evaluation blocks the change.

**Relevance**
Expand Down
Loading