argus
LLM-as-optimizer for RAG pipelines — treats a retrieval cascade like a neural net and "trains" it with textual gradients until the loss plateaus.
- Python
- LLM
- RAG
- Anthropic API
An LLM computes the gradients. Argus treats a RAG pipeline like a neural network and runs a training pass on it — except the optimizer isn’t SGD, it’s a language model. One pipeline run is a forward pass; a strong model reads the intermediate artifacts plus an F1-based loss and emits a textual gradient: which knob to move, which way, and why. Apply, re-run, re-grade, repeat, until the loss plateaus.
The demo takes a deliberately mis-configured retrieval pipeline (mean F1 ≈ 0.17) and converges every knob — chunk size, hybrid retrieval, candidate count, rerank cut-off, MMR diversity, even the free-text synthesis prompt — to a good setting (mean F1 = 1.0) in a handful of legible steps, each mapping to a named retrieval failure mode.
The analogy
| Neural-net training | Argus |
|---|---|
| forward pass | one pipeline run: chunk → embed → retrieve → rerank → MMR → synthesize |
| activations | per-stage trace: candidates, scores, what got selected, the answer |
| loss | 1 − composite F1 (recall + precision, hallucination penalty) |
| gradient | the LLM’s “move knob X this way because…“ |
| optimizer step | apply the move under a trust region |
| convergence | loss plateau over a patience window |
Standing on prior art
Argus combines ideas from OPRO (trajectory of past attempts in context), TextGrad (the model’s rationale is the gradient), DSPy/MIPRO (the prompt as a first-class parameter), and Reflexion (self-critique before each step). What’s new is jointly optimizing a full retrieval cascade’s heterogeneous knobs — numeric params and free-text prompts — using intermediate pipeline artifacts as activations, with staged forward-pass caching so most steps are cheap.