RAG diagnostics engine. Fix retrieval failures.
Why is your RAG returning wrong answers? PyVectorHound pinpoints the problem—retrieval failure, embedding mismatch, or context confusion.
from pyvectorhound import diagnose
# Analyze retrieval failures
diagnosis = diagnose(
query="What is product pricing?",
retrieved_docs=docs,
expected_answer="Pricing starts at $99/month"
)
print(diagnosis.failures) # Why did retrieval fail?
print(diagnosis.recommendations) # How to fix itRetrieval Failures:
- Queries that don't match any documents
- Documents ranked too low
- Semantic mismatch between query and content
Embedding Issues:
- Poor embedding model for your domain
- Missing domain-specific terminology
- Outdated embeddings
Context Problems:
- Retrieved documents lack critical info
- Too much irrelevant context
- Context window overflow
- Component-level analysis of each RAG stage
- Identifies exact failure points
- Suggests optimization strategies
- Tests embeddings and retrievers separately
- Provides actionable recommendations
pip install pyvectorhound- Debug RAG systems returning wrong answers
- Optimize retrieval performance
- Select better embedding models
- Tune chunking and retrieval parameters
- Monitor RAG quality in production
- A/B test retrieval strategies
from pyvectorhound import diagnose, optimize
# Diagnose a retrieval failure
diagnosis = diagnose(
query="latest security updates",
retrieved_docs=retrieved,
ground_truth="CVE-2024-12345 patch released"
)
# Get specific recommendations
if diagnosis.has_retrieval_failure:
print(diagnosis.retrieval_recommendations)
# Suggest optimization
suggestions = diagnose.optimize_retrieval(
queries=test_queries,
docs=document_collection
)| Problem | Root Cause | Solution |
|---|---|---|
| Wrong answers | Retrieval too broad | Improve chunking |
| Missing context | Retrieval too narrow | Adjust similarity threshold |
| Slow retrieval | Poor indexing | Use better embedding model |
| Ranked wrong | Semantic mismatch | Domain-specific fine-tuning |
MIT License - See LICENSE