Problem
The savings report needs a traffic file ({"text", "llm_label"} JSONL), but there's no helper to produce one. Observation mode already captures exactly this data (chef.start_observing(...) / add_observation), it just never lands on disk in the traffic format — users would have to write their own dump loop.
Current behavior
Observed calls are stored as buffer examples / observations inside RuleChef (see rulechef/engine.py start_observing, add_observation, and the observer in rulechef/observations.py if present). Nothing exports them as JSONL.
What to do
- Add
chef.export_traffic(path) (or dump_observations) that writes observed input-output pairs as the savings-report JSONL: {"text": ..., "llm_label": ...} for classification, {"text": ..., "llm_entities": [...]} for NER.
- Document the loop in
benchmarks/INSPECTING_RULES.md: observe -> export_traffic -> rulechef-savings.
Acceptance
A test that adds a few observations, exports, and round-trips the file through rulechef-savings (see tests/test_reports.py::test_savings_cli for the invocation pattern).
Start here
git clone https://github.com/KRLabsOrg/rulechef.git && cd rulechef
pip install -e ".[dev,grex]" && pytest tests/ -q
grep -n "def start_observing\|def add_observation" rulechef/engine.py # where observations enter
Problem
The savings report needs a traffic file (
{"text", "llm_label"}JSONL), but there's no helper to produce one. Observation mode already captures exactly this data (chef.start_observing(...)/add_observation), it just never lands on disk in the traffic format — users would have to write their own dump loop.Current behavior
Observed calls are stored as buffer examples / observations inside RuleChef (see
rulechef/engine.pystart_observing,add_observation, and the observer inrulechef/observations.pyif present). Nothing exports them as JSONL.What to do
chef.export_traffic(path)(ordump_observations) that writes observed input-output pairs as the savings-report JSONL:{"text": ..., "llm_label": ...}for classification,{"text": ..., "llm_entities": [...]}for NER.benchmarks/INSPECTING_RULES.md: observe -> export_traffic ->rulechef-savings.Acceptance
A test that adds a few observations, exports, and round-trips the file through
rulechef-savings(seetests/test_reports.py::test_savings_clifor the invocation pattern).Start here