Span-based safety classification (GliGuard / GuardReasoner-style)

[adaamko]

Problem

LettuceDetect's core competence is token-level span detection over long inputs: which spans of an answer are unsupported, and what kind of error each one is. The same stack should transfer to safety / guardrail classification: instead of "which spans are unsupported", predict "which spans are unsafe" and their category. Span localization is more actionable than the sequence-level safe/unsafe verdict most guard models emit — a moderation or agent pipeline can quote, strip, or redact the offending span instead of discarding the whole message.

Nothing in the architecture is hallucination-specific; only the data is:

• Binary span tagging: scripts/train_span_detector.py trains a token classifier on any data in the HallucinationSample schema (prompt tokens -100, target tokens 0/1 — see the docstring at scripts/train_span_detector.py:1-21).
• Typing without a fixed head: the taxonomy head matches span embeddings against label descriptions by cosine similarity, so "labels enter only as text" and the category count is not a fixed-head decision (lettucedetect/detectors/taxonomy_head.py, module docstring; trained by scripts/train_taxonomy_head.py).
• LLM baseline for free: LLMDetector accepts a custom taxonomy as a {category: description} dict (lettucedetect/detectors/llm.py:88-94, wired at llm.py:119-121), so a safety label set works today with zero code changes.
• Shared metrics: scripts/span_eval_metrics.py (stdlib char-overlap span metrics) and scripts/evaluate_span_model.py (per-source / per-language tables).

Current behavior

No safety classifier exists in the repo. The main blocker is data: existing safety datasets (Aegis 2.0, GuardReasoner training mixes) are example-level, so span supervision has to be generated first — that is the companion issue #44. This issue covers the modeling and evaluation once (pilot) data exists.

What to do

1. Take the span-level safety data from Generate span-level safety supervision from Aegis 2.0 #44 — or build a small pilot slice yourself following that issue; the two can proceed together.
2. Define the safety label set as {category: description} text (Aegis or GuardReasoner-style categories are reasonable starting points). Descriptions are load-bearing: the typing head and the LLM baseline both consume them.
3. Establish the cheap baseline first: LLMDetector(include_taxonomy={...your safety categories...}) over the test slice, scored with scripts/span_eval_metrics.py.
4. Encoder path: fine-tune a binary unsafe-span tagger with scripts/train_span_detector.py on the generated data, then train a safety variant of the typing head with scripts/train_taxonomy_head.py using the safety label descriptions.
5. Evaluate span char-F1 and example-level F1 overall and per category. Also report the example-level safe/unsafe accuracy derived from the spans (any span → unsafe), so results are comparable to sequence-level guard models.
6. Write up what worked and what did not, including annotation-quality caveats inherited from Generate span-level safety supervision from Aegis 2.0 #44.

Acceptance

This is a research issue: experiments, design notes, and negative results are all valid contributions. Concretely, any of:

• an eval table (span F1 / example F1 per category) for the LLM baseline and/or a pilot encoder on a held-out slice;
• a design note on label-set granularity with measured confusion between categories;
• a negative result with analysis (e.g. "unsafe spans are too diffuse for token tagging on category X").

For code that lands: new scripts go under scripts/ with descriptive verb-phrase names and docstring usage blocks (mirror scripts/evaluate_span_model.py:1-14); data-prep and metric-glue functions are unit-tested in tests/test_safety_spans_pytest.py (pattern required by tests/pytest.ini); python -m pytest tests/test_safety_spans_pytest.py -v passes; python tests/run_pytest.py stays green.

Non-goals

• No shipped guard product, API endpoint, or release commitment — exploratory research.
• No new fixed safety taxonomy baked into the core package; labels stay in data and configuration.
• No dataset generation work here — that is Generate span-level safety supervision from Aegis 2.0 #44.
• Prompt-injection / jailbreak localization is Span-level prompt-injection / jailbreak detection #55.

Start here

git clone https://github.com/KRLabsOrg/LettuceDetect.git
cd LettuceDetect
pip install -e ".[dev]"
python tests/run_pytest.py   # should be green before you change anything

# The code to read first:
sed -n '1,10p' lettucedetect/detectors/taxonomy_head.py   # label-conditioned typing
grep -n "include_taxonomy" lettucedetect/detectors/llm.py | head
sed -n '1,21p' scripts/train_span_detector.py             # binary tagger entry point