Research spike: zero-shot token classification via label conditioning

[adaamko]

Problem

GLiNER-family models give zero-shot extraction by conditioning on label text, but they score enumerated candidate spans with a maximum span width tuned for entity-scale mentions. Hallucination and safety spans are sentence-scale and often discontinuous-adjacent, so span enumeration does not transfer: what we need is a zero-shot token classifier, where labels are free text and each token is scored against each label embedding (BIO or binary per label).

If this works, one compact encoder could handle unseen span taxonomies (new hallucination categories, new safety dimensions such as the ones in #43/#44/#55) without retraining, the same way GLiNER handles unseen entity types.

Direction

• Encode label descriptions with the same encoder (bi-encoder) or jointly with the input (cross-encoder prompt packing, GLiNER-style), then score per token against each label representation instead of enumerating spans.
• Start from a compact multilingual encoder and the typed-span data already in this repo's ecosystem; train on one set of span types, evaluate zero-shot on a held-out label set.
• Compare against GLiNER/GLiClass baselines run within their max-span-width limits, and against the supervised per-label ceiling.

Acceptance

• A prototype training/eval script and a results table showing zero-shot performance on span types never seen in training, with the GLiNER baseline comparison.
• A short design note on what conditioning scheme won and why. Reproducible negative results are a valid outcome.

Non-goals

• A production API or a committed model release. This is an experiment.