DATA Add 488 TrustAIRLab in-the-wild jailbreak templates

[romanlutz]

Motivation

PyRIT ships a small set of hand-authored jailbreak templates, but the research community has collected far larger corpora of "in-the-wild" prompts. The TrustAIRLab "Do Anything Now" dataset (Shen et al., 2024, shen2023donotanything -- already cited in PyRIT) contains 1,405 jailbreak prompts scraped from Discord, Reddit, and open-source/website sources under an MIT license. Making these available as static, reviewed SeedPrompt YAML templates lets any PyRIT user run large-scale jailbreak evaluations without a runtime download step.

Approach

1. Curation pipeline (offline, not committed): downloaded the dataset from HuggingFace (TrustAIRLab/in-the-wild-jailbreak-prompts, config jailbreak_2023_12_25, split train, commit a10aab8). Deduplicated the 1,405 rows by normalized text, then assessed each prompt individually for conversion into a reusable template with a single {{ prompt }} Jinja slot.

2. Template conversion: for each included prompt, the natural request/question/input slot was identified and replaced with {{ prompt }}. Prompts that hard-code a specific non-removable request, are malformed/incomplete, are not reusable wrappers, contain PII/user handles, or cannot be faithfully templated were excluded. A second deduplication pass after conversion removed semantically identical templates.

3. Final counts: 1,405 raw -> 1,152 after exact dedup -> 488 included as templates (664 excluded for reasons including: not a reusable wrapper, hard-coded specific request with no coherent prompt slot, duplicate after conversion, malformed/incomplete, or content that cannot be safely redistributed).

4. Repository integration:

   • 488 YAML files in pyrit/datasets/jailbreak/templates/in_the_wild/, each named trustairlab_<sha256_prefix>.yaml with a stable content-based hash.
   • Every template declares parameters: [prompt], is_general_technique: true, data_type: text, and contains exactly one {{ prompt }} placeholder.
   • Source text containing literal braces or template-like syntax is wrapped in {% raw %}...{% endraw %} blocks to prevent Jinja interpretation of non-placeholder content.
   • Rich metadata: source_platform, source_collection, source_dataset, source_dataset_commit, source_prompt_sha256, conversion_path, and conversion_method for full provenance.

5. Attribution: the source field points to the canonical HuggingFace dataset URL. The description accurately states the prompts were "collected in the wild" by the TrustAIRLab study authors, not authored by them. authors is left empty (the original posters are anonymous community members). groups: [TrustAIRLab] credits the dataset curators. No user handles or PII are included.

6. Tests: added tests/unit/datasets/test_jailbreak_in_the_wild.py covering uniqueness, attribution fields, placeholder contract, and rendering for the new subfolder. Existing jailbreak template tests continue to pass.

Non-obvious details

• Templates that contain literal curly braces (e.g., JSON examples, code snippets, emoji shortcodes) required {% raw %} escaping to prevent Jinja from treating them as variable references. The {{ prompt }} slot is placed outside the raw block.
• Filenames use a truncated SHA-256 of the source prompt text, making them stable across regeneration and globally unique across all template subfolders (required by TextJailBreak's filename-based resolution).
• The auditable assessment artifact with per-row inclusion/exclusion decisions is kept as a session artifact outside the repository.