Skip to content

Validate torch FLUX.2 LoRA trainer end-to-end on a CUDA/CPU box #2786

Description

@atomantic

Split out of #1227. The mflux (MLX) runtime is fully validated end-to-end on Apple Silicon (see docs/research/2026-07-18-lora-training-e2e-validation.md), but the torch/diffusers fallback trainer (scripts/train_flux2_lora.py) cannot train on Apple Silicon — PyTorch's MPS backend has no linear_backward for the FLUX.2 transformer Linear layers, so a LoRA backward dies at the first optimizer step (bf16 AND fp32). PortOS now fails fast on darwin with TRAINING_MPS_UNSUPPORTED and points the user at mflux.

This item: run the torch trainer end-to-end on a CUDA (or CPU) machine — its real target — to confirm the fallback path (entrypoint, STAGE: protocol, checkpoints/step-NNNNNN/ layout, optimizer.pt resume, RESULT, adapter round-trip) works there. bf16 linear_backward exists on CUDA, so the backward that fails on MPS should succeed.

Blocker: requires a CUDA (or a patient CPU) box — cannot be validated on the Apple-Silicon dev machine. Hardware/availability-gated (hence future).

Acceptance: a short (≤20-step) torch run on CUDA producing a checkpoint + adapter, resumed once, and round-tripped through scripts/flux2_macos.py --lora-paths; record the result alongside the mflux validation doc.

Metadata

Metadata

Assignees

No one assigned

    Labels

    blockedBlocked on another issue/PR — skip in autonomous claim runs

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions