DeepSeek V4 Flash on AMD Strix Halo
ROCm 7.2 diagnostics, smoke tests, and one-shot setup for antirez/ds4

---

A tuned fork of antirez/ds4 for AMD Ryzen AI MAX+ ("Strix Halo") hardware. This repository does not change how DS4 runs models — it adds hardware diagnostics, smoke tests, and one-shot configuration on top of upstream's ROCm backend.

Target hardware: 128 GB Strix Halo (Ryzen AI MAX+ 395 / Radeon 8060S) with Ubuntu 24.04 HWE kernel.

What's in the box

Feature	Status	Description
Startup diagnostics	make rocm-diag	Prints gfx1151 arch, HIP/ROCm versions, memory flags, TTM/GTT limit
Misconfiguration warnings	auto	Detects non-gfx1151, old HIP, low TTM, model too close to limit
Actionable fix	auto	Prints exact sudo amd-ttm --set-pages N command
Model fit verdict	make rocm-model-fit	Pass/fail gate for DS4_TEST_MODEL
Machine-readable diag	DS4_ROCM_DIAG	key=value or JSON output for CI and bug reports
Hardware smoke test	make rocm-smoke	Allocation/copy/mapping — no weights needed
Quick bench	make rocm-bench-quick	Confirms gfx1151 kernels execute; reports bandwidth
10-step doctor	make rocm-doctor	OS, permissions, arch, ROCm, TTM, tuned, bench, model-fit
One-shot setup	misc/strix-halo-setup.sh	GRUB, modprobe, udev, tuned, ROCm install
CI guard	make ci	Strict smoke + bench + upstream sync check

Quick start

git clone https://github.com/julianmb/ds4fa.git ds4-strix-halo
cd ds4-strix-halo
bash misc/strix-halo-setup.sh        # configure system; reboot after
make strix-halo -j"$(nproc)"         # build for gfx1151
make rocm-doctor                     # verify setup
./download_model.sh rocmfpx-strix    # download DeepSeek V4 Flash ROCmFPX (~102 GB)
./download_model.sh dspark-drafter   # download DSpark draft model (~11 GB)
./run-deepseek-v4.sh                 # run 32 tok/s high-throughput server

Full step-by-step guide

1. Clone and one-shot setup (Ubuntu 24.04 HWE)

git clone https://github.com/julianmb/ds4fa.git ds4-strix-halo
cd ds4-strix-halo
bash misc/strix-halo-setup.sh     # configures GRUB/udev/tuned; reboot after

The script applies RAM-sized GRUB gttsize/pages_limit, CWSR off, modprobe.d tuning, udev GPU-access rules, render/video group membership, and the tuned accelerator-performance profile. It sizes the GTT aperture to ~90% of visible RAM. Requires Ubuntu 24.04 with HWE kernel (6.18.4+ with KFD fixes).

2. Install the toolchain

sudo apt-get update
sudo apt-get install -y \
  hipcc rocminfo rocm-smi libamdhip64-dev \
  libhipblas-dev libhipblaslt-dev librocblas-dev \
  librocwmma-dev libhipcub-dev
# rocWMMA may be missing internal headers; add a matching tree:
git clone --depth 1 --branch rocm-7.2.3 https://github.com/ROCm/rocWMMA.git /tmp/rocWMMA
sudo cp -a /tmp/rocWMMA/library/include/rocwmma /usr/local/include/

3. Build

make strix-halo -j"$(nproc)"      # builds ds4, ds4-server, ds4-bench, ds4-eval, ds4-agent for gfx1151

4. Check the hardware (no model needed)

make rocm-diag

On a healthy machine you'll see gcnArchName=gfx1151 and a TTM/GTT limit near your RAM size. On a misconfigured one you'll get a WARNING plus a SUGGESTED: sudo amd-ttm --set-pages ... line.

5. Fix the GTT limit (if warned)

sudo amd-ttm --set-pages 8126464     # 8126464 * 4 KiB ≈ 31 GiB; pick ~90% of your RAM
# or, for a single run without touching the system:
DS4_ROCM_TTM_PAGES=8126464 ./ds4 -m your-model.gguf

Re-run make rocm-diag to confirm the limit rose and the warning is gone.

6. Run the smoke and quick-bench tests

make rocm-smoke            # allocation/copy/mapping + optional real-model gate
make rocm-bench-quick      # confirms gfx1151 kernels execute; prints bandwidth

Both should print PASSED. To make CI fail on any config warning:

ROCM_SMOKE_STRICT=1 make rocm-smoke

7. Run a model

./download_model.sh                       # or drop in your own GGUF
./ds4-server ds4flash.gguf                # HTTP server
# or
./ds4 ds4flash.gguf                       # interactive CLI

If the model is larger than the GTT-mapped memory allows, use upstream SSD streaming:

./ds4 --ssd-streaming -m your-large-model.gguf

8. Capture a report for help/CI

DS4_ROCM_DIAG=./diag.txt DS4_ROCM_DIAG_JSON=1 ./ds4 -m ds4flash.gguf
cat ./diag.txt          # machine-readable profile (JSON), attach to issues

How it works

Strix Halo is a unified-memory part: the CPU and GPU share one physical RAM pool, and the GPU reaches that memory through a TTM/GTT mapping limit rather than a fixed VRAM allocation. The single biggest source of "it won't load my model" or "it OOMs" problems on this hardware is a GTT limit that is too small relative to RAM — usually caused by an oversized BIOS dedicated-VRAM carveout.

This fork adds a small, ROCm-only diagnostics layer that runs at startup (ds4_gpu_init) and answers three questions:

┌─────────────────────────────────────────────────────────────────────┐
│  1. Is this the right hardware/driver?                             │
│     → gfx1151 arch, HIP/ROCm versions, memory capabilities         │
│                                                                     │
│  2. Is there enough mapped memory?                                  │
│     → reads TTM/GTT pages_limit, compares to RAM, warns if < 75%   │
│     → cross-checks amdgpu.gttsize boot param vs live limit          │
│                                                                     │
│  3. What is the exact fix?                                          │
│     → prints: sudo amd-ttm --set-pages <N>                         │
└─────────────────────────────────────────────────────────────────────┘

All diagnostics are advisory — they print to stderr and never alter inference. Two escape hatches let you act on them:

• DS4_ROCM_TTM_PAGES — override the limit for a single run
• DS4_ROCM_TTM_AUTORAISE=1 — let the engine call amd-ttm itself (as root)

DeepSeek V4 Flash model sizes

This fork targets 128 GB Strix Halo systems running DeepSeek V4 Flash (including the new DeepSeek-V4-Flash-0731 release). After a 512 MB BIOS VRAM carveout, expect ~120 GiB usable RAM.

Model / Quant	Size	Fits in 120 GiB?	Speed	Notes
DeepSeek-V4-Flash-0731 (ROCmFP2)	~98 GB	✅	32.0 t/s	July 31 official release (TerminalBench 82.7, DeepSWE 54.4)
ROCmFPX STRIX (284B)	~102 GB	✅	32.0 t/s	High-throughput LocalMaxxing route (with DSpark draft)
UD-IQ2_XXS	~91 GB	✅	~13 t/s	Capacity proof route
IQ2_XXS	~100 GB	✅	~12 t/s	Low quality
UD-Q2_K	~110 GB	⚠️	~10 t/s	May need SSD streaming
Q4_K_M	~200 GB	❌	—	Requires multi-GPU or SSD streaming

Quantizing the New DeepSeek-V4-Flash-0731 (July 31 Release)

To quantize the newly released DeepSeek-V4-Flash-0731 to ROCmFP2 (Q2_0_ROCMFPX ~98 GB):

# One-shot conversion and quantization script
./download_model.sh v4-flash-0731

Or run manually via ROCmFPX:

# Convert HF safetensors to GGUF using ROCmFPX converter
python3 ../ROCmFPX/scripts/convert_deepseek_v4_modular.py \
  ~/.cache/huggingface/hub/models--deepseek-ai--DeepSeek-V4-Flash-0731/snapshots/9e165c30e2704aec5d9d593cce3eebd58bbef1cb \
  --outfile gguf/DeepSeek-V4-Flash-0731-BF16.gguf --deepseek4-include-mtp

# Quantize to ROCmFP2 (Q2_0_ROCMFPX) ~98 GB
../ROCmFPX/build-strix-rocmfp4/bin/llama-quantize \
  gguf/DeepSeek-V4-Flash-0731-BF16.gguf \
  gguf/DeepSeek-V4-Flash-0731-ROCMFP2-STRIX.gguf Q2_0_ROCMFPX

Check fit before loading:

make rocm-model-fit DS4_TEST_MODEL=your.gguf

BIOS guidance

Strix Halo uses unified physical memory. A large BIOS dedicated-VRAM carveout permanently removes RAM from the operating system and helps compute very little. AMD recommends a small reservation, preferably 512 MB, and a larger dynamic TTM/GTT mapping limit instead.

cat /sys/module/ttm/parameters/pages_limit      # check current limit

GTT is a dynamic mapping limit, not permanently reserved memory. Raise it with AMD's amd-ttm helper (preferred) or a kernel parameter as a fallback. See STRIXHALO.md.

Performance tuning

The one-shot setup script applies the accelerator-performance tuned profile. To verify or re-apply manually:

sudo systemctl enable --now tuned
sudo tuned-adm profile accelerator-performance
tuned-adm active   # should show accelerator-performance

Key parameters:

• GRUB: amdgpu.gttsize=<MiB> ttm.pages_limit=<4KiB-pages> ttm.page_pool_size=<4KiB-pages> amdgpu.cwsr_enable=0
• modprobe.d: /etc/modprobe.d/amdgpu_strix_halo.conf (same values)
• udev: /etc/udev/rules.d/99-amd-kfd.rules (render group access)
• Groups: user must be in render and video groups

Environment variables

Variable	Description
DS4_ROCM_TTM_PAGES	Override TTM/GTT limit in 4 KiB pages (e.g. 8126464 ≈ 31 GiB)
DS4_ROCM_TTM_AUTORAISE	Auto-raise limit via amd-ttm when model wouldn't fit (requires root)
DS4_ROCM_AUTO_RAISE_ONCE	Bound autoraise to one call per process
DS4_ROCM_DIAG	Write startup profile to file (key=value)
DS4_ROCM_DIAG_JSON	Write diag as JSON instead of key=value
DS4_ROCM_DIAG_FIELDS	basic | full (default) | all
DS4_TEST_MODEL	GGUF path for make rocm-smoke model gate
ROCM_SMOKE_STRICT	Fail smoke/CI on any config warning

Troubleshooting

Problem	Fix
Low visible memory / OOM	Raise TTM/GTT limit; keep BIOS VRAM carveout at 512 MB
Missing gfx1151	Build with make strix-halo; verify ROCm supports gfx1151
rocWMMA issues	Ensure ROCm 7.2.x + matching rocwmma package installed
Driver/KFD errors	Use Ubuntu 24.04 HWE kernel (6.18.4+) with KFD fixes

Syncing upstream

This fork stays close to antirez/ds4:

git fetch upstream
git merge upstream/main          # or rebase your Strix Halo work on top

Preferred policy: keep Strix Halo additions confined to rocm/, ds4_rocm.h, tests/rocm_smoke.c, the Makefile ROCm targets, and this repository's docs. Do not reintroduce the old ds4fa backend or RPC implementation. See FORK_NOTES.md.

Documentation

Document	Description
STRIXHALO.md	ROCm install, GRUB params, TTM priority, hardware notes
FORK_NOTES.md	Audit of what was retained/rejected from upstream

License

This project is a fork of antirez/ds4, which in turn builds on the path opened by llama.cpp / GGML. See the upstream README.md and LICENSE for full acknowledgements and the MIT license terms.