custom-llama-sh

Small helper script for llama.cpp (llama-server) for bash-compatible shells on macOS, Linux, and Windows.

What it does

• Lists locally cached Hugging Face GGUF models in a compact repo-and-filename view by default
• Starts a local GGUF model with the default llama-server flags (see below)
• Starts directly from Hugging Face via -hf with the same default flags
• Tests whether a model fits in device memory via llama-fit-params, printing the fitted CLI arguments by default (append -fitp on for the memory estimate) and exiting
• Enables a safe built-in tool subset by default
• Auto-loads a sibling mmproj file for start when one is found
• Auto-loads a sibling mtp-*.gguf draft model for start when one is found
• Can preview and remove a cached model with a confirmation prompt
• Checks cached Hugging Face repos for newer commits and reports available updates
• Can clean old cached files before re-downloading a repo

Script: ./llama-models.sh

The executable is intentionally kept as a small entrypoint. Its Bash implementation is split into source-only modules under lib/ for platform/path handling, model discovery, llama.cpp invocation, attachment discovery, commands, and dispatch. Run the entrypoint rather than sourcing the individual modules directly.

Requirements

• llama-server available in your PATH
• Hugging Face cache present (usually ~/.cache/huggingface/hub)
• Bash 3.2 or newer (bash --version), plus the usual tools (find, sed, tr, basename, dirname, readlink, curl; optional: jq)

The script is a Bash script even when launched from a zsh prompt. Use ./llama-models.sh ...; do not run it as zsh llama-models.sh ... or sh llama-models.sh ....

Windows notes:

• Run the script from 64-bit Git Bash (MINGW64), MSYS2, Cygwin, or WSL. Git Bash must have Bash and find, sed, tr, basename, dirname, and readlink available.
• Git Bash paths such as /c/Users/... are supported. Native C:/Users/... paths are normalized when passed to the wrapper.
• llama-server must also be invokable from that same Bash environment; set LLAMA_SERVER_CMD to its path if it is not on PATH.
• install creates a symlink, which may require Windows Developer Mode or an elevated shell. If symlink creation is unavailable, run the script directly instead of using install.

Install / uninstall

Create a symlink in your home directory after confirming the target path:

./llama-models.sh install

By default this installs to ~/llama-models.sh.

Use a custom symlink path if you prefer:

./llama-models.sh install ~/bin/llama-models

Remove the symlink later:

./llama-models.sh uninstall

Or remove a custom symlink path:

./llama-models.sh uninstall ~/bin/llama-models

install always prints the resolved source and target first, then asks for confirmation before creating or replacing anything. uninstall only removes a symlink when it points back to this repo's script, and asks for confirmation before deleting it.

Usage

Show help:

./llama-models.sh help

List downloaded models:

./llama-models.sh list

This prints a compact table with the list index, repo, and filename.

Print full cached file paths as well:

./llama-models.sh list --paths
# or
./llama-models.sh list -p

Start by index from list:

./llama-models.sh start 1

Preview and remove by index:

./llama-models.sh remove 1

Start by search query:

./llama-models.sh start gemma-4-E4B-it-Q4_K_M

Start from explicit file path:

./llama-models.sh start ~/models/my-model.gguf

Pass extra llama-server args:

./llama-models.sh start 1 --port 8080 --ctx-size 8192

The wrapper fills in these defaults for both start and hf unless you already pass the corresponding flags yourself:

-ctk q8_0, -ctv q8_0, -np 1, -fa on

--jinja is enabled by default in llama-server, so the wrapper does not add it (pass --no-jinja to opt out). -ngl is not added either; llama-server's --fit (on by default) tunes it to fit device memory, so pass -ngl explicitly to pin a value. fit-params applies the same -ctk/-ctv/-np/-fa defaults but omits --tools, which llama-fit-params does not support.

Manual overrides win, so explicit args such as --cache-type-k q4_0, --cache-type-v q4_0, --parallel 2, or --flash-attn off are left untouched.

You can disable automatic MTP draft loading with:

LLAMA_AUTO_MTP=0 ./llama-models.sh start 1

The wrapper also adds this safe default tool set for both start and hf:

read_file,file_glob_search,grep_search,get_datetime

Opt out entirely by setting:

LLAMA_ENABLE_TOOLS=0 ./llama-models.sh start 1

Opt in to all built-in tools by setting:

LLAMA_DEFAULT_TOOLS=all ./llama-models.sh start 1

If a matching mmproj file is next to the resolved model, start adds it automatically. If you already pass --mmproj, the script leaves it alone.

If a matching mtp-*.gguf file is next to the resolved model, start adds it automatically as a draft model with --model-draft <mtp> --spec-type draft-mtp --spec-draft-n-max $LLAMA_DEFAULT_SPEC_DRAFT_N_MAX. If you already pass --model-draft or --spec-type, the script leaves those alone.

When using hf, MTP is not enabled automatically. Hugging Face repositories do not necessarily provide a compatible mtp-*.gguf artifact, so pass --model-draft and the desired --spec-type explicitly when MTP is supported by the repository.

remove prints the exact paths it will delete, then asks for confirmation before removing anything. The prompt defaults to y/N, so pressing Enter aborts the deletion.

You can disable automatic mmproj loading with:

LLAMA_AUTO_MMPROJ=0 ./llama-models.sh start 1

Start directly from HF repo:

./llama-models.sh hf ggml-org/gemma-4-e4b-it-GGUF --port 8080

Test whether a model fits

Test whether a downloaded (or Hugging Face) model fits in device memory using llama-fit-params. It loads the model, runs the --fit logic, prints the fitted CLI arguments (-c/-ngl chosen by --fit), and exits without serving:

./llama-models.sh fit-params 1
./llama-models.sh fit-params gemma-4-E4B-it-Q4_K_M
./llama-models.sh fit-params -hf unsloth/Qwen3.6-27B-MTP-GGUF:UD-Q4_K_XL

Append -fitp on to print the estimated memory (model/context/compute) per device instead of the fitted arguments. The wrapper applies the same KV-cache/perf defaults as start/hf (-ctk, -ctv, -np, -fa); -ngl is left unset so --fit can tune it. --tools, mmproj, and MTP auto-loading are not applied because llama-fit-params does not accept them.

To test an exact configuration (no auto-fitting), pass -fit off:

./llama-models.sh fit-params 1 -c 32768 -fit off

Check for updates

Check whether any cached repos have newer commits on their default branch:

./llama-models.sh check-updates

This prints each repo, its current cached commit, the latest remote commit, and the cached GGUF files for that repo. Different quantizations are listed separately.

Update a cached repo

To re-run llama-server -hf for a repo and fetch the latest version:

./llama-models.sh update 1
# or by repo id
./llama-models.sh update unsloth/gemma-4-12B-it-qat-GGUF

By default this does not delete your old cached files — it relies on the Hugging Face cache to add a new snapshot and reuse unchanged blobs.

To delete the old cached files for that repo first, pass --clean:

./llama-models.sh update --clean 1
./llama-models.sh update --clean unsloth/gemma-4-12B-it-qat-GGUF

The --clean mode lists the files it will remove and asks for confirmation before deleting anything.

Extra llama-server args work with both forms:

./llama-models.sh update 1 --port 8080
./llama-models.sh update --clean unsloth/gemma-4-12B-it-qat-GGUF --ctx-size 8192

Quantization selection

When using hf, append :<QUANT> to the repo id:

./llama-models.sh hf ggml-org/gemma-4-e4b-it-GGUF:Q5_K_M

Notes:

• Quant names are case-insensitive in llama-server.
• If you do not provide a quant, llama-server chooses its default (typically Q4_K_M).
• To force an exact filename from a repo, pass --hf-file:

./llama-models.sh hf ggml-org/gemma-4-e4b-it-GGUF --hf-file gemma-4-E4B-it-Q4_K_M.gguf

If the model is already cached locally, start can match by query text:

./llama-models.sh start Q5_K_M

You can also provide multiple terms (space-separated, order-independent):

./llama-models.sh start "Q5_K_M gemma 4"

If no exact token match exists, the script can fall back to the unique best local match. Example: if only Q4_K_M is cached, querying Q5_K_M gemma 4 will pick that Q4_K_M model.

Optional environment variables

• LLAMA_SERVER_CMD (default: llama-server)
• LLAMA_FIT_PARAMS_CMD (default: llama-fit-params)
• LLAMA_ENABLE_TOOLS (default: 1; set 0, false, no, or off to opt out)
• LLAMA_DEFAULT_TOOLS (default: read_file,file_glob_search,grep_search,get_datetime; set all to opt in to all tools)
• LLAMA_DEFAULT_CTK (default: q8_0)
• LLAMA_DEFAULT_CTV (default: q8_0)
• LLAMA_DEFAULT_NP (default: 1)
• LLAMA_DEFAULT_FA (default: on)
• LLAMA_AUTO_MMPROJ (default: 1)
• LLAMA_AUTO_MTP (default: 1; controls automatic MTP draft loading for start; set 0, false, no, or off to opt out)
• LLAMA_DEFAULT_SPEC_DRAFT_N_MAX (default: 2)
• HF_HUB_CACHE (explicit HF hub cache path)
• HF_HOME (uses $HF_HOME/hub)

Example:

LLAMA_DEFAULT_CTK=q4_0 ./llama-models.sh start 1