llm-server

HostAI - Android LLM API Server

Please see the newer: https://github.com/Quazmoz/openvino-windows-llm

Windows-first OpenAI-compatible local LLM server powered by OpenVINO GenAI for Intel CPU/GPU/NPU, with chat UI, model conversion, and setup scripts.

A lightweight LiteLLM server boilerplate pre-configured with uv and Docker for hosting your own OpenAI- and Anthropic-compatible endpoints. Includes LibreChat as an optional web UI.

single-executable / library which combines llama.cpp, whisper.cpp, and stable-diffusion.cpp

Function-calling API for LLM from multiple providers

macOS GUI for managing pure mlx_lm.server on Apple Silicon in Direct Mode.

A complete, menu-driven AI model interface for Windows that simplifies running local GGUF language models with llama.cpp. This tool automatically manages dependencies, provides multiple interaction modes, and prioritizes user privacy through fully offline operation.

API server for `llm` CLI tool

PHP Frontend for Hosting local LLM's (run via VSCode or basic php execution methods/ add to project)

Headless CLI for managing local MLX language-model HTTP servers on Apple Silicon Macs. Supports model discovery, server lifecycle management, performance benchmarking, and provider integration with OpenCode, Claude Code, and LiteLLM.

OpenAI-compatible local inference server for Apple Silicon using MLX. FastAPI server with Chat Completions and Responses APIs, multi-turn conversations, and streaming support.

A unified Monolithic API Server for Ollama that provides the architecture for System Mapping, Task Farming, Local Consultation, and Tool Orchestration.

A flexible FastAPI-based framework for handling AI tasks using Large Language Models (LLMs). Supports multiple providers, extensible tasks and routers, Redis caching, and OpenAI integration. Easily scalable for various LLM-based applications.

Turn a fresh Apple Silicon Mac into a tuned local-LLM dev server in an evening (MIT)

Host an LLM and make it accessible on a network via API.

Unified simple LLM server wrapper with intelligent routing based on model ID

A lightweight, zero-cost FastAPI server for deploying GGUF models via llama.cpp, featuring streaming support and an OpenAI-compatible API.

Run local AI models in VS Code with automatic model detection, server start, and built-in MCP endpoint—no cloud or manual setup required.