Gpustack

GPUStack is an AI infrastructure platform for deploying, managing and scaling large language models and GPU compute on hardware you control. It runs on premise, in the cloud or in hybrid setups, and presents one control plane for two jobs: model as a service for serving inference, and GPU as a service for handing out raw compute. It is aimed at platform and infrastructure teams that run models across mixed hardware. The site lists support for NVIDIA, AMD, Ascend, Hygon, MetaX, Moore Threads, Cambricon and other accelerators, on x86_64 and ARM64 Linux under Docker, Podman or Kubernetes. Models can be pulled from Hugging Face, ModelScope or local files, and the platform maps your hardware to a suitable inference engine such as vLLM, SGLang, llama.cpp, TensorRT LLM or MindIE. Large models can be spread across nodes and GPUs with tensor and pipeline parallelism, and finished deployments are exposed through OpenAI compatible and Anthropic compatible endpoints. Management features include load balancing, failover, traffic weight routing for gradual rollouts and observability. An Enterprise edition adds control plane high availability with automatic failover, multi tenancy with separate organizations, quotas and endpoints, role based access control and white label branding. No pricing page is published. Structured data on the site lists a price of zero, and the Enterprise edition is sold through a talk to sales form, so enterprise costs are not public. The site highlights benchmark reports showing throughput and latency gains over unoptimized vLLM baselines on H100 and H200 clusters, with full reports linked from the home page.