Magnitude

Magnitude is an inference engine that runs open weight AI models locally and tunes itself to the user's exact hardware. It ships as a desktop app for macOS, Linux and Windows, and supports Apple Silicon, NVIDIA and AMD GPUs, or a CPU alone. There is no fixed minimum hardware requirement, though smaller machines run smaller models. It is aimed at developers who run coding agents against local models and want better speed without per token costs. Instead of precompiled kernels for broad classes of hardware, Magnitude compiles and tunes its kernels on the device before a model runs, and includes hand optimized kernels for popular open weight model families. The vendor reports up to twice the speed of llama.cpp, citing 92 percent faster decode on Metal and 19 percent on CUDA in its own benchmark with a 35 billion parameter Qwen model. It also reports 27 percent less memory per agent, with memory released when agents stop, and prefix cache sharing so concurrent sessions do not slow each other. A one click setup connects agents including Pi, OpenCode, Hermes, OpenClaw, Codex, Claude Code, Oh My Pi and Cline, and anything else can use the OpenAI compatible API. Prompts, files and models stay on the machine, and no internet connection is needed once a model is downloaded. Magnitude is free and open source under the Apache 2.0 license.