Vmlx

vMLX is a local inference engine for Apple Silicon Macs that runs large language models entirely on the user's machine. It is built on MLX and aims to keep long conversations fast by avoiding repeated work. A prefix cache recognizes the part of a conversation already processed, a paged key value cache spills to SSD at full precision so a long thread survives a restart, and speculative decoding drafts several tokens ahead and verifies them in one pass. Continuous batching lets several agents share one loaded model. It is aimed at developers and privacy conscious users who want to run models locally. The engine speaks both the OpenAI and Anthropic APIs, so coding tools such as Claude Code and Cursor can be pointed at it without changing configuration. A built in converter turns GGUF models into MLX format and supports JANG quantization, and the unified memory design avoids copies between host and device. The app runs on macOS 14 or later with M1 through M5 chips. The site states that nothing in the inference path leaves the Mac, with no account, no sign in and no telemetry. A benchmark table is marked as pending and has not been published. vMLX is open source under the MIT license and is free to download, with no paid plans listed.