FastFlowLM

FastFlowLM is a runtime for running large language models directly on the neural processing units built into AMD Ryzen AI chips, rather than pushing the work to a GPU or the CPU. It installs as a download of roughly 17 MB and exposes a single command interface familiar from other local model tools: list available models, pull one, run it interactively or start a server. Context windows reach up to 256k tokens, and the vendor states more than ten times the power efficiency of stacks designed around the GPU. It is built for developers working on Ryzen AI hardware on Windows and Linux, with supported silicon listed as Strix, Strix Halo, Kraken and Gorgon Point. Model coverage spans text generation, vision, audio, reasoning, embeddings and mixture of experts architectures, with curated recipes for families including Llama, Gemma, Qwen and Whisper, so image description and long audio transcription both run on the device. A server mode exposes an API compatible with OpenAI, which means existing applications and front ends such as Open WebUI work without rewrites. Signed installers are provided for Ryzen AI laptops, no separate drivers are required, and the runtime holds tokens locally and supports a fully offline mode. The runtime is free to download and the source is published under the MIT license. There is no pricing page, subscription or paid edition. One development worth knowing: the team behind the project has joined AMD, and the repository now sits under the ROCm organization on GitHub, which is also where the installer is published.