privateGPT

PrivateGPT is an open source API layer that turns locally hosted language models into production AI applications. It does not run models itself. Instead it connects to any OpenAI compatible inference server, such as Ollama, llama.cpp or vLLM, through a configurable base URL, and adds higher level building blocks on top, including retrieval augmented generation, skills, tools, MCP support and text to SQL. The API follows the Claude API model, and the project's goal is to let teams build private AI products without depending on cloud APIs. The project is aimed at developers and organizations that need to keep data on their own hardware. A built in workbench interface is served at the /ui path for testing and demos, though the maintainers state that the API is the actual product. The repository, which began under a personal account and now lives under the Zylon AI organization on GitHub, is production tested as the foundation of Zylon, an on premise AI platform for enterprises from the same team. The code is released under the Apache 2.0 license and is free to use. Running it requires supplying your own inference server and hardware, and any costs for that infrastructure fall on the user. The repository carries a security policy and citation file, and the project page at zylon.ai describes the commercial Zylon offering built on it.