EffGen

effGen is an open source Python framework for building AI agents designed around small language models. The same agent, tools and result object work in three settings: with model weights loaded in your own process through transformers, vLLM, GGUF or MLX engines with no API key or network; against any server you already run that speaks the OpenAI protocol, such as vLLM, SGLang, TGI, llama.cpp, Ollama, LM Studio or LiteLLM; or through a hosted provider. Version 1.0.0 is the first stable release. It is aimed at developers and researchers who want agents that run on modest hardware or private infrastructure without being locked to one vendor. The package ships 66 built in tools, 9 agent presets, 35 prompt templates, memory, retrieval augmented generation, guardrails, evaluation, a coding agent and a command line with 29 commands. Ten provider adapters are included, nine of them with a bundled catalogue of 417 models listing context windows, prices and capabilities, and the framework reports per run cost where it knows the price rather than inventing a zero. The effgen serve command puts an authenticated server compatible with the OpenAI protocol in front of agents, with role based access, a Prometheus metrics endpoint, error budget tracking and failover when a provider stops responding. It supports Python 3.11 to 3.14 and is installed from PyPI. effGen is free and open source, with no paid tier or hosted service advertised. Costs come only from any hosted model provider you choose to call. The project is accompanied by an arXiv paper, and the site notes that every figure it quotes is read from the installed package at build time.