Semantic Router

Semantic Router is an open source Python library from Aurelio AI that acts as a fast decision making layer for LLMs and agents. Instead of waiting for a language model to decide which tool to call or whether a request is safe, it routes each request by meaning, using embeddings and vector similarity. Developers define routes, each with example utterances, and attach an encoder such as Cohere or Jina embeddings. When a user message arrives, the router scores it against each route. If no route scores above the configured threshold it returns nothing, and the turn falls through to the default handler, which avoids invented tool calls. The site describes static routes, dynamic routes and guardrails. The vendor says it enables tool selection that can scale to many thousands of tools, since tool definitions no longer need to fit in the model's context window, and it compares a router decision time of a few milliseconds against several seconds for a large model. The homepage includes a playground that runs locally with no LLM calls. It suits developers building agents, chatbots and retrieval systems who want deterministic routing, faster responses and cheaper tool use. Installation is through pip, and the project requires Python 3.9 or later. The library is MIT licensed and code and documentation are on GitHub. No pricing is published for it. Aurelio AI also offers consultancy and development services.