Vision Agents

Vision Agents is an open source Python framework for building real time voice and video AI agents. A developer plugs in language, speech or vision models from more than 35 providers, including OpenAI, Gemini, Anthropic, Deepgram and ElevenLabs, and ships agents that see, hear and respond. The documentation cites sub 500 millisecond latency on Stream's global edge network, and a quickstart scaffolds a first agent in a few minutes. It is aimed at developers building telehealth assistants, voice support, live coaching and similar interactive applications. Agents can run in two modes: realtime APIs over WebRTC or WebSocket, or custom pipelines that chain speech to text, a language model and text to speech. Video processing lets YOLO, Roboflow or custom models run on every frame. A Twilio integration handles phone calls with two way audio, and retrieval augmented generation is supported through TurboPuffer vector search and Gemini FileSearch. For production, the framework includes an HTTP server, Prometheus metrics and Docker and Kubernetes deployment. The documentation shows example builds such as a golf coach that uses pose detection and live feedback, a phone support agent backed by a knowledge base, a smart security camera with face recognition, a live sports commentator, a virtual try on and an interactive avatar. The site publishes no pricing page for the framework itself. It points to Stream Video, which it says offers 333,000 free participant minutes.