Safelabs Eval

Safe Labs Eval is an open source red teaming and evaluation framework for AI agents. It probes an agent for vulnerabilities before attackers do, and it targets agents built with LangChain, CrewAI, AutoGPT or any custom agent that exposes an HTTP endpoint. The framework ships 47 adversarial test cases mapped to the OWASP Agentic Security Initiative categories ASI01 through ASI10. Coverage includes prompt injection, jailbreak attacks, data leakage such as system prompt exposure and PII leaking through tool calls, hallucination attacks, unsafe tool use and privilege escalation, and behavioral drift, memory poisoning and multi agent collusion over longer runs. It is used from Python or from a command line. A developer points a red team agent at a target URL, picks a framework and a test suite such as the full OWASP suite, a prompt injection only suite or a jailbreak suite, and runs it. Results come back as a structured summary with a score, and reports can be exported as PDF, JSON or SARIF for audit and compliance use. The package is licensed under Apache 2.0 and installs with pip. The version shown is 0.1.1, and the site offers a waitlist alongside the open source code. No paid plans or prices are published.