Kashikoi

Kashikoi is a simulation platform for testing AI agents. The vendor says it lets teams build, evaluate and test an agent by simulating real world interactions, so bugs surface before customers see them. Its headline claim is that an agent can be benchmarked from one prompt with no code, replacing manual evaluation work. The home page shows example reports for several kinds of agent: a security operations analyst scored on threat detection, actionable recommendations, false positives and severity accuracy, a data analyst agent compared across model versions, a head to head comparison of OpenAI and Perplexity answers, and a code explorer scored by task type. The vendor says metrics, comparisons and visualizations can be tailored to a team's requirements. It also shows run level data such as success rate, response time and cost per call, and mentions synthetic data and prompt optimization. It is aimed at teams shipping AI agents who want repeatable evaluation instead of hand testing. The vendor notes it is backed by Y Combinator. The calls to action on the site are Book a Demo and Talk to the Founders, with no self serve signup visible. The vendor does not publish pricing. No plans appear on the site, and a conversation with the founders is the stated way to start.