Arksim

ArkSim is an open source agent testing framework from Arklex that simulates realistic multi turn conversations with an AI agent and scores the results. It helps developers find failures that only appear across several turns, such as losing context, calling the wrong tool or contradicting an earlier answer, before real users meet them. It is built for developers who want to validate and harden agents without relying on manual testing. The workflow has three parts. Scenarios define a simulated user's attributes, goals and prior knowledge, and can be reused to compare agent versions and catch regressions. Simulation turns those scenarios into live conversations between LLM powered synthetic users and the agent, producing full transcripts. Evaluation scores every turn with quantitative and qualitative metrics, such as helpfulness and faithfulness, and flags behavior failures to prioritize, with support for custom metrics. It works with agents built on LangChain, CrewAI, the OpenAI Agents SDK or custom code, connected through a Chat Completions HTTP endpoint, the A2A protocol or a loaded Python agent. Getting started takes pip install arksim, arksim init to scaffold a config, and a single simulate and evaluate command. ArkSim is free under the Apache 2.0 license and published on GitHub. No paid plan or pricing is published for the framework; the model calls that power synthetic users and evaluation are billed by whichever provider the user configures. The documentation is versioned, and this listing points to the 0.3 release line.