DreamLayer Eval

DreamLayer Eval is an open source benchmarking and evaluation project for image and video diffusion models, published by DreamLayer AI. It replaces hand wired evaluation scripts with a single workflow that manages prompts, seeds, configurations, metric scoring and run logging, so teams can compare generation models faster and reproduce every result later. It is built for AI researchers, ML engineers, labs and developers who need controlled model comparisons. One configuration covers model comparisons and parameter sweeps, hundreds of prompts can be loaded at once, and multiple seeds can be applied per run while prompts, seeds and configs stay consistent across models. Built in metrics include CLIP Score, FID, precision, recall, F1, LPIPS, SSIM, PSNR and temporal consistency measures for video. Every experiment is frozen into a reproducible bundle that ties metrics to their prompts, seeds, outputs and configs, and results export as CSV, JSON and configuration files for papers, reports and leaderboards. The lightweight version runs locally. DreamLayer Eval itself is free. The code is on GitHub under the GPL version 3 license. The prices on the DreamLayer pricing page belong to the separate DreamLayer AI image product, not to this benchmarking project: app credit packs start at 30 credits for $8.40 per month or $10.50 as a one off purchase, and the Agent API charges $0.17 per finished image. The vendor draws a clear line between the two offerings. DreamLayer AI is a commercial image generation and editing platform with a browser app, API, CLI and MCP server, while DreamLayer Eval is the research project that measures models.