Crawl4ai

Crawl4AI is an open source web crawler built to produce output that language models can use directly, and it is now also available as a hosted API. Given a URL it returns clean Markdown with boilerplate stripped, and it can extract typed JSON records from a page using a plain English instruction instead of selectors, run web searches that return ranked JSON results, and, as an experimental feature, answer a question directly with its sources. It serves developers building agents, retrieval pipelines and data tools. The Python library can be installed with pip and run on your own machines and proxies under the Apache 2.0 license. The hosted service handles browsers, proxies, queues, jobs and retries, and plugs into Claude Code, Codex, Cursor, Windsurf, OpenCode, Gemini CLI and other clients through a remote MCP server, a plain HTTP API or a one line setup prompt that points an agent at the project's llms.txt file. A browser playground lets you test calls and estimate cost before running them. The hosted API is priced in credits, where one credit is $0.001; a page costs one credit, a search half a credit, a real browser render two and a hard site four. The Community tier starts with $10 of free credit, no card and an extra $5 each month. The Supporter tier sells packs of $10, $25 or $100 that never expire. A Developer subscription is listed as coming soon, and Enterprise is by contact. A $1 trial credit for seven days needs no signup. The site describes these as discounted launch prices for the first year that may change, with a launch offer of a free first $10 pack until 31 December 2026.