Groq

Groq describes itself as a neocloud for fast AI inference. The company designed the LPU, a processor built specifically for running trained models, and now offers it as a cloud platform for developers and enterprises who need low latency, high throughput inference in production. It states that it operates 13 data centers across four continents, including sites in the United States, Canada, Finland, the United Kingdom and Sydney, Australia. The platform is organized in three layers that build on one another. GroqMetal provides dedicated bare metal infrastructure with full control left to the customer. GroqCore adds a tested inference stack that turns dedicated capacity into production ready performance without requiring infrastructure expertise. GroqAssured adds enterprise governance, auditability and control on top. The company also describes LPX, a rack design pairing 256 LPU accelerators with NVIDIA's next generation GPUs, with published figures of 1,000 tokens per second per user and 315 PFLOPS of FP8 inference compute per rack, for racks when released. The customers are developers building applications on hosted models and enterprises running large inference workloads. The home page and platform page direct visitors to Start building, and they do not list per token or capacity prices.