OneTriangle

OneTriangle is an inference platform that lowers the cost and delay of running large language models on long inputs by transferring the key value cache between models. Instead of paying large model rates to read a long prompt, the service runs the prefill step on a small model and hands the resulting cache to a larger model, which then decodes the answer. The vendor calls this the first KV cache aware inference engine and cites its own research on cross model KV cache transfer. The technique projects the small model's per layer key and value tensors into the larger model's attention space, after removing the source rotary embeddings and re embedding them at target positions. The vendor says each model pair is gated on held out quality and latency, and ordinary prefill takes over whenever a pair does not clear. Its measured results on a 100,000 token context include Llama 3.1 8B feeding 70B, with first token time falling from 16.9 seconds to 2.7 seconds, and prefill cost per million requests falling from $46.9k to $7.5k. Other tested pairs are Qwen3 8B to 32B and Qwen3 4B to 14B. OneTriangle also runs open weight models such as Llama, Qwen, Mistral, Gemma and DeepSeek privately for customers, handling the GPUs and serving so the customer changes one line of configuration. Pricing is not published; the site directs visitors to book a demo or sign in. The team is a group of MIT researchers and engineers backed by Y Combinator.