Onnxruntime

ONNX Runtime is an open source engine for running and accelerating machine learning models across platforms. It executes models exported to the ONNX format and applies built in optimizations to improve latency, throughput, memory use and binary size on CPUs, GPUs and NPUs. It is aimed at developers who need to ship trained models inside existing applications. Language support covers Python, C#, JavaScript, Java, C++ and Rust, among others, and it runs on Linux, Windows, Mac, iOS, Android and in web browsers through ONNX Runtime Web. A mobile build serves Android and iOS apps. Installation is a package install, for example pip install onnxruntime, and a few lines of code load a model into an inference session and run it. The project also includes a generative AI package, onnxruntime genai, for running large language models and image synthesis models in applications. Beyond inference, ONNX Runtime Training is offered to reduce the cost of large model training and to enable on device training. The vendor states that the runtime powers AI features in Microsoft products including Windows, Office, Azure Cognitive Services and Bing. ONNX Runtime is free to use. The site publishes no pricing page and no paid plans, and support is through documentation, a community and the project's GitHub repository.