Openvino

OpenVINO is an open source toolkit from Intel for deploying AI models across cloud, AI PCs, edge devices and robotics. It converts, optimizes and runs inference on models from popular frameworks, with the aim of getting the most out of Intel hardware. The documentation describes four main tools. The base package runs and deploys conventional AI models. OpenVINO GenAI covers generative models. OpenVINO Physical AI deploys vision language action models on robots. OpenVINO Model Server serves both generative and conventional inference from a server or a Kubernetes environment, and includes features such as LLM serving, embeddings and reranking, model pipelines, versioning policies and runtime model reshaping. Models from TensorFlow, ONNX and PaddlePaddle can be loaded directly or converted to the OpenVINO format, and PyTorch applications can use it through torch.compile. Post training and training time compression is available through NNCF. The runtime supports Linux, Windows and macOS, with Python, C++ and C APIs. Automatic device discovery lets an application written once run on different hardware. Pre optimized OpenVINO models are published on Hugging Face, and the project publishes performance benchmarks for popular models. It is aimed at developers and machine learning engineers who deploy models in production. The vendor publishes no pricing page; the toolkit is distributed as open source software with public documentation.