Lance

Lance is an open source lakehouse format for multimodal AI, released under the Apache 2.0 license. It is a file format, table format and catalog specification for building a lakehouse on object storage, designed for the fast random access and scans that AI workloads need. The project describes support for vector search and full text search (BM25) combined with SQL analytics on the same dataset, accelerated by secondary indexes that are part of the specification. It stores images, video, audio, text and embeddings alongside tabular data, with blob encoding for large binary objects that load lazily. Data can be evolved by adding columns with backfilled values without rewriting the whole table. Lance also keeps ACID transactions and time travel, and it integrates with engines such as Apache Spark, Ray, PyTorch, Trino and DuckDB, and with catalogs including Apache Polaris, Unity Catalog, Apache Gravitino and Hive Metastore. Parquet data can be converted to Lance in one line, and a peer reviewed paper from VLDB 2025 describes the design. It is aimed at data and machine learning engineers who build search, feature engineering and model training pipelines, and it is used from Python through the lance package. Lance is free and open source, and the site publishes no paid plans.