Glow

Glow is an open source toolkit for large scale genomic analysis, built natively on Apache Spark. It lets researchers aggregate genomic data and run data preparation, statistical analysis and machine learning at biobank scale and beyond, using the same Spark SQL engine that many organizations already use for petabyte class workloads. It is part of the Project Glow community, with code, documentation, a forum and a Slack workspace linked from the site. The toolkit reads common genomics file formats such as VCF and BGEN into Spark DataFrames and works with the standard big data formats. Queries can be written in Python, SQL, R, Java and Scala, and genomic data can be joined with other datasets such as electronic health records, real world evidence and medical images. Building blocks include functions for quality control and data manipulation, variant normalization and lift over, regression functions, integration with Spark ML libraries for population stratification, and utilities for piping DataFrames through command line tools or parallelizing existing tools and Pandas functions. It is aimed at bioinformaticians, computational biologists and data engineers who already know Spark and want tertiary genomics workflows without learning a new set of APIs. Glow is free open source software and the site publishes no commercial pricing. Note that it is a data and statistics toolkit with machine learning integration rather than a general AI coding assistant, so its listing category is a loose fit.