Langtest

LangTest is an open source Python library from John Snow Labs for testing language models and NLP models. It generates and runs test cases against a model and produces a report on how it performs. The library offers more than 60 test types covering robustness, bias, fairness, representation, accuracy, coverage and data leakage. Examples shown include adding typos to inputs and swapping names or attributes to check for age, origin, ethnicity or gender bias. A test harness is created in a few lines of Python with a task and a model, then generating, running and reporting are chained together. It works with models from John Snow Labs, Hugging Face, OpenAI and spaCy, and it can augment training data to correct weaknesses that the tests find. It is aimed at data scientists and ML engineers who want to check model quality before deployment. It installs with pip. LangTest is released under the Apache 2.0 license and is free to use. The site publishes no paid plans.