Google Open-Sources LIT: A Visual, Interactive Model-Understanding Tool For NLP Models

Google AI Researchers recently released its open-source tool LIT (Language Interpretability Tool) for visualizing, auditing, and understanding the NLP models.

LIT focuses on core questions about model behavior, such as : 

Why did my model make this prediction? What kind of examples does my model perform poorly on? What happens under a controlled change in the input? This prediction be attributed to adversarial behavior or too undesirable priors in the training set? Etc.

LIT could be really helpful in understanding the model’s tendencies to behave according to biases and heuristics and save time for data scientists. LIT supports a variety of debugging workflows through a browser-based UI, and it is set up in such a way so that users can try choosing between visualizations and analysis for testing hypotheses and validate those hypotheses over a selected dataset.

LIT supports various models—including classification, seq2seq, and structured prediction. It is highly extensible through a declarative, framework-agnostic API.

Features 

  • Local explanations via salience maps, attention, and rich visualization of model predictions.
  • Aggregate analysis including custom metrics, slicing & binning, and visualization of embedding spaces.
  • Counterfactual generation via manual edits or generator plug-ins to dynamically create and evaluate new examples.
  • Side-by-side mode to compare two or more models or one model on a pair of examples.
  • Highly extensible to new model types, including classification, regression, span labeling, seq2seq, and language modeling. Supports multi-head models and multiple input features out of the box.
  • Framework-agnostic and compatible with TensorFlow, PyTorch, and more.

Github: https://github.com/pair-code/lit

Paper: https://arxiv.org/pdf/2008.05122.pdf

Asif Razzaq is the CEO of Marktechpost Media Inc.. As a visionary entrepreneur and engineer, Asif is committed to harnessing the potential of Artificial Intelligence for social good. His most recent endeavor is the launch of an Artificial Intelligence Media Platform, Marktechpost, which stands out for its in-depth coverage of machine learning and deep learning news that is both technically sound and easily understandable by a wide audience. The platform boasts of over 2 million monthly views, illustrating its popularity among audiences.

🚀 LLMWare Launches SLIMs: Small Specialized Function-Calling Models for Multi-Step Automation [Check out all the models]