Uber recently open-sourced its model-agnostic visual debugging tool ‘Manifold’ for machine learning models. The objective of this tool is to help data scientists and data engineers identify performance issues across datasets and models in a visually intuitive way.
Machine learning applications are different than general software applications in terms of their constantly changing and evolving structure as the model builds more knowledge. Therefore debugging and interpreting machine learning models has become one of the most challenging roles of real-world AI solutions. Performance issues across ML data slices and models can be easily identified using Manifold.
Features in Version-1 Release
- Model-agnostic support for general binary classification and regression model debugging.
- Visualization support for tabular feature input including numerical, categorical, and geospatial feature types.
- Integration with Jupyter Notebook.
- Interactive data slicing and performance comparisons based on per-instance prediction loss and other feature values.
Paper (2018): https://arxiv.org/pdf/1808.00196.pdf
Demo Web: http://manifold.mlvis.io/