This Open Sourced Python library ‘audit-AI’ does Bias Testing for Generalized Machine Learning Applications

0
1090
Image Source: https://github.com/pymetrics/audit-ai
-Advertisement-

This open-sourced bias testing python library ‘audit AI’ can be used for measuring and mitigating the effects of discriminatory patterns in training data and the predictions made by machine learning algorithms. This tool is built over pandas and sklearn by pymetrics.

Github Link: https://github.com/pymetrics/audit-ai

-Advertisement-Python Specialization from University of Michigan

Related Paper: https://arxiv.org/pdf/1906.09208.pdf

Installation/Process

You can install the latest released version with pip

# pip
pip install audit-AI

If you install with pip, you’ll need to install scikit-learn, numpy, and pandas with either pip or conda. Version requirements:

  • numpy
  • scipy
  • pandas

For vizualization:

  • matplotlib
  • seaborn

How to use this package: (Copied from https://github.com/pymetrics/audit-ai )

from auditai.misc import bias_test_check

X = df.loc[:,features]
y_pred = clf.predict_proba(X)

# test for bias
bias_test_check(labels=df['gender'], results=y_pred, category='Gender')

>>> *Gender passes 4/5 test, Fisher p-value, Chi-Squared p-value, z-test p-value and Bayes Factor at 50.00*

To get a plot of the different tests at different thresholds:

from auditai.viz import plot_threshold_tests

X = df.loc[:,features]
y_pred = clf.predict_proba(X)

# test for bias
plot_threshold_tests(labels=df['gender'], results=y_pred, category='Gender')

LEAVE A REPLY

Please enter your comment!
Please enter your name here

This site uses Akismet to reduce spam. Learn how your comment data is processed.