Researchers From MIT-IBM Watson AI Lab, the University of Michigan, and ShanghaiTech University Study Ways to Detect Biases and Increase Machine Learning (ML) model’s Individual Fairness

AI systems are widely adopted in several real-world industries for decision-making. Despite their essential roles in numerous tasks, many studies show that such systems are frequently prone to biases resulting in discrimination against individuals based on racial and gender characteristics.

A team of researchers from MIT-IBM Watson AI Lab, the University of Michigan, and ShanghaiTech University has explored ways to detect biases and increase individual fairness in ML models. 

A Statistical Approach to assess ML model’s individual fairness

In their first paper, ‘Statistical Inference for Individual Fairness’, the researchers introduce a statistically principled approach to determining ML models’ individual fairness. They have built a set of inference tools for the adversarial cost function that allows a researcher to calibrate the method, such as prescribing a Type I error rate.

The proposed approach consists of two parts: 

  1. Generating unfair examples: Unfair example is similar to a training example but treated differently by the ML models. Such examples are similar to adversarial examples, but they are only allowed to differ from a training example in some secure ways.
  2. Summarizing the ML model’s behavior on unfair examples: The team proposes a loss-ratio-based approach that is both scale-free and interpretable. 

First, the team employs a gradient flow-based method to find unfair samples. It solves a continuous-time ordinary differential equation. Then it extracts an “unfair map” that maps data samples to similar areas of the sample space where the ML model performs poorly. Additionally, it identifies areas where the ML model violates individual fairness. The approach of defining the test statistic in terms of the unfair map offers computational tractability and reproducibility benefits. 

The researchers present a case study to test individual fairness on the Adult dataset. For this, they used four classifiers: Baseline NN, Group fairness reductions algorithm, Individual fairness SenSR algorithm, and a basic project algorithm. They used AOD: Average Odds Difference to compare group fairness for gender and race.


On the evaluation of results, they note that the baseline violated the individual fairness condition. At the same time, the primary project algorithm improved individual fairness but failed to pass the null hypothesis test. The test verifies the tool’s ability to reveal gender and racial biases in an income prediction model.

Enforcing individual fairness in Gradient Boosting

In the second paper, ‘Individually Fair Gradient Boosting’, the team has focused on enforcing individual fairness in gradient boosting. The study introduces a method that converges globally and directs to individually fair ML models. Additionally, the paper empirically shows that the proposed method preserves gradient boosting accuracy while improving widely used group and individual fairness metrics.

Unlike previous approaches that are not suitable for training non-smooth ML models and perform poorly with flexible non-parametric ML models, the new system handles non-smooth ML models such as decision trees.

The team also explored the convergence and generalization properties of fair gradient boosting. They found that certifying a non-smooth ML model’s individual fairness is possible by checking the empirical performance gap.


To achieve distributionally robust fairness (ensuring that an ML model should have similar performance on similar samples), the researchers used adversarial learning to train an individually fair ML resistant to malicious attacks.

The researchers applied fair gradient boosted trees (BuDRO) to three datasets: German credit dataset, Adult dataset, and COMPAS recidivism prediction dataset. On evaluation, they note that GBDTs performance with XGBoost is very accurate on the German credit dataset, while BuDRA remarks for highest individual fairness and maintaining high accuracy at the same time. Similarly, the results on other datasets demonstrate the effectiveness of BuDRO’s approach to individual fairness.

Paper on Statistical Inference for Individual Fairness:

Paper on Individually Fair Gradient Boosting:

Tanushree Shenwai is a consulting intern at MarktechPost. She is currently pursuing her B.Tech from the Indian Institute of Technology(IIT), Bhubaneswar. She is a Data Science enthusiast and has a keen interest in the scope of application of artificial intelligence in various fields. She is passionate about exploring the new advancements in technologies and their real-life application.