Although there are many approaches and libraries presently in practice to analyze fairness in machine learning models, they all are still limited to some or other factors. These tools are either specifically not addressing large-scale problems (and the inherent challenges that come with such scale) or tied to a specific cloud environment.
To solve this issue, LinkedIn introduces an open-source tool called LiFT (LinkedIn Fairness Toolkit). LiFT is a Scala/Spark library that enables the measurement of fairness, according to a multitude of fairness definitions, in large-scale machine learning workflows.
The main features of LiFT include flexibility towards deployment in ML workflows and integration in different stages of ML training systems. It is designed as a reusable library at its core and can also be used for ad-hoc fairness analysis.. Apart from flexibility, LiFT is also scalable and can be distributed on several nodes to analyze and scale bias measurements in various machine learning models.
Github: https://github.com/linkedin/LiFT
Linkedin: https://engineering.linkedin.com/blog/2020/lift-addressing-bias-in-large-scale-ai-applications
Asif Razzaq is the CEO of Marktechpost Media Inc.. As a visionary entrepreneur and engineer, Asif is committed to harnessing the potential of Artificial Intelligence for social good. His most recent endeavor is the launch of an Artificial Intelligence Media Platform, Marktechpost, which stands out for its in-depth coverage of machine learning and deep learning news that is both technically sound and easily understandable by a wide audience. The platform boasts of over 2 million monthly views, illustrating its popularity among audiences.
