![](https://crypto4nerd.com/wp-content/uploads/2023/05/1auac68T04PI_CHyzsxUOCA.png)
Introduction
With the increasing popularity of machine learning models like ChatGPT, the issue of bias has become more prominent. OpenAI, the artificial intelligence research organization behind ChatGPT, publicly said it is working on making ChatGPT more fair, accurate, and less biased following a series of reports about inappropriate results.
Despite guardrails, machine learning systems, including ChatGPT, are works in progress. Machine learning models can generate biased results for many reasons, one being models can be trained on datasets that may reflect societal prejudices and contain biases. Additionally, models that tend to have the highest predictive accuracy often pose challenges in terms of interpretability. They are commonly referred to as “black boxes” due to their inherent complexity, making it difficult to understand their decision-making processes. However, explainability tools such as Shapley values can assist in mitigating bias in machine learning models.
So, what are Shapley values, and how can they help us reduce bias in machine learning models? We will explore what Shapley values are, how they can and have helped reduce bias, and provide an example of how to implement Shapley values with python.
What are Shapley Values?
Shapley values are a concept from cooperative game theory, introduced by Lloyd Shapley in 1953. Shapley values have since been widely used in various fields, including economics, political science, and machine learning.
In a machine learning model, each feature contributes a certain amount to the final prediction. Shapley values can be used to measure the contribution of each feature in a model towards the final prediction. Specifically, Shapley values calculate the marginal contribution of each feature by comparing the prediction with and without that feature, and then averaging those differences over all possible feature combinations. By doing so, we are able to determine the importance of each feature and how much it contributes to the overall prediction, this can help us to identify any biases and improve the model’s performance.
“[Shapley values] are useful metrics for deciding how important a feature is or what it contributes to the decision a machine learning model is going to make. It’s a very nice methodology, one that is quite intuitive, its only drawback being its complexity for computing it.” — Kyle Polich, from Data Skeptic
How Can Shapley Values Help Reduce Bias in Machine Learning Models?
Shapley values can help reduce bias in machine learning models in several ways. First, they can help identify which features are contributing to bias, this allows us to remove or modify the features to reduce their impact. This is important because bias can be introduced into models even after removing the identified features through the use of proxy data. Proxy data refers to data that is used, possibly unintentionally, as stand-in for unavailable data. Take for example, a model that is trying to predict whether an applicant will default on their loan and is trained on data that includes regional district information. While the data does not explicitly introduce features about race or socioeconomic status to the model, features like the regional district of an applicant may act as proxy data and inadvertently introduce bias if certain regional districts are associated with certain races and/or socioeconomic statuses.
Second, Shapley values can help ensure that the model is not over-relying on a single feature, which can also lead to bias. Measuring the contribution of each feature allows us to explore if the model is overly dependent on a single feature and address the issue.
Bias in machine learning models can occur for many reasons including, but not limited to, biased data, biased assumptions, incomplete or incorrect data, and biased model parameters. Shapley values can help to identify which features are contributing to the bias and allows us to adjust or remove them to reduce the bias.
Real-world Examples of Shapley Values Reducing Bias:
Shapley values have been used in various industries to reduce bias in machine learning models. For example, in healthcare, Shapley values have been used to identify features that are contributing to bias in models predicting hospital mortality in ICU patients. In finance, Shapley values have been used to measure the impact of different variables on loan approvals to ensure that decisions are not biased against certain groups.
Given that government agencies are responsible for serving entire populations, there is a greater responsibility for them to use machine learning models that are transparent and free from bias. Shapley values can help government agencies ensure that their models are fair and unbiased, which can lead to better outcomes for citizens.
One example of a government agency that benefits from Shapley values is the Arizona Department of Revenue. The Arizona Department of Revenue “relies on the GCOM RevHub integrated tax system for risk scoring, issue detection, social network analysis, audit select, and collections optimization.” Unintended biases built into models could result in unfair treatment of certain taxpayers and could lead to legal and ethical issues. Model bias against a certain group of taxpayers could lead to increased scrutiny and penalties, while other groups are treated more leniently. These issues could ultimately undermine the credibility and trust in the tax system and the government agency. RevHub leverages the transparency provided by Shapley values to help ensure that bias isn’t inadvertently built into their models.
Overall, Shapley values can play an important role in helping government agencies reduce bias to ensure that their policies and practices are fair and equitable for all citizens.
Code Example:
To demonstrate how Shapley values can be implemented in machine learning with Python, we will use the SHAP (SHapley Additive exPlanations) library, a popular explainable AI tool for calculating Shapley values and interpreting the predictions of machine learning models.
We will use a modified Random Forest model to predict whether the applicant will default on their loan. (Here ‘0’ will represent that the person has paid the loan and ‘1’ represents that the person has defaulted). Then we will use Shapley values via the SHAP library to calculate the impact of each feature in the model.
Limitations:
As noted by Polich earlier, computing Shapley values is computationally expensive. The computational concern increases as the number of features and size of the dataset increase. While the SHAP library is able to aid with the computational implementation and costs, generating SHAP values can be more time-consuming than training the model. However, there are ongoing developments working towards reducing the calculation times and different techniques to offset the costs such as using a subsample of the dataset as input as opposed to the entire dataset.
Conclusion:
In conclusion, Shapley values are a powerful tool that can help reduce bias in machine learning models. By measuring the contribution of each feature, in every combination and in every order, we can identify and remove or modify features that are contributing to bias, and ensure that the model is not over-reliant on a single feature.
Resources:
https://shap.readthedocs.io/en/latest/example_notebooks/api_examples/plots/waterfall.html