Understanding algorithmic bias

Algorithmic bias is a pivotal issue in the world of ML. It occurs when a system, intentionally or not, generates outputs that are unfair or systematically prejudiced toward certain individuals or groups. This prejudice often originates from the fact that these systems learn from existing data, which itself can be riddled with inherent societal bias.

Fairness, as it relates to ML, is defined as the absence of any bias. While it might sound simple, achieving fairness can be an intricate process that calls for careful management at every step of model creation.

To paint a more detailed picture, let’s consider protected features. These are attributes that could potentially introduce bias into the system. They can be legally mandated, such as race and gender, or stem from organizational values, such as location or zip code. While seemingly benign, these features, when used in an ML model, can result in decisions that are biased or discriminatory.

Diving deeper, we find two major types of algorithmic bias:

• Disparate impact occurs when a model explicitly relies on protected attributes, favoring a certain group over others
• In contrast, disparate treatment arises when a model implicitly uses protected attributes through related variables, thereby indirectly resulting in biased outcomes

A prime example of disparate treatment can be someone’s zip code, which might predominantly contain a particular race or socioeconomic status. This, in turn, could lead to skewed predictions or decisions that inadvertently favor or disfavor that particular group. Similarly, variables such as whether someone has been arrested before can introduce bias, given that certain groups have historically faced more arrests due to societal bias.

An initial approach to addressing bias is “unawareness,” which entails removing any explicit mention of protected features. However, this approach is a low bar in addressing bias, as it doesn’t account for disparate impacts that can still occur.

Fairness can also be approached via statistical measures such as statistical parity and equalized odds. Statistical parity states that a model’s predictions should be independent of any given sensitive feature. For example, a model predicting recidivism should do so equally, irrespective of race. However, this approach ignores any actual relationship between the label and the sensitive attribute, which can result in biased outcomes.

Here, we have individual and group fairness. The former ensures similar individuals have similar outcomes, while the latter insists on equal statistical outcomes across groups divided by protected attributes. Each brings its own unique perspective on fairness and its achievement.

Equalized odds, on the other hand, propose that a model’s predictions should be independent of any sensitive feature, conditional on the response value. This approach encourages the model to be more accurate across all groups but can overlook larger socioeconomic reasons that might result in a certain group falling into a label more frequently.

By understanding the nuances of algorithmic bias, we can better devise strategies to prevent, identify, and mitigate bias in ML models.