Can AI Credit Scoring Be Biased Against Certain Groups?

Bias Can Enter a Model in Several Ways

AI credit scoring models learn patterns from historical data, and if that historical data reflects past lending disparities, whether from explicit past discrimination or from broader economic and social inequities, the model can learn and reproduce those same patterns. This is one of the central concerns researchers, regulators, and consumer advocates raise about AI in credit decisions: a model isn’t inherently neutral just because it’s mathematical, since it’s only as unbiased as the data and design choices behind it.

A second, subtler way bias can enter a model involves proxy variables. Lenders are generally prohibited from using protected characteristics like race, religion, national origin, or gender directly in credit decisions. But AI models can inadvertently rely on other data points that correlate closely with those characteristics, such as zip code, which can correlate with race due to historical housing patterns, or certain types of purchase history, which can correlate with gender or age. A model can end up producing disparate outcomes across protected groups even if it was never given those protected characteristics directly as inputs.

Why This Is Especially Hard to Catch With Complex Models

More complex machine learning models, particularly ones that combine many variables in non-obvious ways, can make it harder to identify exactly which factors are driving a given outcome. This is sometimes called the “black box” problem, and it complicates bias testing because it’s not always straightforward to trace a disparate outcome back to a specific cause, even when a disparity is detected in aggregate results.

This challenge is a major reason bias testing for credit models generally focuses on outcomes (whether a model’s approval, denial, and pricing decisions differ meaningfully across protected groups after controlling for legitimate credit risk factors) rather than solely on inspecting inputs. Regulators and researchers have developed statistical methods for this kind of disparate impact testing, and it’s an active area of both regulatory guidance and academic research.

What the Law Requires

In the United States, the Equal Credit Opportunity Act prohibits discrimination in credit decisions based on protected characteristics, and this legal requirement applies to AI-driven underwriting just as it applies to traditional, human-driven underwriting. The Consumer Financial Protection Bureau has stated that lenders remain responsible for ensuring their credit models, including AI-based ones, comply with fair lending laws, and that complexity or opacity in a model doesn’t excuse a lender from providing legally required adverse action notices or complying with anti-discrimination requirements.

Bottom Line

AI credit scoring can be biased against certain groups, primarily through historical data that reflects past disparities or through proxy variables that correlate with protected characteristics even when those characteristics aren’t used directly, which is why fair lending laws require lenders to test and monitor these models for discriminatory outcomes rather than assuming a mathematical model is automatically neutral.

Go deeper

• What happens when an ai underwriting model is trained on biased historical claims data?
• Can you get college credit for completing an online ai certification?
• What Real-World Harms Have Resulted From Biased AI Systems?