Mitigating Bias in AI: How Companies Can Build Fairer AI Systems

Bias doesn’t stop with people—it’s embedded in the algorithms shaping our world. By training AI on historical data, we risk amplifying human prejudice instead of eliminating it. As AI becomes more prevalent, so do instances of biased AI systems doing measurable harm.

One study found that when mortgage lenders use AI, 80% of Black applicants are more likely to be denied home loans, even when their financials are similar to white applicants who were accepted. In another instance, an AI tool used across several US health systems prioritized white patients over Black patients with more severe conditions.

While these outcomes may have been unintentional, they highlight the potential damage irresponsible AI can inflict.

What Causes Bias in AI?

AI bias, or machine learning bias, is a multifaceted issue. It can stem from its datasets, algorithms, and developers’ perspectives. Dr. Ricardo Baeza-Yates, Director of Research for the Institute of Experiential Artificial Intelligence at Northeastern University, states, “Bias is a mirror of the designers of the intelligent system, not the system itself." According to Baeza-Yates, the bias stems largely from the training data developers provide. It can also come from the objectives of the learning algorithm and the feedback loop of user interactions.

There are three main sources of bias in AI systems:

• Sampling bias
• Algorithmic bias
• User bias

Each type of bias causes a unique set of issues, and all can be intertwined.

Sampling Bias

Sampling bias arises when the data used to train an AI system is too narrow for generalized use. This prevents it from accurately representing the diversity of real-world scenarios.

For example, Joy Buolamwini, an MIT researcher, discovered that several facial recognition systems struggled to detect darker-skinned women. The systems had an error rate of 35% compared to rates of no more than 1% for lighter-skinned men. Buolamwini determined that this resulted from these systems lacking data on faces of people of color, especially women.

Algorithmic Bias

Algorithmic bias is when an algorithm prioritizes or discriminates against certain attributes. This bias often accompanies sampling bias, as it can stem from flawed and non-representative data.

In other cases, algorithmic bias can arise from incorrectly categorized information, where data errors distort predictions. These errors may occur due to developers consciously or unconsciously applying subjective rules or factors. Additionally, models might misinterpret data patterns, mistaking correlation for causation, which perpetuates bias.

User Bias

User bias occurs when AI learns from biased user-generated data as people interact with the system. User feedback loops are useful for continuous learning and refinement. However, if users hold unconscious biases or deliberately attempt to train a system with their feedback, the algorithm can pick up on that bias. Microsoft’s chatbot Tay was one of the most egregious examples of this. Shortly after its launch in 2016, Twitter users fed it racist, offensive information. It ultimately led to Tay tweeting “wildly inappropriate and reprehensible words and images.”

What Are the Impacts of AI Biases?

AI systems can help automate hiring processes, target advertisements, and make data-driven decisions. However, if the system is trained on biased or inaccurate data, the algorithm will reflect that bias and produce harmful outputs. 

This exact situation played out with Amazon’s experimental hiring algorithm. Amazon trained the algorithm on hiring and resume data from the past 10 years. However, the data was biased: The majority of applicants over the past decade had been men, reflecting male dominance in the industry.

As a result, the AI system learned that bias and penalized resumes that included the word “women.” This included rejecting candidates who had attended women’s colleges or had affiliations with women’s sports teams and organizations. While Amazon attempted to fix the issue by editing the ML model to be more neutral, it was ultimately scrapped in 2018.

There have been many other instances of AI displaying bias over the years, with some incidents doing more harm than others. In another instance, underrepresented data on women or minority groups led to CAD systems in healthcare returning lower accuracy results for Black patients. AI-powered predictive policing tools, meanwhile, rely on historical arrest data that can reinforce patterns of racial profiling.

What Are the Benefits of Fairer Systems?

Fairer systems and other responsible AI strategies protect companies and their users from many of AI’s drawbacks, both ethical and legal.

Boston Consulting Group found that companies with responsible AI experienced nearly 30% fewer AI failures. These failures refer to system malfunctions that negatively impacted the company or its consumers. As only 35% of global consumers trust how businesses implement AI technology, these fairer, safer systems can help preserve company reputations and build client trust.

Fairer systems are also notably more profitable than their counterparts: Outside of avoiding the fees and legal costs that might stem from biased outputs, Bain & Company found that companies with a comprehensive RAI approach earn double the profit from their AI efforts.

How Can Companies Counter Bias?

Unfortunately, addressing AI bias is easier said than done. Datasets often reflect historical or unconscious prejudices. As such, they may over- or underrepresent certain demographics or interpret correlations as causation.

According to McKinsey & Co., “Such biases have a tendency to stay embedded because recognizing them, and taking steps to address them, requires a deep mastery of data-science techniques, as well as a more meta-understanding of existing social forces.”

Companies can address AI biases by implementing clear guidelines, oversight, and strategies for diversification. Many resources are available; for example, Section508.gov provides valuable resources to help businesses minimize bias in AI, machine learning models, and emerging technologies.

By leveraging these strategies, companies can create more equitable systems.

Diversification

One key strategy companies can use to mitigate bias is diversification. A report conducted by the EEOC found that 63.5 to 68.5% of high-tech employees were white and that men held 80% of executive positions in the industry. However, talent exists across all demographics and regions. Companies that embrace diversity unlock exceptional expertise often overlooked due to systemic biases.

Advancements in infrastructure have made it easier than ever to collaborate globally. Many companies are turning to nearshore AI talent to both reduce costs and bring global perspectives to their teams. FullStack, a leading AI and software consultancy, partners with senior and PhD-level AI experts in Latin America to do just that. These developers play a critical role in delivering cutting-edge, responsible AI projects.

A diverse team is also uniquely positioned to identify and address biases that might otherwise go unnoticed.

Diversification isn't just for humans, either. Section508.gov encourages companies to leverage diversified data sources to train and test their AI models. USC computer science researchers created a novel approach to this earlier this year. Their method uses “quality-diversity algorithms” to create diverse synthetic datasets. These datasets “plug the gaps” in training data for AI-driven cancer diagnosis systems, increasing the representation of intersectional groups.

Allen Chang, one of the project's researchers, referred to QD algorithms as a “promising direction for augmenting models with bias-aware sampling." They hope their method can help AI systems perform accurately for all users. 

Oversight

AI oversight, or AI governance, are the steps companies take to ensure their systems work as intended. System complexity, safe margins of error, and company size directly shape the level of governance required. However, all systems need some human oversight to prevent bias.

As CEO of FullStack, Ben Carle has extensive experience working with trustworthy AI systems. “Even the most advanced AI systems work best with human oversight, particularly for complex tasks," says Carle. Such human oversight plays a critical role in identifying and reducing bias.

Industry leaders, including IBM, address some oversight needs through AI ethics boards. These boards consist of experts who establish AI policies and ensure accountability. These boards tackle 'could vs. should' questions, identify high-risk bias areas, and decide when human involvement is critical for safe operation.

Smaller companies may not require full ethics boards, but they still need some governance practices. Automated monitoring and regular human audits help maintain accountability and address risks over time.

Understanding how an AI system operates is critical for managing risks and failures. Explainable artificial intelligence (XAI) practices address this need by making decision-making processes transparent. XAI tools enable leaders to identify biases, enhance accountability, and build trust in their systems. Overall, it complements the broader governance strategies outlined earlier.

Guidelines

Companies looking to reduce bias in their AI models must build comprehensive guidelines for development and post-development monitoring. These guidelines outline the business's standards, providing clarity for its development team, user transparency, and a stronger sense of accountability. 

Various companies and organizations worldwide have already established their own AI guidelines. UNESCO, for instance, created the first-ever global standard on AI ethics in November 2021: the Recommendation on the Ethics of Artificial Intelligence. 

The Recommendation features principles such as explainability, accountability, and non-discrimination. Together, these principles work to create a human-rights-centered approach to AI. More than 50 countries and eight major tech companies, including Microsoft, are now committed to its implementation.

Many governments also have AI regulations in place or are working to introduce them. The White House published an Executive Order outlining the secure development and use of AI in late 2023. In March 2024, the EU followed suit and officially passed its AI Act.

By building fairness into their systems now, companies can avoid costly pivots or potential infractions as legislation evolves to keep pace. Business owners should consider their local AI regulations, as well as general nondiscrimination laws, when developing their guidelines.

Building Better AI Systems

AI systems have the power to transform industries and address human biases—but only if companies take deliberate action. By embracing responsible AI practices, leveraging diverse talent, and staying informed on the latest ethical guidelines, businesses can build systems that are not only innovative but equitable. 

Interested in building better AI? Download the Business Leader's AI Handbook for an accessible guide to understanding, building, and deploying AI systems.