Mitigate Machine Bias
Control machine bias to prevent discriminating against your consumers and damaging your organization.
- AI is the new electricity. It is fundamentally and radically changing the fabric of our world, from the way we conduct business, to how we work and live, make decisions, and engage with each other, to how we organize our society, and ultimately, to who we are. Organizations are starting to adopt AI to increase efficiency, better engage customers, and make faster, more accurate decisions.
- Like with any new technology, there is a flip side, a dark side, to AI – machine biases. If unchecked, machine biases replicate, amplify, and systematize societal biases. Biased AI systems may treat some of your customers (or employees) differently, based on their race, gender, identity, age, etc. This is discrimination, and it is against the law. It is also bad for business, including missed opportunities, lost consumer confidence, reputational risk, regulatory sanctions, and lawsuits.
Our Advice
Critical Insight
- Machine biases are not intentional. They reflect the cognitive biases, preconceptions, and judgement of the creators of AI systems and the societal structures encoded in the data sets used for machine learning.
- Machine biases cannot be prevented or fully eliminated. Early identification and diversity in and by design are key. Like with privacy and security breaches, early identification and intervention – ideally at the ideation phase – is the best strategy. Forewarned is forearmed. Prevention starts with a culture of diversity, inclusivity, openness, and collaboration.
- Machine bias is enterprise risk. Machine bias is not a technical issue. It is a social, political, and business problem. Integrate it into your enterprise risk management (ERM).
Impact and Result
- Just because machine biases are induced by human behavior, which is also captured in data silos, they are not inevitable. By asking the right questions upfront during application design, you can prevent many of them.
- Biases can be introduced into an AI system at any stage of the development process, from the data you collect, to the way you collect it, to which algorithms are used, to which assumptions are made, etc. Ask your data science team a lot of questions; leave no stone unturned.
- Don’t wait until “Datasheets for Datasets” and “Model Cards for Model Reporting” (or similar frameworks) become standards. Start creating these documents now to identify and analyze biases in your apps. If using open-source data sets or libraries, you may need to create them yourself for now. If working with partners or using AI/ ML services, demand that they provide such information as part of the engagement. You, not your partners, are ultimately responsible for the AI-powered product or service you deliver to your customers or employees.
- Build a culture of diversity, transparency, inclusivity, and collaboration – the best mechanism to prevent and address machine biases.
- Treat machine bias as enterprise risk. Use your ERM to guide all decisions around machine biases and their mitigation.
Member Testimonials
After each Info-Tech experience, we ask our members to quantify the real-time savings, monetary impact, and project improvements our research helped them achieve. See our top member experiences for this blueprint and what our clients have to say.
Client
Experience
Impact
$ Saved
Days Saved
King County
Workshop
6/10
N/A
5
More valuable: the coming together of people from across the organization to discuss example use cases; exposure to current tools for identifying b... Read More
King County
Guided Implementation
10/10
N/A
N/A
It's extremely difficult to estimate time and cost savings as ML is a new area for us and just starting to figure out our strategy and direction. A... Read More
Workshop: Mitigate Machine Bias
Workshops offer an easy way to accelerate your project. If you are unable to do the project yourself, and a Guided Implementation isn't enough, we offer low-cost delivery of our project workshops. We take you through every phase of your project and ensure that you have a roadmap in place to complete your project successfully.
Module 1: Prepare
The Purpose
- Understand your organization’s maturity with respect to data and analytics in order to maximize workshop value.
Key Benefits Achieved
- Workshop content aligned to your organization’s level of maturity and business objectives.
Activities
Outputs
Execute Data Culture Diagnostic.
- Data Culture Diagnostic report.
Review current analytics strategy.
Review organization's business and IT strategy.
Review other supporting documentation.
Confirm participant list for workshop.
Module 2: Understand Machine Biases
The Purpose
- Develop a good understanding of machine biases and how they emerge from human cognitive and societal biases. Learn about the machine learning process and how it relates to machine bias.
- Select an ML/AI project and complete a bias risk assessment.
Key Benefits Achieved
- A solid understanding of algorithmic biases and the need to mitigate them.
- Increased insight into how new technologies such as ML and AI impact organizational risk.
- Customized bias risk assessment template.
- Completed bias risk assessment for selected project.
Activities
Outputs
Review primer on AI and machine learning (ML).
Review primer on human and machine biases.
Understand business context and objective for AI in your organization.
Discuss selected AI/ML/data science project or use case.
Review and modify bias risk assessment.
- Bias risk assessment template customized for your organization.
Complete bias risk assessment for selected project.
- Completed bias risk assessment for selected project.
Module 3: Identify Data Biases
The Purpose
- Learn about data biases: what they are and where they originate.
- Learn how to address or mitigate data biases.
- Identify data biases in selected project.
Key Benefits Achieved
- A solid understanding of data biases and how to mitigate them.
- Customized Datasheets for Data Sets Template.
- Completed datasheet for data sets for selected project.
Activities
Outputs
Review machine learning process.
Review examples of data biases and why and how they happen.
Identify possible data biases in selected project.
Discuss “Datasheets for Datasets” framework.
Modify Datasheets for Data Sets Template for your organization.
- Datasheets for Data Sets Template customized for your organization.
Complete datasheet for data sets for selected project.
- Completed datasheet for data sets for selected project.
Module 4: Identify Model Biases
The Purpose
- Learn about model biases: what they are and where they originate.
- Learn how to address or mitigate model biases.
- Identify model biases in selected project.
Key Benefits Achieved
- A solid understanding of model biases and how to mitigate them.
- Customized Model Cards for Model Reporting Template.
- Completed model card for selected project.
Activities
Outputs
Review machine learning process.
Review examples of model biases and why and how they happen.
Identify potential model biases in selected project.
Discuss Model Cards For Model Reporting framework.
Modify Model Cards for Model Reporting Template for your organization.
- Model Cards for Model Reporting Template customized for your organization.
Complete model card for selected project.
- Completed model card for selected project.
Module 5: Create Mitigation Plan
The Purpose
- Review mitigation approach and best practices to control machine bias.
- Create mitigation plan to address machine biases in selected project. Align with enterprise risk management (ERM).
Key Benefits Achieved
- A solid understanding of the cultural dimension of algorithmic bias prevention and mitigation and best practices.
- Drafted plan to mitigate machine biases in selected project.
Activities
Outputs
Review and discuss lessons learned.
- Summary of challenges and recommendations to systematically identify and mitigate machine biases.
Create mitigation plan to address machine biases in selected project.
Review mitigation approach and best practices to control machine bias.
Identify gaps and discuss remediation.
- Plan to mitigate machine biases in selected project.
About Info-Tech
Info-Tech Research Group is the world’s fastest-growing information technology research and advisory company, proudly serving over 30,000 IT professionals.
We produce unbiased and highly relevant research to help CIOs and IT leaders make strategic, timely, and well-informed decisions. We partner closely with IT teams to provide everything they need, from actionable tools to analyst guidance, ensuring they deliver measurable results for their organizations.
What Is a Blueprint?
A blueprint is designed to be a roadmap, containing a methodology and the tools and templates you need to solve your IT problems.
Each blueprint can be accompanied by a Guided Implementation that provides you access to our world-class analysts to help you get through the project.
Need Extra Help?
Speak With An Analyst
Get the help you need in this 4-phase advisory process. You'll receive 8 touchpoints with our researchers, all included in your membership.
Guided Implementation 1: Understand AI biases
- Call 1: Learn about machine biases and why they are dangerous if unchecked. Learn about human biases and their relation to machine biases. Learn about the ML process and how it aligns with machine biases.
Guided Implementation 2: Identify data biases
- Call 1: Learn about data biases.
- Call 2: Learn about “Datasheets for Datasets” framework to combat biases in training data. Review customized datasheets template. Review completed datasheet(s) for selected project.
Guided Implementation 3: Identify model biases
- Call 1: Learn about model biases.
- Call 2: Learn about “Model Cards For Model Reporting” framework to combat model biases. Review customized model card template. Review completed model card for selected project.
Guided Implementation 4: Mitigate machine biases and risk
- Call 1: Discuss organizational principles to create a culture for effective bias prevention and mitigation.
- Call 2: Review completed bias risk assessment.
- Call 3: Discuss why AI biases are an enterprise risk. Assess biases identified earlier. Determine mitigation where desired.
Build a Reporting and Analytics Strategy
Establish Data Governance
Build Your Data Quality Program
Select and Implement a Reporting and Analytics Solution
Business Intelligence and Analytics Platform Selection Guide
Mitigate Machine Bias
Get Started With Artificial Intelligence
Create an Architecture for AI
Drive Business Value With Off-the-Shelf AI
Govern the Use of AI Responsibly With a Fit-for-Purpose Structure
AI Trends 2023
Data and Analytics Trends 2023
Build Your Generative AI Roadmap
Tell Your Story With Data Visualization
Select Your Generative AI Vendor
An AI Primer for Business Leaders
Identify and Select Pilot AI Use Cases
Design Your AI Target Operating Model
Plan Your Early AI Moves: Training for Business Leaders