On Detecting Biased Predictions with Post-hoc Explanation Methods

Matteo Ruggeri, Alice Dethise, Marco Canini

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

Abstract

We develop a methodology for the analysis of machine learning (ML) models to detect and understand biased decisions and apply it to two specific scenarios. In particular, we show how analyzing model predictions across the dataset, comparing models trained on different subsets of the original data, and applying model-agnostic post-hoc explanation tools can help identify bias in a model in general as well as in specific instances. Further, we consider several definitions of bias and fairness, and show how each provides a different interpretation of the model decisions. Our results show that the analysis of models through the lens of statistical analysis and post-hoc explanations helps to detect and understand bias. We also observe that post-hoc explanations often fail to detect individual biased instances, and caution against using this category of tools to guarantee model fairness. Finally, we provide insights on how this analysis can help understand the origin and shape of bias.

Original languageEnglish (US)
Title of host publicationSAFE 2023 - Proceedings of the 2023 Explainable and Safety Bounded, Fidelitous, Machine Learning for Networking
PublisherAssociation for Computing Machinery, Inc
Pages17-23
Number of pages7
ISBN (Electronic)9798400704499
DOIs
StatePublished - Dec 8 2023
Event2023 Explainable and Safety Bounded, Fidelitous, Machine Learning for Networking, SAFE 2023 - Paris, France
Duration: Dec 8 2023 → …

Publication series

NameSAFE 2023 - Proceedings of the 2023 Explainable and Safety Bounded, Fidelitous, Machine Learning for Networking

Conference

Conference2023 Explainable and Safety Bounded, Fidelitous, Machine Learning for Networking, SAFE 2023
Country/TerritoryFrance
CityParis
Period12/8/23 → …

Bibliographical note

Publisher Copyright:
© 2023 Association for Computing Machinery.

Keywords

  • Explainable Machine Learning
  • Feature Analysis
  • Post-hoc Explanations

ASJC Scopus subject areas

  • Computer Science Applications
  • Information Systems
  • Software

Fingerprint

Dive into the research topics of 'On Detecting Biased Predictions with Post-hoc Explanation Methods'. Together they form a unique fingerprint.

Cite this