XAI

Science and technology for the
eXplanation of AI decision making

Publications


Factual and Counterfactual Explanations for Black Box Decision Making

TitleFactual and Counterfactual Explanations for Black Box Decision Making
Publication TypeJournal Article
Year of Publication2019
AuthorsGuidotti, R, Monreale, A, Giannotti, F, Pedreschi, D, Ruggieri, S, Turini, F
JournalIEEE Intelligent Systems
AbstractThe rise of sophisticated machine learning models has brought accurate but obscure decision systems, which hide their logic, thus undermining transparency, trust, and the adoption of artificial intelligence (AI) in socially sensitive and safety-critical contexts. We introduce a local rule-based explanation method, providing faithful explanations of the decision made by a black box classifier on a specific instance. The proposed method first learns an interpretable, local classifier on a synthetic neighborhood of the instance under investigation, generated by a genetic algorithm. Then, it derives from the interpretable classifier an explanation consisting of a decision rule, explaining the factual reasons of the decision, and a set of counterfactuals, suggesting the changes in the instance features that would lead to a different outcome. Experimental results show that the proposed method outperforms existing approaches in terms of the quality of the explanations and of the accuracy in mimicking the black box.
URLhttps://ieeexplore.ieee.org/abstract/document/8920138
DOI10.1109/MIS.2019.2957223

On The Stability of Interpretable Models

TitleOn The Stability of Interpretable Models
Publication TypeConference Paper
Year of Publication2019
AuthorsGuidotti, R, Ruggieri, S
Conference Name2019 International Joint Conference on Neural Networks (IJCNN)
PublisherIEEE
AttachmentSize
PDF icon ijcnn2019stability.pdf847.98 KB

Investigating Neighborhood Generation Methods for Explanations of Obscure Image Classifiers

TitleInvestigating Neighborhood Generation Methods for Explanations of Obscure Image Classifiers
Publication TypeConference Paper
Year of Publication2019
AuthorsGuidotti, R, Monreale, A, Cariaggi, L
Conference NamePacific-Asia Conference on Knowledge Discovery and Data Mining
PublisherSpringer
AbstractGiven the wide use of machine learning approaches based on opaque prediction models, understanding the reasons behind decisions of black box decision systems is nowadays a crucial topic. We address the problem of providing meaningful explanations in the widely-applied image classification tasks. In particular, we explore the impact of changing the neighborhood generation function for a local interpretable model-agnostic explanator by proposing four different variants. All the proposed methods are based on a grid-based segmentation of the images, but each of them proposes a different strategy for generating the neighborhood of the image for which an explanation is required. A deep experimentation shows both improvements and weakness of each proposed approach.
URLhttps://link.springer.com/chapter/10.1007/978-3-030-16148-4_5
DOI10.1007/978-3-030-16148-4_5
AttachmentSize
PDF icon pakdd2019investigating.pdf3.21 MB

Explaining multi-label black-box classifiers for health applications

TitleExplaining multi-label black-box classifiers for health applications
Publication TypeConference Paper
Year of Publication2019
AuthorsPanigutti, C, Guidotti, R, Monreale, A, Pedreschi, D
Conference NameInternational Workshop on Health Intelligence
PublisherSpringer
AbstractToday the state-of-the-art performance in classification is achieved by the so-called “black boxes”, i.e. decision-making systems whose internal logic is obscure. Such models could revolutionize the health-care system, however their deployment in real-world diagnosis decision support systems is subject to several risks and limitations due to the lack of transparency. The typical classification problem in health-care requires a multi-label approach since the possible labels are not mutually exclusive, e.g. diagnoses. We propose MARLENA, a model-agnostic method which explains multi-label black box decisions. MARLENA explains an individual decision in three steps. First, it generates a synthetic neighborhood around the instance to be explained using a strategy suitable for multi-label decisions. It then learns a decision tree on such neighborhood and finally derives from it a decision rule that explains the black box decision. Our experiments show that MARLENA performs well in terms of mimicking the black box behavior while gaining at the same time a notable amount of interpretability through compact decision rules, i.e. rules with limited length.
URLhttps://link.springer.com/chapter/10.1007/978-3-030-24409-5_9
DOI10.1007/978-3-030-24409-5_9
AttachmentSize
PDF icon w3phiai2019marlena.pdf400.29 KB

A survey of methods for explaining black box models

TitleA survey of methods for explaining black box models
Publication TypeJournal Article
Year of Publication2019
AuthorsGuidotti, R, Monreale, A, Ruggieri, S, Turini, F, Giannotti, F, Pedreschi, D
JournalACM computing surveys (CSUR)
Volume51
Issue5
Pagination93
AbstractIn recent years, many accurate decision support systems have been constructed as black boxes, that is as systems that hide their internal logic to the user. This lack of explanation constitutes both a practical and an ethical issue. The literature reports many approaches aimed at overcoming this crucial weakness, sometimes at the cost of sacrificing accuracy for interpretability. The applications in which black box decision systems can be used are various, and each approach is typically developed to provide a solution for a specific problem and, as a consequence, it explicitly or implicitly delineates its own definition of interpretability and explanation. The aim of this article is to provide a classification of the main problems addressed in the literature with respect to the notion of explanation and the type of black box system. Given a problem definition, a black box type, and a desired explanation, this survey should help the researcher to find the proposals more useful for his own work. The proposed classification of approaches to open black box models should also be useful for putting the many research open questions in perspective.
URLhttps://dl.acm.org/doi/abs/10.1145/3236009
DOI10.1145/3236009
AttachmentSize
PDF icon csur2018survey.pdf3.84 MB

The AI black box Explanation Problem

TitleThe AI black box Explanation Problem
Publication TypeJournal Article
Year of Publication2019
AuthorsGuidotti, R, Monreale, A, Pedreschi, D
JournalERCIM NEWS
Issue116
Pagination12–13

Meaningful explanations of Black Box AI decision systems

TitleMeaningful explanations of Black Box AI decision systems
Publication TypeConference Paper
Year of Publication2019
AuthorsPedreschi, D, Giannotti, F, Guidotti, R, Monreale, A, Ruggieri, S, Turini, F
Conference NameProceedings of the AAAI Conference on Artificial Intelligence
AbstractBlack box AI systems for automated decision making, often based on machine learning over (big) data, map a user’s features into a class or a score without exposing the reasons why. This is problematic not only for lack of transparency, but also for possible biases inherited by the algorithms from human prejudices and collection artifacts hidden in the training data, which may lead to unfair or wrong decisions. We focus on the urgent open challenge of how to construct meaningful explanations of opaque AI/ML systems, introducing the local-toglobal framework for black box explanation, articulated along three lines: (i) the language for expressing explanations in terms of logic rules, with statistical and causal interpretation; (ii) the inference of local explanations for revealing the decision rationale for a specific case, by auditing the black box in the vicinity of the target instance; (iii), the bottom-up generalization of many local explanations into simple global ones, with algorithms that optimize for quality and comprehensibility. We argue that the local-first approach opens the door to a wide variety of alternative solutions along different dimensions: a variety of data sources (relational, text, images, etc.), a variety of learning problems (multi-label classification, regression, scoring, ranking), a variety of languages for expressing meaningful explanations, a variety of means to audit a black box.
URLhttps://aaai.org/ojs/index.php/AAAI/article/view/5050
DOI10.1609/aaai.v33i01.33019780