Selected Publications

This section presents the most significant scientific publications produced within the ERC XAI project. The works listed here have been selected for their strategic importance and foundational impact on the project’s research lines. They represent the main contributions to the development of Explainable Artificial Intelligence (XAI) methodologies and algorithms. The selection covers key progress across our research areas (RA1-RA5), with a particular focus on the impact in critical sectors such as healthcare, finance, and mobility, outlining our vision for a more transparent and trustworthy AI.

2025

A Practical Approach to Causal Inference over Time

Martina Cinquini, Isacco Beretta, Salvatore Ruggieri, and Isabel Valera

Proceedings of the AAAI Conference on Artificial Intelligence, Apr 2025

RESEARCH LINE
2

Abs Bib HTML

In this paper, we focus on estimating the causal effect of an intervention over time on a dynamical system. To that end, we formally define causal interventions and their effects over time on discrete-time stochastic processes (DSPs). Then, we show under which conditions the equilibrium states of a DSP, both before and after a causal intervention, can be captured by a structural causal model (SCM). With such an equivalence at hand, we provide an explicit mapping from vector autoregressive models (VARs), broadly applied in econometrics, to linear, but potentially cyclic and/or affected by unmeasured confounders, SCMs. The resulting causal VAR framework allows us to perform causal inference over time from observational time series data. Our experiments on synthetic and real-world datasets show that the proposed framework achieves strong performance in terms of observational forecasting while enabling accurate estimation of the causal effect of interventions on dynamical systems. We demonstrate, through a case study, the potential practical questions that can be addressed using the proposed causal VAR framework.
@article{CBR2025, author = {Cinquini, Martina and Beretta, Isacco and Ruggieri, Salvatore and Valera, Isabel}, doi = {10.1609/aaai.v39i14.33626}, issn = {2159-5399}, journal = {Proceedings of the AAAI Conference on Artificial Intelligence}, line = {2}, month = apr, number = {14}, open_access = {Gold}, pages = {14832–14839}, publisher = {Association for the Advancement of Artificial Intelligence (AAAI)}, title = {A Practical Approach to Causal Inference over Time}, visible_on_website = {YES}, volume = {39}, year = {2025} }
Embracing Diversity: A Multi-Perspective Approach with Soft Labels

Benedetta Muscato, Praveen Bushipaka, Gizem Gezici, Lucia Passaro, Fosca Giannotti, and 1 more author

Sep 2025

RESEARCH LINE
4

5

Abs Bib HTML

In subjective tasks like stance detection, diverse human perspectives are often simplified into a single ground truth through label aggregation i.e. majority voting, potentially marginalizing minority viewpoints. This paper presents a Multi-Perspective framework for stance detection that explicitly incorporates annotation diversity by using soft labels derived from both human and large language model (LLM) annotations. Building on a stance detection dataset focused on controversial topics, we augment it with document summaries and new LLM-generated labels. We then compare two approaches: a baseline using aggregated hard labels, and a multi-perspective model trained on disaggregated soft labels that capture annotation distributions. Our findings show that multi-perspective models consistently outperform traditional baselines (higher F1-scores), with lower model confidence, reflecting task subjectivity. This work highlights the importance of modeling disagreement and promotes a shift toward more inclusive, perspective-aware NLP systems.
@inbook{MBG2025, author = {Muscato, Benedetta and Bushipaka, Praveen and Gezici, Gizem and Passaro, Lucia and Giannotti, Fosca and Cucinotta, Tommaso}, booktitle = {HHAI 2025}, doi = {10.3233/faia250654}, isbn = {9781643686110}, issn = {1879-8314}, line = {4,5}, month = sep, open_access = {Gold}, pages = {370--384}, publisher = {IOS Press}, title = {Embracing Diversity: A Multi-Perspective Approach with Soft Labels}, visible_on_website = {YES}, year = {2025} }
SafeGen: safeguarding privacy and fairness through a genetic method

Martina Cinquini, Marta Marchiori Manerba, Federico Mazzoni, Francesca Pratesi, and Riccardo Guidotti

Machine Learning, Sep 2025

RESEARCH LINE
5

Abs Bib HTML

To ensure that Machine Learning systems produce unharmful outcomes, pursuing a joint optimization of performance and ethical profiles such as privacy and fairness is crucial. However, jointly optimizing these two ethical dimensions while maintaining predictive accuracy remains a fundamental challenge. Indeed, privacy-preserving techniques may worsen fairness and restrain the model’s ability to learn accurate statistical patterns, while data mitigation techniques may inadvertently compromise privacy. Aiming to bridge this gap, we propose safeGen, a preprocessing fairness enhancing and privacy-preserving method for tabular data. SafeGen employs synthetic data generation through a genetic algorithm to ensure that sensitive attributes are protected while maintaining the necessary statistical properties. We assess our method across multiple datasets, comparing it against state-of-the-art privacy-preserving and fairness approaches through a threefold evaluation: privacy preservation, fairness enhancement, and generated data plausibility. Through extensive experiments, we demonstrate that SafeGen consistently achieves strong anonymization while preserving or improving dataset fairness across several benchmarks. Additionally, through hybrid privacy-fairness constraints and the use of a genetic synthesizer, SafeGen ensures the plausibility of synthetic records while minimizing discrimination. Our findings demonstrate that modeling fairness and privacy within a unified generative method yields significantly better outcomes than addressing these constraints separately, reinforcing the importance of integrated approaches when multiple ethical objectives must be simultaneously satisfied.
@article{CMM2025, author = {Cinquini, Martina and Marchiori Manerba, Marta and Mazzoni, Federico and Pratesi, Francesca and Guidotti, Riccardo}, doi = {10.1007/s10994-025-06835-9}, issn = {1573-0565}, journal = {Machine Learning}, line = {5}, month = sep, number = {10}, open_access = {Gold}, publisher = {Springer Science and Business Media LLC}, title = {SafeGen: safeguarding privacy and fairness through a genetic method}, visible_on_website = {YES}, volume = {114}, year = {2025} }
Human-AI coevolution

Dino Pedreschi, Luca Pappalardo, Emanuele Ferragina, Ricardo Baeza-Yates, Albert-László Barabási, and 12 more authors

Artificial Intelligence, Feb 2025

RESEARCH LINE
1

Abs Bib HTML

Human-AI coevolution, defined as a process in which humans and AI algorithms continuously influence each other, increasingly characterises our society, but is understudied in artificial intelligence and complexity science literature. Recommender systems and assistants play a prominent role in human-AI coevolution, as they permeate many facets of daily life and influence human choices through online platforms. The interaction between users and AI results in a potentially endless feedback loop, wherein users’ choices generate data to train AI models, which, in turn, shape subsequent user preferences. This human-AI feedback loop has peculiar characteristics compared to traditional human-machine interaction and gives rise to complex and often “unintended” systemic outcomes. This paper introduces human-AI coevolution as the cornerstone for a new field of study at the intersection between AI and complexity science focused on the theoretical, empirical, and mathematical investigation of the human-AI feedback loop. In doing so, we: (i) outline the pros and cons of existing methodologies and highlight shortcomings and potential ways for capturing feedback loop mechanisms; (ii) propose a reflection at the intersection between complexity science, AI and society; (iii) provide real-world examples for different human-AI ecosystems; and (iv) illustrate challenges to the creation of such a field of study, conceptualising them at increasing levels of abstraction, i.e., scientific, legal and socio-political.
@article{PPF2025, address = {Netherlands}, author = {Pedreschi, Dino and Pappalardo, Luca and Ferragina, Emanuele and Baeza-Yates, Ricardo and Barabási, Albert-László and Dignum, Frank and Dignum, Virginia and Eliassi-Rad, Tina and Giannotti, Fosca and Kertész, János and Knott, Alistair and Ioannidis, Yannis and Lukowicz, Paul and Passarella, Andrea and Pentland, Alex Sandy and Shawe-Taylor, John and Vespignani, Alessandro}, doi = {10.1016/j.artint.2024.104244}, issn = {0004-3702}, journal = {Artificial Intelligence}, line = {1}, month = feb, open_access = {Gold}, pages = {104244}, publisher = {Elsevier BV}, title = {Human-AI coevolution}, visible_on_website = {YES}, volume = {339}, year = {2025} }
MASCOTS: Model-Agnostic Symbolic COunterfactual Explanations for Time Series

Dawid Płudowski, Francesco Spinnato, Piotr Wilczyński, Krzysztof Kotowski, Evridiki Vasileia Ntagiou, and 2 more authors

Sep 2025

RESEARCH LINE
1

Abs Bib HTML

Counterfactual explanations provide an intuitive way to understand model decisions by identifying minimal changes required to alter an outcome. However, applying counterfactual methods to time series models remains challenging due to temporal dependencies, high dimensionality, and the lack of an intuitive human-interpretable representation. We introduce MASCOTS, a method that leverages the Bag-of-Receptive-Fields representation alongside symbolic transformations inspired by Symbolic Aggregate Approximation. By operating in a symbolic feature space, it enhances interpretability while preserving fidelity to the original data and model. Unlike existing approaches that either depend on model structure or autoencoder-based sampling, MASCOTS directly generates meaningful and diverse counterfactual observations in a model-agnostic manner, operating on both univariate and multivariate data. We evaluate MASCOTS on univariate and multivariate benchmark datasets, demonstrating comparable validity, proximity, and plausibility to state-of-the-art methods, while significantly improving interpretability and sparsity. Its symbolic nature allows for explanations that can be expressed visually, in natural language, or through semantic representations, making counterfactual reasoning more accessible and actionable.
@inbook{PSW2025, author = {Płudowski, Dawid and Spinnato, Francesco and Wilczyński, Piotr and Kotowski, Krzysztof and Ntagiou, Evridiki Vasileia and Guidotti, Riccardo and Biecek, Przemysław}, booktitle = {Machine Learning and Knowledge Discovery in Databases. Research Track}, doi = {10.1007/978-3-032-06078-5_6}, isbn = {9783032060785}, issn = {1611-3349}, line = {1}, month = sep, open_access = {Gold}, pages = {94–112}, publisher = {Springer Nature Switzerland}, title = {MASCOTS: Model-Agnostic Symbolic COunterfactual Explanations for Time Series}, visible_on_website = {YES}, year = {2025} }
Mathematical Foundation of Interpretable Equivariant Surrogate Models

Jacopo Joy Colombini, Filippo Bonchi, Francesco Giannini, Fosca Giannotti, Roberto Pellungrini, and 1 more author

Oct 2025

RESEARCH LINE
1

Abs Bib HTML

This paper introduces a rigorous mathematical framework for neural network explainability, and more broadly for the explainability of equivariant operators called Group Equivariant Operators (GEOs), based on Group Equivariant Non-Expansive Operators (GENEOs) transformations. The central concept involves quantifying the distance between GEOs by measuring the non-commutativity of specific diagrams. Additionally, the paper proposes a definition of interpretability of GEOs according to a complexity measure that can be defined according to each user’s preferences. Moreover, we explore the formal properties of this framework and show how it can be applied in classical machine learning scenarios, like image classification with convolutional neural networks. (SpringerLink)
@inbook{CBG2025, address = {Istanbul, Turkey}, author = {Colombini, Jacopo Joy and Bonchi, Filippo and Giannini, Francesco and Giannotti, Fosca and Pellungrini, Roberto and Frosini, Patrizio}, booktitle = {Explainable Artificial Intelligence}, doi = {10.1007/978-3-032-08324-1_13}, isbn = {9783032083241}, issn = {1865-0937}, line = {1}, month = oct, open_access = {Gold}, pages = {294–318}, publisher = {Springer Nature Switzerland}, title = {Mathematical Foundation of Interpretable Equivariant Surrogate Models}, visible_on_website = {YES}, year = {2025} }
Balancing Fairness and Interpretability in Clustering with FairParTree

Cristiano Landi, Alessio Cascione, Marta Marchiori Manerba, and Riccardo Guidotti

Oct 2025

RESEARCH LINE
1

5

Abs Bib HTML

The revolution involving Machine Learning has transformed data analytics, making algorithms important in decision-making processes across various domains, even in sensitive scenarios. Indeed, traditional clustering algorithms often lack interpretability and exhibit biases, leading to discriminatory practices and opaque decision-making. To overcome these limitations, we introduce FairParTree, a fair and interpretable clustering algorithm that integrates fairness constraints directly into the clustering process, ensuring that the resulting clusters do not disproportionately disadvantage any particular group. By leveraging the structure of decision trees, FairParTree enhances the interpretability of clustering results by providing clear and understandable motivations for cluster assignments through rule-based explanations. We evaluate FairParTree against state-of-the-art competitors. Through extensive experiments, we show that it maintains strong performances w.r.t. fairness, interpretability, and clustering quality across different dataset sizes, thus positioning itself as a competitive, fair, and interpretable clustering algorithm.
@inbook{LCM2025, author = {Landi, Cristiano and Cascione, Alessio and Manerba, Marta Marchiori and Guidotti, Riccardo}, booktitle = {Explainable Artificial Intelligence}, doi = {10.1007/978-3-032-08324-1_5}, isbn = {9783032083241}, issn = {1865-0937}, line = {1,5}, month = oct, open_access = {Gold}, pages = {104–127}, publisher = {Springer Nature Switzerland}, title = {Balancing Fairness and Interpretability in Clustering with FairParTree}, visible_on_website = {YES}, year = {2025} }
Towards Building a Trustworthy RAG-Based Chatbot for the Italian Public Administration

Chandana Sree Mala, Christian Maio, Mattia Proietti, Gizem Gezici, Fosca Giannotti, and 3 more authors

Sep 2025

RESEARCH LINE
3

5

Abs Bib HTML

Building a Trustworthy Retrieval-Augmented Generation (RAG) chatbot for Italy’s public sector presents challenges that go beyond selecting an appropriate Large Language Model. A major issue is the retrieval phase, where Italian text embedders often underperform compared to English and multilingual counterparts, hindering precise identification and contextualization of critical information. Regulatory constraints further complicate matters by disallowing closed source or cloud based models, forcing reliance on on-premise or fully open source solutions that may not fully address the linguistic complexities of Italian documents. In our study, we evaluate three embedding approaches using a publicly available Italian dataset: a monolingual Italian approach, a translation based method leveraging English only embedders with backward reference mapping, and a multilingual framework applied to both original and translated texts. Our methodology involves chunking documents into coherent segments, embedding them in a high dimensional semantic space, and measuring retrieval accuracy via top-k similarity searches. Our results indicate that the translation based approach significantly improves retrieval performance over Italian specific models, suggesting that bilingual mapping can effectively address both domain specific challenges and regulatory constraints in developing RAG pipelines for public administration.
@inbook{MDP2025, author = {Mala, Chandana Sree and di Maio, Christian and Proietti, Mattia and Gezici, Gizem and Giannotti, Fosca and Melacci, Stefano and Lenci, Alessandro and Gori, Marco}, booktitle = {HHAI 2025}, doi = {10.3233/faia250637}, isbn = {9781643686110}, issn = {1879-8314}, line = {3,5}, month = sep, open_access = {Gold}, pages = {196--204}, publisher = {IOS Press}, title = {Towards Building a Trustworthy RAG-Based Chatbot for the Italian Public Administration}, visible_on_website = {YES}, year = {2025} }
Perspectives in Play: A Multi-Perspective Approach for More Inclusive NLP Systems

Benedetta Muscato, Lucia Passaro, Gizem Gezici, and Fosca Giannotti

In Proceedings of the Thirty-Fourth International Joint Conference on Artificial Intelligence , Sep 2025

RESEARCH LINE
4

5

Abs Bib HTML

In the realm of Natural Language Processing (NLP), common approaches for handling human disagreement consist of aggregating annotators’ viewpoints to establish a single ground truth. However, prior studies show that disregarding individual opinions can lead to the side-effect of under-representing minority perspectives, especially in subjective tasks, where annotators may systematically disagree because of their preferences. Recognizing that labels reflect the diverse backgrounds, life experiences, and values of individuals, this study proposes a new multi-perspective approach using soft labels to encourage the development of the next generation of perspective-aware models—more inclusive and pluralistic. We conduct an extensive analysis across diverse subjective text classification tasks including hate speech, irony, abusive language, and stance detection, to highlight the importance of capturing human disagreements, often overlooked by traditional aggregation methods. Results show that the multi-perspective approach not only better approximates human label distributions, as measured by Jensen-Shannon Divergence (JSD), but also achieves superior classification performance (higher F1-scores), outperforming traditional approaches. However, our approach exhibits lower confidence in tasks like irony and stance detection, likely due to the inherent subjectivity present in the texts. Lastly, leveraging Explainable AI (XAI), we explore model uncertainty and uncover meaningful insights into model predictions. All implementation details are available at our github repo.
@inproceedings{MPG2025, author = {Muscato, Benedetta and Passaro, Lucia and Gezici, Gizem and Giannotti, Fosca}, booktitle = {Proceedings of the Thirty-Fourth International Joint Conference on Artificial Intelligence}, collection = {IJCAI-2025}, doi = {10.24963/ijcai.2025/1092}, line = {4,5}, month = sep, open_access = {Gold}, pages = {9827–9835}, publisher = {International Joint Conferences on Artificial Intelligence Organization}, series = {IJCAI-2025}, title = {Perspectives in Play: A Multi-Perspective Approach for More Inclusive NLP Systems}, visible_on_website = {YES}, year = {2025} }
An Interpretable Data-Driven Unsupervised Approach for the Prevention of Forgotten Items

Luca Corbucci, Javier Alejandro Borges Legrottaglie, Francesco Spinnato, Anna Monreale, and Riccardo Guidotti

Oct 2025

RESEARCH LINE
1

Abs Bib HTML

Accurately identifying items forgotten during a supermarket visit and providing clear, interpretable explanations for recommending them remains an underexplored problem within the Next Basket Prediction (NBP) domain. Existing NBP approaches typically only focus on forecasting future purchases, without explicitly addressing the detection of unintentionally omitted items. This gap is partly due to the scarcity of real-world datasets that allow for the reliable estimation of forgotten items. Furthermore, most current NBP methods rely on black-box models, which lack transparency and limit the ability to justify recommendations to end users. In this paper, we formally introduce the forgotten-item prediction task and propose two novel interpretable-by-design algorithms. These methods are tailored to identify forgotten items while offering intuitive, human-understandable explanations. Experiments on a real-world retail dataset show our algorithms outperform state-of-the-art NBP baselines by 10–15% across multiple evaluation metrics.
@inbook{CBS2025, author = {Corbucci, Luca and Borges Legrottaglie, Javier Alejandro and Spinnato, Francesco and Monreale, Anna and Guidotti, Riccardo}, booktitle = {ECAI 2025}, doi = {10.3233/faia250912}, isbn = {9781643686318}, issn = {1879-8314}, line = {1}, month = oct, open_access = {Gold}, publisher = {IOS Press}, title = {An Interpretable Data-Driven Unsupervised Approach for the Prevention of Forgotten Items}, visible_on_website = {YES}, year = {2025} }
Group Explainability Through Local Approximation

Mattia Setzu, Riccardo Guidotti, Dino Pedreschi, and Fosca Giannotti

Oct 2025

RESEARCH LINE
2

Abs Bib HTML

Machine learning models are becoming increasingly complex and widely adopted. Interpretable machine learning allows us to not only make predictions but also understand the rationale behind automated decisions through explanations. Explanations are typically characterized by their scope: local explanations are generated by local surrogate models for specific instances, while global explanations aim to approximate the behavior of the entire black-box model. In this paper, we break this dichotomy of locality to explore an underexamined area that lies between these two extremes: meso-level explanations. The goal of meso-level explainability is to provide explanations using a set of meso-level interpretable models, which capture patterns at an intermediate level of abstraction. To this end, we propose GrouX, an explainable-by-design algorithm that generates meso-level explanations in the form of feature importance scores. Our approach includes a partitioning phase that identifies meso groups, followed by the training of interpretable models within each group. We evaluate GrouX on a collection of tabular datasets, reporting both the accuracy and complexity of the resulting meso models, and compare it against other meso-level explainability algorithms. Additionally, we analyze the algorithm’s sensitivity to its hyperparameters to better understand its behavior and robustness.
@inbook{SGP2025, address = {ECAI 2025}, author = {Setzu, Mattia and Guidotti, Riccardo and Pedreschi, Dino and Giannotti, Fosca}, booktitle = {ECAI 2025}, doi = {10.3233/faia250902}, isbn = {9781643686318}, issn = {1879-8314}, line = {2}, month = oct, open_access = {Gold}, pages = {952 - 958}, publisher = {IOS Press}, title = {Group Explainability Through Local Approximation}, visible_on_website = {YES}, year = {2025} }
Interpretable Instance-Based Learning Through Pairwise Distance Trees

Andrea Fedele, Alessio Cascione, Riccardo Guidotti, and Cristiano Landi

Sep 2025

RESEARCH LINE
1

Abs Bib HTML

Instance-based models offer natural interpretability by making decisions based on concrete examples. However, their transparency is often hindered by the use of complex similarity measures, which are difficult to interpret, especially in high-dimensional datasets. To address this issue, this paper presents a meta-learning framework that enhances the interpretability of instance-based models by replacing traditional, complex pairwise distance functions with interpretable pairwise distance trees. These trees are designed to prioritize simplicity and transparency while preserving the model’s effectiveness. By offering a clear decision-making process, the framework makes the instance selection more understandable. Also, the framework mitigates the computational burden of instance-based models, which typically require calculating all pairwise distances. Leveraging the generalization capabilities of pairwise distance trees and employing sampling strategies to select representative subsets, the method significantly reduces computational complexity. Our experiments demonstrate that the proposed approach improves computational efficiency with only a modest trade-off in accuracy while substantially enhancing the interpretability of the learned distance measure.
@inbook{FCG2025, author = {Fedele, Andrea and Cascione, Alessio and Guidotti, Riccardo and Landi, Cristiano}, booktitle = {Machine Learning and Knowledge Discovery in Databases. Research Track}, doi = {10.1007/978-3-032-06078-5_1}, isbn = {9783032060785}, issn = {1611-3349}, line = {1}, month = sep, open_access = {Gold}, pages = {3–21}, publisher = {Springer Nature Switzerland}, title = {Interpretable Instance-Based Learning Through Pairwise Distance Trees}, visible_on_website = {YES}, year = {2025} }
Unsupervised and Interpretable Detection of User Personalities in Online Social Networks

Alessio Cascione, Laura Pollacci, and Riccardo Guidotti

Oct 2025

RESEARCH LINE
1

Abs Bib HTML

Personalized moderation interventions in online social networks foster healthier interactions by adapting responses to both individual traits and contextual factors. However, implementing such interventions is challenging due to transparency concerns and the lack of ground-truth behavioral data from expert psychologists. Interpretability is crucial for addressing these challenges, as it enables platforms to tailor moderation strategies while ensuring fairness and user trust. In this paper, we present an unsupervised, data-driven framework to build an interpretable predictive model capable of distinguishing between toxic and non-toxic users with different personality traits. We leverage personality representations from an external resource to uncover behavioral profiles through clustering, utilizing embeddings of both toxic and non-toxic users. Then, we model users with features capturing linguistic and affective dimensions, training an interpretable personality detector capable of distinguishing between behavioral profiles in a transparent and explainable manner. A case study on Reddit demonstrates the effectiveness of our approach, highlighting how an interpretable model can achieve competitive performance comparable to a black-box alternative while offering meaningful insights into toxic and non-toxic users behavior.
@inbook{CPG2025, author = {Cascione, Alessio and Pollacci, Laura and Guidotti, Riccardo}, booktitle = {Explainable Artificial Intelligence}, doi = {10.1007/978-3-032-08327-2_8}, isbn = {9783032083272}, issn = {1865-0937}, line = {1}, month = oct, open_access = {Gold}, pages = {162–179}, publisher = {Springer Nature Switzerland}, title = {Unsupervised and Interpretable Detection of User Personalities in Online Social Networks}, visible_on_website = {YES}, year = {2025} }
Human-AI coevolution

Dino Pedreschi, Luca Pappalardo, Emanuele Ferragina, Ricardo Baeza-Yates, Albert-László Barabási, and 12 more authors

Artificial Intelligence, Feb 2025

RESEARCH LINE
2

Abs Bib HTML

We present xspells, a model-agnostic local approach for explaining the decisions of black box models in classification of short texts. The explanations provided consist of a set of exemplar sentences and a set of counter-exemplar sentences. The former are examples classified by the black box with the same label as the text to explain. The latter are examples classified with a different label (a form of counter-factuals). Both are close in meaning to the text to explain, and both are meaningful sentences – albeit they are synthetically generated. xspells generates neighbors of the text to explain in a latent space using Variational Autoencoders for encoding text and decoding latent instances. A decision tree is learned from randomly generated neighbors, and used to drive the selection of the exemplars and counter-exemplars. Moreover, diversity of counter-exemplars is modeled as an optimization problem, solved by a greedy algorithm with theoretical guarantee. We report experiments on three datasets showing that xspells outperforms the well-known lime method in terms of quality of explanations, fidelity, diversity, and usefulness, and that is comparable to it in terms of stability.
@article{Pedreschi_2025, address = {Netherlands}, author = {Pedreschi, Dino and Pappalardo, Luca and Ferragina, Emanuele and Baeza-Yates, Ricardo and Barabási, Albert-László and Dignum, Frank and Dignum, Virginia and Eliassi-Rad, Tina and Giannotti, Fosca and Kertész, János and Knott, Alistair and Ioannidis, Yannis and Lukowicz, Paul and Passarella, Andrea and Pentland, Alex Sandy and Shawe-Taylor, John and Vespignani, Alessandro}, doi = {10.1016/j.artint.2024.104244}, issn = {0004-3702}, journal = {Artificial Intelligence}, line = {2}, month = feb, open_access = {Gold}, pages = {104244}, publisher = {Elsevier BV}, title = {Human-AI coevolution}, visible_on_website = {YES}, volume = {339}, year = {2025} }
Explanations Go Linear: Post-hoc Explainability for Tabular Data with Interpretable Meta-Encoding

Simone Piaggesi, Riccardo Guidotti, Fosca Giannotti, and Dino Pedreschi

Dec 2025

RESEARCH LINE
1

Bib HTML
@misc{PGG2025, author = {Piaggesi, Simone and Guidotti, Riccardo and Giannotti, Fosca and Pedreschi, Dino}, line = {1}, month = dec, title = {Explanations Go Linear: Post-hoc Explainability for Tabular Data with Interpretable Meta-Encoding}, year = {2025} }
Evaluating the Privacy Exposure of Interpretable Global and Local Explainers.

Francesca Naretto, Anna Monreale, and Fosca Giannotti

Dec 2025

RESEARCH LINE
1

5

Abs Bib HTML

During the last few years, the abundance of data has significantly boosted the performance of Machine Learning models, integrating them into several aspects of daily life. However, the rise of powerful Artificial Intelligence tools has introduced ethical and legal complexities. This paper proposes a computational framework to analyze the ethical and legal dimensions of Machine Learning models, focusing specifically on privacy concerns and interpretability. In fact, recently, the research community proposed privacy attacks able to reveal whether a record was part of the black-box training set or inferring variable values by accessing and querying a Machine Learning model. These attacks highlight privacy vulnerabilities and prove that GDPR regulation might be violated by making data or Machine Learning models accessible. At the same time, the complexity of these models, often labelled as “black-boxes”, has made the development of explanation methods indispensable to enhance trust and facilitate their acceptance and adoption in high-stake scenarios.
@misc{NMG2025, author = {Naretto, Francesca and Monreale, Anna and Giannotti, Fosca}, line = {1,5}, month = dec, publisher = {Trans. Data Priv. 18 (2), 67-93}, title = {Evaluating the Privacy Exposure of Interpretable Global and Local Explainers.}, year = {2025} }

2024

The ALTAI checklist as a tool to assess ethical and legal implications for a trustworthy AI development in education

Andrea Fedele, Clara Punzi, and Stefano Tramacere

Computer Law & Security Review, Jul 2024

RESEARCH LINE
5

Abs Bib HTML

The rapid proliferation of Artificial Intelligence (AI) applications in various domains of our lives has prompted a need for a shift towards a human-centered and trustworthy approach to AI. In this study we employ the Assessment List for Trustworthy Artificial Intelligence (ALTAI) checklist to evaluate the trustworthiness of Artificial Intelligence for Student Performance Prediction (AI4SPP), an AI-powered system designed to detect students at risk of school failure. We strongly support the ethical and legal development of AI and propose an implementation design where the user can choose to have access to each level of a three-tier outcome bundle: the AI prediction alone, the prediction along with its confidence level, and, lastly, local explanations for each grade prediction together with the previous two information. AI4SPP aims to raise awareness among educators and students regarding the factors contributing to low school performance, thereby facilitating the implementation of interventions not only to help students, but also to address biases within the school community. However, we also emphasize the ethical and legal concerns that could arise from a misuse of the AI4SPP tool. First of all, the collection and analysis of data, which is essential for the development of AI models, may lead to breaches of privacy, thus causing particularly adverse consequences in the case of vulnerable individuals. Furthermore, the system’s predictions may be influenced by unacceptable discrimination based on gender, ethnicity, or socio-economic background, leading to unfair actions. The ALTAI checklist serves as a valuable self-assessment tool during the design phase of AI systems, by means of which commonly overlooked weaknesses can be highlighted and addressed. In addition, the same checklist plays a crucial role throughout the AI system life cycle. Continuous monitoring of sensitive features within the dataset, alongside survey assessments to gauge users’ responses to the systems, is essential for gathering insights and intervening accordingly. We argue that adopting a critical approach to AI development is essential for societal progress, believing that it can evolve and accelerate over time without impeding openness to new technologies. By aligning with ethical principles and legal requirements, AI systems can make significant contributions to education while mitigating potential risks and ensuring a fair and inclusive learning environment.
@article{FPT2024, author = {Fedele, Andrea and Punzi, Clara and Tramacere, Stefano}, doi = {10.1016/j.clsr.2024.105986}, issn = {2212-473X}, journal = {Computer Law & Security Review}, line = {5}, month = jul, open_access = {Gold}, pages = {105986}, publisher = {Elsevier BV}, title = {The ALTAI checklist as a tool to assess ethical and legal implications for a trustworthy AI development in education}, visible_on_website = {YES}, volume = {53}, year = {2024} }
Fast, Interpretable, and Deterministic Time Series Classification With a Bag-of-Receptive-Fields

Francesco Spinnato, Riccardo Guidotti, Anna Monreale, and Mirco Nanni

IEEE Access, Jul 2024

RESEARCH LINE
1

Abs Bib HTML

The current trend in time series classification is to develop highly accurate algorithms by combining multiple models in ensemble hybrids, representing time series in complex feature spaces, and extracting features from different representations. As a consequence, the best time series classifiers are black-box models, not understandable for humans. Even the approaches regarded as interpretable, such as shapelet-based ones, rely on randomization to maintain efficiency, which poses challenges for interpretability. We propose the Bag-Of-Receptive-Fields (BORF), a fast, interpretable, and deterministic time series transform. Building on the Bag-Of-Patterns, we bridge the gap between convolutional operators and discretization, enhancing the Symbolic Aggregate Approximation with dilation and stride to better capture temporal patterns at multiple scales. We propose an algorithmic speedup that reduces the time complexity associated with SAX-based classifiers, enabling the more flexible BORF, represented as a sparse multivariate tensor.
@article{SGM2024, author = {Spinnato, Francesco and Guidotti, Riccardo and Monreale, Anna and Nanni, Mirco}, doi = {10.1109/access.2024.3464743}, issn = {2169-3536}, journal = {IEEE Access}, line = {1}, open_access = {Gold}, pages = {137893–137912}, publisher = {Institute of Electrical and Electronics Engineers (IEEE)}, title = {Fast, Interpretable, and Deterministic Time Series Classification With a Bag-of-Receptive-Fields}, visible_on_website = {YES}, volume = {12}, year = {2024} }
FairBelief - Assessing Harmful Beliefs in Language Models

Mattia Setzu, Marta Marchiori Manerba, Pasquale Minervini, and Debora Nozza

In Proceedings of the 4th Workshop on Trustworthy Natural Language Processing (TrustNLP 2024) , Jul 2024

RESEARCH LINE
5

Abs Bib HTML

Language Models (LMs) have been shown to inherit undesired biases that might hurt minorities and underrepresented groups if such systems were integrated into real-world applications without careful fairness auditing.This paper proposes FairBelief, an analytical approach to capture and assess beliefs, i.e., propositions that an LM may embed with different degrees of confidence and that covertly influence its predictions. With FairBelief, we leverage prompting to study the behavior of several state-of-the-art LMs across different previously neglected axes, such as model scale and likelihood, assessing predictions on a fairness dataset specifically designed to quantify LMs’ outputs’ hurtfulness.Finally, we conclude with an in-depth qualitative assessment of the beliefs emitted by the models.We apply FairBelief to English LMs, revealing that, although these architectures enable high performances on diverse natural language processing tasks, they show hurtful beliefs about specific genders. Interestingly, training procedure and dataset, model scale, and architecture induce beliefs of different degrees of hurtfulness.
@inproceedings{SMM2024, author = {Setzu, Mattia and Marchiori Manerba, Marta and Minervini, Pasquale and Nozza, Debora}, booktitle = {Proceedings of the 4th Workshop on Trustworthy Natural Language Processing (TrustNLP 2024)}, doi = {10.18653/v1/2024.trustnlp-1.3}, line = {5}, open_access = {Gold}, pages = {27–39}, publisher = {Association for Computational Linguistics}, title = {FairBelief - Assessing Harmful Beliefs in Language Models}, visible_on_website = {YES}, year = {2024} }
A Survey on Graph Counterfactual Explanations: Definitions, Methods, Evaluation, and Research Challenges

Mario Alfonso Prado-Romero, Bardh Prenkaj, Giovanni Stilo, and Fosca Giannotti

ACM Computing Surveys, Apr 2024

RESEARCH LINE
1

Abs Bib HTML

Graph Neural Networks (GNNs) perform well in community detection and molecule classification. Counterfactual Explanations (CE) provide counter-examples to overcome the transparency limitations of black-box models. Due to the growing attention in graph learning, we focus on the concepts of CE for GNNs. We analysed the SoA to provide a taxonomy, a uniform notation, and the benchmarking datasets and evaluation metrics. We discuss fourteen methods, their evaluation protocols, twenty-two datasets, and nineteen metrics. We integrated the majority of methods into the GRETEL library to conduct an empirical evaluation to understand their strengths and pitfalls. We highlight open challenges and future work.
@article{PPS2024, author = {Prado-Romero, Mario Alfonso and Prenkaj, Bardh and Stilo, Giovanni and Giannotti, Fosca}, doi = {10.1145/3618105}, issn = {1557-7341}, journal = {ACM Computing Surveys}, line = {1}, month = apr, number = {7}, open_access = {Gold}, pages = {1–37}, publisher = {Association for Computing Machinery (ACM)}, title = {A Survey on Graph Counterfactual Explanations: Definitions, Methods, Evaluation, and Research Challenges}, visible_on_website = {YES}, volume = {56}, year = {2024} }
FLocalX - Local to Global Fuzzy Explanations for Black Box Classifiers

Guillermo Fernandez, Riccardo Guidotti, Fosca Giannotti, Mattia Setzu, Juan A. Aledo, and 2 more authors

Apr 2024

RESEARCH LINE
1

2

Abs Bib HTML

The need for explanation for new, complex machine learning models has caused the rise and growth of the field of eXplainable Artificial Intelligence. Different explanation types arise, such as local explanations which focus on the classification for a particular instance, or global explanations which aim to show a global overview of the inner workings of the model. In this paper, we propose FLocalX, a framework that builds a fuzzy global explanation expressed in terms of fuzzy rules by using local explanations as a starting point and a metaheuristic optimization process to obtain the result. An initial experimentation has been carried out with a genetic algorithm as the optimization process. Across several datasets, black-box algorithms and local explanation methods, FLocalX has been tested in terms of both fidelity of the resulting global explanation, and complexity The results show that FLocalX is successfully able to generate short and understandable global explanations that accurately imitate the classifier.
@inbook{FGG2024, address = {Cham, Switzerland}, author = {Fernandez, Guillermo and Guidotti, Riccardo and Giannotti, Fosca and Setzu, Mattia and Aledo, Juan A. and Gámez, Jose A. and Puerta, Jose M.}, booktitle = {Advances in Intelligent Data Analysis XXII}, doi = {10.1007/978-3-031-58553-1_16}, isbn = {9783031585531}, issn = {1611-3349}, line = {1,2}, open_access = {NO}, pages = {197–209}, publisher = {Springer Nature Switzerland}, title = {FLocalX - Local to Global Fuzzy Explanations for Black Box Classifiers}, visible_on_website = {YES}, year = {2024} }
An Interactive Interface for Feature Space Navigation

Eleonora Cappuccio, Isacco Beretta, Marta Marchiori Manerba, and Salvatore Rinzivillo

Jun 2024

RESEARCH LINE
3

Abs Bib HTML

In this paper, we present Feature Space Navigator, an interactive interface that allows an exploration of the decision boundary of a model. The proposal aims to provide users with an intuitive and direct way to navigate through the feature space, inspect model behavior, and perform what-if analyses via feature manipulations and visual feedback. We integrate multiple views including projections of high-dimensional data, decision boundary surfaces, and sensitivity indicators. The interface also supports real-time adjustments of feature values to observe the corresponding changes in the model predictions. Our experiments show that the system can help both novice and expert users to detect regions of uncertainty, identify influential features, and generate hypotheses for model improvement.
@inbook{CBM2024, author = {Cappuccio, Eleonora and Beretta, Isacco and Marchiori Manerba, Marta and Rinzivillo, Salvatore}, booktitle = {HHAI 2024: Hybrid Human AI Systems for the Social Good}, doi = {10.3233/faia240184}, isbn = {9781643685229}, issn = {1879-8314}, line = {3}, month = jun, open_access = {Gold}, publisher = {IOS Press}, title = {An Interactive Interface for Feature Space Navigation}, visible_on_website = {YES}, year = {2024} }
Explainable Authorship Identification in Cultural Heritage Applications

Mattia Setzu, Silvia Corbara, Anna Monreale, Alejandro Moreo, and Fabrizio Sebastiani

Journal on Computing and Cultural Heritage, Jun 2024

RESEARCH LINE
1

Abs Bib HTML

While a substantial amount of work has recently been devoted to enhance the performance of computational Authorship Identification (AId) systems, little to no attention has been paid to endowing AId systems with the ability to explain the reasons behind their predictions. This lacking substantially hinders the practical employment of AId methodologies, since the predictions returned by such systems are hardly useful unless they are supported with suitable explanations. In this paper, we explore the applicability of existing general-purpose eXplainable Artificial Intelligence (XAI) techniques to AId, with a special focus on explanations addressed to scholars working in cultural heritage. In particular, we assess the relative merits of three different types of XAI techniques (feature ranking, probing, factuals and counterfactual selection) on three different AId tasks (authorship attribution, authorship verification, same-authorship verification) by running experiments on real AId data. Our analysis shows that, while these techniques make important first steps towards explainable Authorship Identification, more work remains to be done in order to provide tools that can be profitably integrated in the workflows of scholars.
@article{SCM2024, author = {Setzu, Mattia and Corbara, Silvia and Monreale, Anna and Moreo, Alejandro and Sebastiani, Fabrizio}, doi = {10.1145/3654675}, issn = {1556-4711}, journal = {Journal on Computing and Cultural Heritage}, line = {1}, month = jun, number = {3}, open_access = {Gold}, pages = {1–23}, publisher = {Association for Computing Machinery (ACM)}, title = {Explainable Authorship Identification in Cultural Heritage Applications}, visible_on_website = {YES}, volume = {17}, year = {2024} }
Towards Transparent Healthcare: Advancing Local Explanation Methods in Explainable Artificial Intelligence

Carlo Metta, Andrea Beretta, Roberto Pellungrini, Salvatore Rinzivillo, and Fosca Giannotti

Bioengineering, Apr 2024

RESEARCH LINE
1

Abs Bib HTML

Local explanation methods, such as SHAP and LIME, are increasingly adopted to justify predictions of clinical decision support systems. However, their reliability and clinical usefulness remain limited by instability, lack of contextualization, and poor alignment with medical reasoning. In this work, we propose an enhanced pipeline for generating trustworthy local explanations in healthcare. Our approach incorporates domain constraints, medical ontologies, and temporal reasoning over patient histories. We evaluate the method on multiple clinical prediction tasks and compare it against standard explainability tools using expert-driven criteria. Results show that explanations become more stable and more aligned with clinically plausible factors. A qualitative analysis with clinicians further indicates improved interpretability and actionability, supporting safer and more transparent AI-assisted healthcare.
@article{MBP2024b, author = {Metta, Carlo and Beretta, Andrea and Pellungrini, Roberto and Rinzivillo, Salvatore and Giannotti, Fosca}, doi = {10.3390/bioengineering11040369}, issn = {2306-5354}, journal = {Bioengineering}, line = {1}, month = apr, number = {4}, open_access = {Gold}, pages = {369}, publisher = {MDPI AG}, title = {Towards Transparent Healthcare: Advancing Local Explanation Methods in Explainable Artificial Intelligence}, visible_on_website = {YES}, volume = {11}, year = {2024} }
Drifting explanations in continual learning

Andrea Cossu, Francesco Spinnato, Riccardo Guidotti, and Davide Bacciu

Neurocomputing, Sep 2024

RESEARCH LINE
1

Abs Bib HTML

Continual Learning (CL) trains models on streams of data, with the aim of learning new information without forgetting previous knowledge. We study the behavior of different explanation methods in CL and propose CLEX (ContinuaL EXplanations), an evaluation protocol to robustly assess the change of explanations in Class-Incremental scenarios, where forgetting is pronounced. We observed that models with similar predictive accuracy do not generate similar explanations. (openportal.isti.cnr.it)
@article{CSG2024, author = {Cossu, Andrea and Spinnato, Francesco and Guidotti, Riccardo and Bacciu, Davide}, doi = {10.1016/j.neucom.2024.127960}, issn = {0925-2312}, journal = {Neurocomputing}, line = {1}, month = sep, open_access = {Gold}, pages = {127960}, publisher = {Elsevier BV}, title = {Drifting explanations in continual learning}, visible_on_website = {YES}, volume = {597}, year = {2024} }
Advancing Dermatological Diagnostics: Interpretable AI for Enhanced Skin Lesion Classification

Carlo Metta, Andrea Beretta, Riccardo Guidotti, Yuan Yin, Patrick Gallinari, and 2 more authors

Diagnostics, Apr 2024

RESEARCH LINE
1

2

Abs Bib HTML

A crucial challenge in critical settings like medical diagnosis is making deep learning models used in decision-making systems interpretable. Efforts in Explainable Artificial Intelligence (XAI) are underway to address this challenge. Yet, many XAI methods are evaluated on broad classifiers and fail to address complex, real-world issues, such as medical diagnosis. In our study, we focus on enhancing user trust and confidence in automated AI decision-making systems, particularly for diagnosing skin lesions, by tailoring an XAI method to explain an AI model’s ability to identify various skin lesion types. We generate explanations using synthetic images of skin lesions as examples and counterexamples, offering a method for practitioners to pinpoint the critical features influencing the classification outcome. A validation survey involving domain experts, novices, and laypersons has demonstrated that explanations increase trust and confidence in the automated decision system. Furthermore, our exploration of the model’s latent space reveals clear separations among the most common skin lesion classes, a distinction that likely arises from the unique characteristics of each class and could assist in correcting frequent misdiagnoses by human professionals.
@article{MBG2024, author = {Metta, Carlo and Beretta, Andrea and Guidotti, Riccardo and Yin, Yuan and Gallinari, Patrick and Rinzivillo, Salvatore and Giannotti, Fosca}, doi = {10.3390/diagnostics14070753}, issn = {2075-4418}, journal = {Diagnostics}, line = {1,2}, month = apr, number = {7}, open_access = {Gold}, pages = {753}, publisher = {MDPI AG}, title = {Advancing Dermatological Diagnostics: Interpretable AI for Enhanced Skin Lesion Classification}, visible_on_website = {YES}, volume = {14}, year = {2024} }
Mapping the landscape of ethical considerations in explainable AI research

Luca Nannini, Marta Marchiori Manerba, and Isacco Beretta

Ethics and Information Technology, Jun 2024

RESEARCH LINE
1

5

Abs Bib HTML

With its potential to contribute to the ethical governance of AI, explainable AI (XAI) research frequently asserts its relevance to ethical considerations. Yet the substantiation of these claims with rigorous ethical analysis and reflection remains largely unexamined. This contribution endeavours to scrutinise the relationship between XAI and ethical considerations. We conduct a systematic review of the XAI literature by applying a multi-stage filtering process and then developing a taxonomy for classifying the depth and quality of ethical engagement in the field. Our findings show that although a growing body of XAI research references ethical issues, the majority does so at a superficial level. We identify nine ethical themes (e.g., autonomy, justice, transparency) and show how they are engaged in the literature. We discuss trends, deficits, and future directions for integrating ethical considerations meaningfully into XAI research.
@article{NMB2024, author = {Nannini, Luca and Marchiori Manerba, Marta and Beretta, Isacco}, doi = {10.1007/s10676-024-09773-7}, issn = {1572-8439}, journal = {Ethics and Information Technology}, line = {1,5}, month = jun, number = {3}, open_access = {Gold}, publisher = {Springer Science and Business Media LLC}, title = {Mapping the landscape of ethical considerations in explainable AI research}, visible_on_website = {YES}, volume = {26}, year = {2024} }
Commodity-specific triads in the Dutch inter-industry production network

Marzio Di Vece, Frank P. Pijpers, and Diego Garlaschelli

Scientific Reports, Feb 2024

RESEARCH LINE
4

Abs Bib HTML

Triadic motifs are the smallest building blocks of higher-order interactions in complex networks and can be detected as over-occurrences with respect to null models with only pair-wise interactions. Recently, the motif structure of production networks has attracted attention in light of its possible role in the propagation of economic shocks. However, its characterization at the level of individual commodities is still poorly understood. Here we analyze both binary and weighted triadic motifs in the Dutch inter-industry production network disaggregated at the level of 187 commodity groups, which Statistics Netherlands reconstructed from National Accounts registers, surveys and known empirical data. We introduce appropriate null models that filter out node heterogeneity and the strong effects of link reciprocity and find that, while the aggregate network that overlays all products is characterized by a multitude of triadic motifs, most single-product layers feature no significant motif, and roughly 85% of the layers feature only two motifs or less. This result paves the way for identifying a simple ‘triadic fingerprint’ of each commodity and for reconstructing most product-specific networks from partial information in a pairwise fashion by controlling for their reciprocity structure. We discuss how these results can help statistical bureaus identify fine-grained information in structural analyses of interest for policymakers.
@article{DPG2024, author = {Di Vece, Marzio and Pijpers, Frank P. and Garlaschelli, Diego}, doi = {10.1038/s41598-024-53655-3}, issn = {2045-2322}, journal = {Scientific Reports}, line = {4}, month = feb, number = {1}, open_access = {Gold}, publisher = {Springer Science and Business Media LLC}, title = {Commodity-specific triads in the Dutch inter-industry production network}, visible_on_website = {YES}, volume = {14}, year = {2024} }
Exploring Large Language Models Capabilities to Explain Decision Trees

Paulo Bruno Serafim, Pierluigi Crescenzi, Gizem Gezici, Eleonora Cappuccio, Salvatore Rinzivillo, and 1 more author

Jun 2024

RESEARCH LINE
1

Abs Bib HTML

Decision trees are widely adopted in Machine Learning tasks due to their operation simplicity and interpretability aspects. However, following the decision process path taken by trees can be difficult in a complex scenario or in a case where a user has no familiarity with them. Prior research showed that converting outcomes to natural language is an accessible way to facilitate understanding for non-expert users in several tasks. More recently, there has been a growing effort to use Large Language Models (LLMs) as a tool for providing natural language texts. In this paper, we examine the proficiency of LLMs to explain decision tree predictions in simple terms through the generation of natural language explanations. By exploring different textual representations and prompt engineering strategies, we identify capabilities that strengthen LLMs as a competent explainer as well as highlight potential challenges and limitations, opening further research possibilities on natural language explanations for decision trees.
@inbook{SGC2024, author = {Serafim, Paulo Bruno and Crescenzi, Pierluigi and Gezici, Gizem and Cappuccio, Eleonora and Rinzivillo, Salvatore and Giannotti, Fosca}, booktitle = {HHAI 2024: Hybrid Human AI Systems for the Social Good}, doi = {10.3233/faia240183}, isbn = {9781643685229}, issn = {1879-8314}, line = {1}, month = jun, open_access = {Gold}, publisher = {IOS Press}, title = {Exploring Large Language Models Capabilities to Explain Decision Trees}, visible_on_website = {YES}, year = {2024} }
Explaining Siamese networks in few-shot learning

Andrea Fedele, Riccardo Guidotti, and Dino Pedreschi

Machine Learning, Apr 2024

RESEARCH LINE
1

Abs Bib HTML

Siamese neural networks are widely used in few-shot learning tasks thanks to their ability to compare pairs of samples and generalize from very limited labeled data. However, their internal decision-making process remains opaque, since similarity-based representations do not provide intuitive explanations for end users. In this work, we investigate how to explain Siamese networks by attributing contribution scores to both input samples involved in the comparison. We introduce an explanation method specifically tailored to pairwise architectures, producing two synchronized saliency maps that highlight which regions of the support and query examples drive the similarity judgment. We evaluate the approach on image-based few-shot classification benchmarks, showing that the explanations highlight semantically meaningful structures and remain consistent across different evaluation episodes.
@article{FGP2024, author = {Fedele, Andrea and Guidotti, Riccardo and Pedreschi, Dino}, doi = {10.1007/s10994-024-06529-8}, issn = {1573-0565}, journal = {Machine Learning}, line = {1}, month = apr, number = {10}, open_access = {Gold}, pages = {7723–7760}, publisher = {Springer Science and Business Media LLC}, title = {Explaining Siamese networks in few-shot learning}, visible_on_website = {YES}, volume = {113}, year = {2024} }
Counterfactual and Prototypical Explanations for Tabular Data via Interpretable Latent Space

Simone Piaggesi, Francesco Bodria, Riccardo Guidotti, Fosca Giannotti, and Dino Pedreschi

IEEE Access, Apr 2024

RESEARCH LINE
1

Abs Bib HTML

We evaluated the effectiveness of the created latent space by showing its capability to preserve pair-wise similarities similarly to well-known dimensionality reduction techniques. Our approach introduces a transparent latent space optimized for interpretability of both counterfactual and prototypical explanations for tabular data. The approach enables the easy extraction of local and global explanations and ensures that the latent space preserves similarity relations, enabling meaningful prototypical and counterfactual examples for any classifier.
@article{PBG2024, author = {Piaggesi, Simone and Bodria, Francesco and Guidotti, Riccardo and Giannotti, Fosca and Pedreschi, Dino}, doi = {10.1109/access.2024.3496114}, issn = {2169-3536}, journal = {IEEE Access}, line = {1}, open_access = {Gold}, pages = {168983–169000}, publisher = {Institute of Electrical and Electronics Engineers (IEEE)}, title = {Counterfactual and Prototypical Explanations for Tabular Data via Interpretable Latent Space}, visible_on_website = {YES}, volume = {12}, year = {2024} }
Multi-Perspective Stance Detection

Benedetta Muscato, Praveen Bushipaka, Gizem Gezici, Lucia Passaro, and Fosca Giannotti

Dec 2024

RESEARCH LINE
4

5

Abs Bib HTML

Subjective NLP tasks usually rely on human annotations provided by multiple annotators, whose judgments may vary due to their diverse backgrounds and life experiences. Traditional methods often aggregate multiple annotations into a single ground truth, disregarding the diversity in perspectives that arises from annotator disagreement. In this preliminary study, we examine the effect of including multiple annotations on model accuracy in classification. Our methodology investigates the performance of perspectiveaware classification models in stance detection task and further inspects if annotator disagreement affects the model confidence. The results show that multi-perspective approach yields better classification performance outperforming the baseline which uses the single label. This entails that designing more inclusive perspective-aware AI models is not only an essential first step in implementing responsible and ethical AI, but it can also achieve superior results than using the traditional approaches.
@misc{MBG2024bb, address = {Aachen, Germany}, author = {Muscato, Benedetta and Bushipaka, Praveen and Gezici, Gizem and Passaro, Lucia and Giannotti, Fosca}, line = {4,5}, month = dec, title = {Multi-Perspective Stance Detection}, year = {2024} }
Beyond Headlines: A Corpus of Femicides News Coverage in Italian Newspapers

Eleonora Cappuccio, Benedetta Muscato, Laura Pollacci, Marta Marchiori Manerba, Clara Punzi, and 5 more authors

Dec 2024

RESEARCH LINE
4

5

Abs Bib HTML

How newspapers cover news significantly impacts how facts are understood, perceived, and processed by the public. This is especially crucial when serious crimes are reported, e.g., in the case of femicides, where the description of the perpetrator and the victim builds a strong, often polarized opinion of this severe societal issue. This paper presents FMNews, a new dataset of articles reporting femicides extracted from Italian newspapers. Our core contribution aims to promote the development of a deeper framing and awareness of the phenomenon through an original resource available and accessible to the research community, facilitating further analyses on the topic. The paper also provides a preliminary study of the resulting collection through several example use cases and scenarios.
@misc{CMP2024, address = {Aachen, Germany}, author = {Cappuccio, Eleonora and Muscato, Benedetta and Pollacci, Laura and Manerba, Marta Marchiori and Punzi, Clara and Mala, Chandana Sree and Lalli, Margherita and Gezici, Gizem and Natilli, Michela and Giannotti, Fosca}, line = {4,5}, month = dec, title = {Beyond Headlines: A Corpus of Femicides News Coverage in Italian Newspapers}, year = {2024} }
Requirements of eXplainable AI in Algorithmic Hiring

A. Beretta, G. Ercoli, A. Ferraro, R. Guidotti, A. Iommi, and 4 more authors

Dec 2024

RESEARCH LINE
1

Abs Bib HTML

AI models for ranking candidates to a job position are increasingly adopted. They bring a new layer of opaqueness in the way candidates are evaluated. We present preliminary research on stakeholder analysis and requirement elicitation for designing an explainability component in AI models for ranking candidates to a job position. (CEUR-WS)
@misc{BEF2024, author = {Beretta, A. and Ercoli, G. and Ferraro, A. and Guidotti, R. and Iommi, A. and Mastropietro, A. and Monreale, A. and Rotelli, D. and Ruggieri, S.}, line = {1}, month = dec, title = {Requirements of eXplainable AI in Algorithmic Hiring}, year = {2024} }
The ethical impact assessment of selling life insurance to titanic passengers

Gezici, Gizem; Mannari, Chiara; Orlandi, and Lorenzo

Dec 2024

RESEARCH LINE
5

Abs Bib

The Artificial Intelligence Act (AIA) is a uniform legal framework to ensure that AI systems within the European Union (EU) are safe and comply with existing law on fundamental rights and constitutional values. The AIA adopts a risk-based approach with the aim of intending to regulate AI systems, especially categorised as high-risk, which have significant harmful impacts on the health, safety and fundamental rights of persons in the Union. The AIA is founded on the Ethics Guidelines of the High-Level Expert Group for Trustworthy AI, which are grounded in fundamental rights and reflect four ethical imperatives in order to ensure ethical and robust AI. While we acknowledge that ethics is not law, we advocate that the analysis of ethical risks can assist us in complying with laws, thereby facilitating the implementation of the AIA requirements. Thus, we first design an AI-driven Decision Support System for individual risk prediction in the insurance domain (categorised as high-risk by the AIA) based on the Titanic case, which is a popular benchmark dataset in machine learning. We then fulfill an ethical impact assessment of the Titanic case study, relying on the four ethical imperatives of respect for human autonomy, prevention of harm, fairness, and explicability, declared by the High-Level Expert Group for Trustworthy AI. In the context of this ethical impact assessment, we also refer to the questions in the ALTAI checklist. Our discussions regarding the ethical impact assessment in the insurance domain demonstrate that ethical principles can intersect but also create tensions (intriguingly, only in this particular context), for which there is no definitive solution. When tensions arise, which may result in unavoidable trade-offs, these trade-offs should be addressed in a rational and methodical manner, paying special attention to the context of the current case study being evaluated.
@misc{GMO2024, author = {Gezici and Mannari, Gizem; and Orlandi, Chiara; and Lorenzo}, line = {5}, month = dec, title = {The ethical impact assessment of selling life insurance to titanic passengers}, year = {2024} }
XAI in healthcare

Gezici G.; Metta C; Beretta A.; Pellungrini R.; Rinzivillo S.; Pedreschi D.; Giannotti F.

Dec 2024

RESEARCH LINE
4

5

Abs Bib HTML

The evolution of Explainable Artificial Intelligence (XAI) within healthcare represents a crucial turn towards more transparent, understandable, and patient-centric AI applications. The main objective is not only to increase the accuracy of AI models but also, and more importantly, to establish user trust in decision support systems through improving their interpretability. This extended abstract outlines the ongoing efforts and advancements of our lab addressing the challenges brought up by complex AI systems in healthcare domain. Currently, there are four main projects: Prostate Imaging Cancer AI, Liver Transplantation & Diabetes, Breast Cancer, and Doctor XAI, and ABELE.
@misc{GMB2024, author = {F., Gezici G.; Metta C; Beretta A.; Pellungrini R.; Rinzivillo S.; Pedreschi D.; Giannotti}, line = {4,5}, month = dec, title = {XAI in healthcare}, year = {2024} }
An Overview of Recent Approaches to Enable Diversity in Large Language Models through Aligning with Human Perspectives

Benedetta Muscato, Chandana Sree Mala, Marta Marchiori Manerba, Gizem Gezici, and Fosca Giannotti

Dec 2024

RESEARCH LINE
3

Abs Bib HTML

The varied backgrounds and experiences of human annotators inject different opinions and potential biases into the data, inevitably leading to disagreements. Yet, traditional aggregation methods fail to capture individual judgments since they rely on the notion of a single ground truth. Our aim is to review prior contributions to pinpoint the shortcomings that might cause stereotypical content generation. As a preliminary study, our purpose is to investigate state-of-the-art approaches, primarily focusing on the following two research directions. First, we investigate how adding subjectivity aspects to LLMs might guarantee diversity. We then look into the alignment between humans and LLMs and discuss how to measure it. Considering existing gaps, our review explores possible methods to mitigate the perpetuation of biases targeting specific communities. However, we recognize the potential risk of disseminating sensitive information due to the utilization of socio-demographic data in the training process. These considerations underscore the inclusion of diverse perspectives while taking into account the critical importance of implementing robust safeguards to protect individuals’ privacy and prevent the inadvertent propagation of sensitive information.
@misc{MMM2024, address = {Torino, Italia}, author = {Muscato, Benedetta and Mala, Chandana Sree and Manerba, Marta Marchiori and Gezici, Gizem and Giannotti, Fosca}, line = {3}, month = dec, pages = {49--55}, publisher = {ELRA and ICCL}, title = {An Overview of Recent Approaches to Enable Diversity in Large Language Models through Aligning with Human Perspectives}, year = {2024} }

2023

Topics in Selective Classification

Andrea Pugnana

Proceedings of the AAAI Conference on Artificial Intelligence, Jun 2023

RESEARCH LINE
1

2

Abs Bib HTML

Selective classification (also known as classification with a reject option, or learning to defer) extends a classifier with a selection function (reject option/strategy) to determine whether or not a prediction should be accepted. This mechanism allows the AI system to abstain in those instances where the classifier is more uncertain about the class to predict, introducing a trade-off between performance and coverage (the percentage of cases where the classifier does not abstain). The reject option has been extensively studied from a theoretical standpoint. However, state-of-the-art practical approaches and tools are model-specific, e.g., they are tailored to Deep Neural Networks and focused/experimented mainly on image datasets. In this work I developed a model-agnostic heuristics that is able to lift any (probabilistic) classifier into a selective classifier. The approach exploits both a cross-fitting strategy and results from quantile estimation to build the selective function. The algorithm is tested on several real-world datasets, showing improvements compared to existing methodologies.
@article{P2024, author = {Pugnana, Andrea}, doi = {10.1609/aaai.v37i13.26925}, issn = {2159-5399}, journal = {Proceedings of the AAAI Conference on Artificial Intelligence}, line = {1,2}, month = jun, number = {13}, open_access = {Gold}, pages = {16129–16130}, publisher = {Association for the Advancement of Artificial Intelligence (AAAI)}, title = {Topics in Selective Classification}, visible_on_website = {YES}, volume = {37}, year = {2023} }
Benchmarking and survey of explanation methods for black box models

Francesco Bodria, Fosca Giannotti, Riccardo Guidotti, Francesca Naretto, Dino Pedreschi, and 1 more author

Data Mining and Knowledge Discovery, Jun 2023

RESEARCH LINE
1

3

Abs Bib HTML

The rise of sophisticated black-box machine learning models in Artificial Intelligence systems has prompted the need for explanation methods that reveal how these models work in an understandable way to users and decision makers. Unsurprisingly, the state-of-the-art exhibits currently a plethora of explainers providing many different types of explanations. With the aim of providing a compass for researchers and practitioners, this paper proposes a categorization of explanation methods from the perspective of the type of explanation they return, also considering the different input data formats. The paper accounts for the most representative explainers to date, also discussing similarities and discrepancies of returned explanations through their visual appearance. A companion website to the paper is provided as a continuous update to new explainers as they appear. Moreover, a subset of the most robust and widely adopted explainers, are benchmarked with respect to a repertoire of quantitative metrics.
@article{BGG2023, address = {Netherlands}, author = {Bodria, Francesco and Giannotti, Fosca and Guidotti, Riccardo and Naretto, Francesca and Pedreschi, Dino and Rinzivillo, Salvatore}, doi = {10.1007/s10618-023-00933-9}, issn = {1573-756X}, journal = {Data Mining and Knowledge Discovery}, line = {1,3}, month = jun, number = {5}, open_access = {Gold}, pages = {1719–1778}, publisher = {Springer Science and Business Media LLC}, title = {Benchmarking and survey of explanation methods for black box models}, visible_on_website = {YES}, volume = {37}, year = {2023} }
Effects of Route Randomization on Urban Emissions

Giuliano Cornacchia, Mirco Nanni, Dino Pedreschi, and Luca Pappalardo

SUMO Conference Proceedings, Jun 2023

RESEARCH LINE
4

5

Abs Bib HTML

Routing algorithms typically suggest the fastest path or slight variation to reach a user’s desired destination. Although this suggestion at the individual level is undoubtedly advantageous for the user, from a collective point of view, the aggregation of all single suggested paths may result in an increasing impact (e.g., in terms of emissions). In this study, we use SUMO to simulate the effects of incorporating randomness into routing algorithms on emissions, their distribution, and travel time in the urban area of Milan (Italy). Our results reveal that, given the common practice of routing towards the fastest path, a certain level of randomness in routes reduces emissions and travel time. In other words, the stronger the random component in the routes, the more pronounced the benefits upon a certain threshold. Our research provides insight into the potential advantages of considering collective outcomes in routing decisions and highlights the need to explore further the relationship between route randomization and sustainability in urban transportation.
@article{CNP2023, author = {Cornacchia, Giuliano and Nanni, Mirco and Pedreschi, Dino and Pappalardo, Luca}, doi = {10.52825/scp.v4i.217}, issn = {2750-4425}, journal = {SUMO Conference Proceedings}, line = {4,5}, month = jun, open_access = {Gold}, pages = {75–87}, publisher = {TIB Open Publishing}, title = {Effects of Route Randomization on Urban Emissions}, visible_on_website = {YES}, volume = {4}, year = {2023} }
Improving trust and confidence in medical skin lesion diagnosis through explainable deep learning

Carlo Metta, Andrea Beretta, Riccardo Guidotti, Yuan Yin, Patrick Gallinari, and 2 more authors

International Journal of Data Science and Analytics, Jun 2023

RESEARCH LINE
1

3

Abs Bib HTML

A key issue in critical contexts such as medical diagnosis is the interpretability of the deep learning models adopted in decision-making systems. Research in eXplainable Artificial Intelligence (XAI) is trying to solve this issue. However, often XAI approaches are only tested on generalist classifier and do not represent realistic problems such as those of medical diagnosis. In this paper, we aim at improving the trust and confidence of users towards automatic AI decision systems in the field of medical skin lesion diagnosis by customizing an existing XAI approach for explaining an AI model able to recognize different types of skin lesions. The explanation is generated through the use of synthetic exemplar and counter-exemplar images of skin lesions and our contribution offers the practitioner a way to highlight the crucial traits responsible for the classification decision. A validation survey with domain experts, beginners, and unskilled people shows that the use of explanations improves trust and confidence in the automatic decision system. Also, an analysis of the latent space adopted by the explainer unveils that some of the most frequent skin lesion classes are distinctly separated. This phenomenon may stem from the intrinsic characteristics of each class and may help resolve common misclassifications made by human experts.
@article{MBG2023, address = {Berlin/Heidelberg, Germany}, author = {Metta, Carlo and Beretta, Andrea and Guidotti, Riccardo and Yin, Yuan and Gallinari, Patrick and Rinzivillo, Salvatore and Giannotti, Fosca}, doi = {10.1007/s41060-023-00401-z}, issn = {2364-4168}, journal = {International Journal of Data Science and Analytics}, line = {1,3}, month = jun, number = {1}, open_access = {Gold}, pages = {183–195}, publisher = {Springer Science and Business Media LLC}, title = {Improving trust and confidence in medical skin lesion diagnosis through explainable deep learning}, visible_on_website = {YES}, volume = {20}, year = {2023} }
Explaining Socio-Demographic and Behavioral Patterns of Vaccination Against the Swine Flu (H1N1) Pandemic

Clara Punzi, Aleksandra Maslennikova, Gizem Gezici, Roberto Pellungrini, and Fosca Giannotti

Jun 2023

RESEARCH LINE
1

4

Abs Bib HTML

Pandemic vaccination campaigns must account for vaccine skepticism as an obstacle to overcome. Using machine learning to identify behavioral and psychological patterns in public survey datasets can provide valuable insights and inform vaccination campaigns based on empirical evidence. However, we argue that the adoption of local and global explanation methodologies can provide additional support to health practitioners by suggesting personalized communication strategies and revealing potential demographic, social, or structural barriers to vaccination requiring systemic changes. In this paper, we first implement a chain classification model for the adoption of the vaccine during the H1N1 influenza outbreak taking seasonal vaccination information into account, and then compare it with a binary classifier for vaccination to better understand the overall patterns in the data. Following that, we derive and compare global explanations using post-hoc methodologies and interpretable-by-design models. Our findings indicate that socio-demographic factors play a distinct role in the H1N1 vaccination as compared to the general vaccination. Nevertheless, medical recommendation and health insurance remain significant factors for both vaccinations. Then, we concentrated on the subpopulation of individuals who did not receive an H1N1 vaccination despite being at risk of developing severe symptoms. In an effort to assist practitioners in providing effective recommendations to patients, we present rules and counterfactuals for the selected instances based on local explanations. Finally, we raise concerns regarding gender and racial disparities in healthcare access by analysing the interaction effects of sensitive attributes on the model’s output.
@inbook{PMG2023, author = {Punzi, Clara and Maslennikova, Aleksandra and Gezici, Gizem and Pellungrini, Roberto and Giannotti, Fosca}, booktitle = {Explainable Artificial Intelligence}, doi = {10.1007/978-3-031-44067-0_31}, isbn = {9783031440670}, issn = {1865-0937}, line = {1,4}, open_access = {Gold}, pages = {621–635}, publisher = {Springer Nature Switzerland}, title = {Explaining Socio-Demographic and Behavioral Patterns of Vaccination Against the Swine Flu (H1N1) Pandemic}, visible_on_website = {YES}, year = {2023} }
Reason to Explain: Interactive Contrastive Explanations (REASONX)

Laura State, Salvatore Ruggieri, and Franco Turini

Jun 2023

RESEARCH LINE
1

3

Abs Bib HTML

Many high-performing machine learning models are not interpretable. As they are increasingly used in decision scenarios that can critically affect individuals, it is necessary to develop tools to better understand their outputs. Popular explanation methods include contrastive explanations. However, they suffer several shortcomings, among others an insufficient incorporation of background knowledge, and a lack of interactivity. While (dialogue-like) interactivity is important to better communicate an explanation, background knowledge has the potential to significantly improve their quality, e.g., by adapting the explanation to the needs of the end-user. To close this gap, we present REASONX, an explanation tool based on Constraint Logic Programming (CLP). REASONX provides interactive contrastive explanations that can be augmented by background knowledge, and allows to operate under a setting of under-specified information, leading to increased flexibility in the provided explanations. REASONX computes factual and contrastive decision rules, as well as closest contrastive examples. It provides explanations for decision trees, which can be the ML models under analysis, or global/local surrogate models of any ML model. While the core part of REASONX is built on CLP, we also provide a program layer that allows to compute the explanations via Python, making the tool accessible to a wider audience. We illustrate the capability of REASONX on a synthetic data set, and on a well-developed example in the credit domain. In both cases, we can show how REASONX can be flexibly used and tailored to the needs of the user.
@inbook{SRT2023, author = {State, Laura and Ruggieri, Salvatore and Turini, Franco}, booktitle = {Explainable Artificial Intelligence}, doi = {10.1007/978-3-031-44064-9_22}, isbn = {9783031440649}, issn = {1865-0937}, line = {1,3}, open_access = {NO}, pages = {421–437}, publisher = {Springer Nature Switzerland}, title = {Reason to Explain: Interactive Contrastive Explanations (REASONX)}, visible_on_website = {YES}, year = {2023} }
Understanding Any Time Series Classifier with a Subsequence-based Explainer

Francesco Spinnato, Riccardo Guidotti, Anna Monreale, Mirco Nanni, Dino Pedreschi, and 1 more author

ACM Transactions on Knowledge Discovery from Data, Nov 2023

RESEARCH LINE
1

Abs Bib HTML

The growing availability of time series data has increased the usage of classifiers for this data type. Unfortunately, state-of-the-art time series classifiers are black-box models and, therefore, not usable in critical domains such as healthcare or finance, where explainability can be a crucial requirement. This paper presents a framework to explain the predictions of any black-box classifier for univariate and multivariate time series. The provided explanation is composed of three parts. First, a saliency map highlighting the most important parts of the time series for the classification. Second, an instance-based explanation exemplifies the black-box’s decision by providing a set of prototypical and counterfactual time series. Third, a factual and counterfactual rule-based explanation, revealing the reasons for the classification through logical conditions based on subsequences that must, or must not, be contained in the time series. Experiments and benchmarks show that the proposed method provides faithful, meaningful, stable, and interpretable explanations.”
@article{SGM2023, author = {Spinnato, Francesco and Guidotti, Riccardo and Monreale, Anna and Nanni, Mirco and Pedreschi, Dino and Giannotti, Fosca}, doi = {10.1145/3624480}, issn = {1556-472X}, journal = {ACM Transactions on Knowledge Discovery from Data}, line = {1}, month = nov, number = {2}, open_access = {Gold}, pages = {1–34}, publisher = {Association for Computing Machinery (ACM)}, title = {Understanding Any Time Series Classifier with a Subsequence-based Explainer}, visible_on_website = {YES}, volume = {18}, year = {2023} }
Declarative Reasoning on Explanations Using Constraint Logic Programming

Laura State, Salvatore Ruggieri, and Franco Turini

Nov 2023

RESEARCH LINE
2

Abs Bib HTML

Explaining opaque Machine Learning models is an increasingly relevant problem. Current explanation in AI methods suffer several shortcomings, among others an insufficient incorporation of background knowledge, and a lack of abstraction and interactivity with the user. We propose REASONX, an explanation method based on Constraint Logic Programming. REASONX can provide declarative, interactive explanations for decision trees, which can be the machine learning models under analysis or global or local surrogate models of any black-box model. Users can express background or common sense knowledge using linear constraints and MILP optimization over features of factual and contrastive instances, and interact with the answer constraints at different levels of abstraction through constraint projection. We present here the architecture of REASONX, which consists of a Python layer, closer to the user, and a CLP layer.
@inbook{SRT2023b, author = {State, Laura and Ruggieri, Salvatore and Turini, Franco}, booktitle = {Logics in Artificial Intelligence}, doi = {10.1007/978-3-031-43619-2_10}, isbn = {9783031436192}, issn = {1611-3349}, line = {2}, open_access = {NO}, pages = {132–141}, publisher = {Springer Nature Switzerland}, title = {Declarative Reasoning on Explanations Using Constraint Logic Programming}, visible_on_website = {YES}, year = {2023} }
EXPHLOT: EXplainable Privacy Assessment for Human LOcation Trajectories

Francesca Naretto, Roberto Pellungrini, Salvatore Rinzivillo, and Daniele Fadda

Nov 2023

RESEARCH LINE
1

3

5

Abs Bib HTML

Human mobility data play a crucial role in understanding mobility patterns and developing analytical services across domains such as urban planning, transportation, and public health. However, due to the sensitive nature of this data, identifying privacy risks is essential before deciding to release it publicly. Recent work has proposed using machine learning models for predicting privacy risk on raw mobility trajectories and using SHAP for risk explanation. However, applying SHAP to mobility data results in explanations of limited use both for privacy experts and end-users. In this work, we present EXPHLOT, a novel version of the Expert privacy risk prediction and explanation framework specifically tailored for human mobility data. We leverage state-of-the-art algorithms in time series classification to improve risk prediction while reducing computation time. We also devise an entropy-based mask to efficiently compute SHAP values and develop a module for interactive analysis and visualization of SHAP values over a map, empowering users with an intuitive understanding of privacy risk.
@inbook{NPR2023, author = {Naretto, Francesca and Pellungrini, Roberto and Rinzivillo, Salvatore and Fadda, Daniele}, booktitle = {Discovery Science}, doi = {10.1007/978-3-031-45275-8_22}, isbn = {9783031452758}, issn = {1611-3349}, line = {1,3,5}, open_access = {Gold}, pages = {325–340}, publisher = {Springer Nature Switzerland}, title = {EXPHLOT: EXplainable Privacy Assessment for Human LOcation Trajectories}, visible_on_website = {YES}, year = {2023} }
Deterministic, quenched, and annealed parameter estimation for heterogeneous network models

Marzio Di Vece, Diego Garlaschelli, and Tiziano Squartini

Physical Review E, Nov 2023

RESEARCH LINE
1

Abs Bib HTML

At least two, different approaches to define and solve statistical models for the analysis of economic systems exist: the typical, econometric one, interpreting the Gravity Model specification as the expected link weight of an arbitrary probability distribution, and the one rooted into statistical physics, constructing maximum-entropy distributions constrained to satisfy certain network properties. In a couple of recent, companion papers they have been successfully integrated within the framework induced by the constrained minimisation of the Kullback-Leibler divergence: specifically, two, broad classes of models have been devised, i.e. the integrated and the conditional ones, defined by different, probabilistic rules to place links, load them with weights and turn them into proper, econometric prescriptions. Still, the recipes adopted by the two approaches to estimate the parameters entering into the definition of each model differ. In econometrics, a likelihood that decouples the binary and weighted parts of a model, treating a network as deterministic, is typically maximised; to restore its random character, two alternatives exist: either solving the likelihood maximisation on each configuration of the ensemble and taking the average of the parameters afterwards or taking the average of the likelihood function and maximising the latter one. The difference between these approaches lies in the order in which the operations of averaging and maximisation are taken — a difference that is reminiscent of the quenched and annealed ways of averaging out the disorder in spin glasses. The results of the present contribution, devoted to comparing these recipes in the case of continuous, conditional network models, indicate that the annealed estimation recipe represents the best alternative to the deterministic one. (ar5iv.labs.arxiv.org)
@article{DGS2023, address = {Aachen, Germany}, author = {Di Vece, Marzio and Garlaschelli, Diego and Squartini, Tiziano}, doi = {10.1103/physreve.108.054301}, issn = {2470-0053}, journal = {Physical Review E}, line = {1}, month = nov, number = {5}, open_access = {Gold}, publisher = {American Physical Society (APS)}, title = {Deterministic, quenched, and annealed parameter estimation for heterogeneous network models}, visible_on_website = {YES}, volume = {108}, year = {2023} }
Co-design of Human-centered, Explainable AI for Clinical Decision Support

Cecilia Panigutti, Andrea Beretta, Daniele Fadda, Fosca Giannotti, Dino Pedreschi, and 2 more authors

ACM Transactions on Interactive Intelligent Systems, Dec 2023

RESEARCH LINE
1

3

Abs Bib HTML

eXplainable AI (XAI) involves two intertwined but separate challenges: the development of techniques to extract explanations from black-box AI models and the way such explanations are presented to users, i.e., the explanation user interface. Despite its importance, the second aspect has received limited attention so far in the literature. Effective AI explanation interfaces are fundamental for allowing human decision-makers to take advantage and oversee high-risk AI systems effectively. Following an iterative design approach, we present the first cycle of prototyping-testing-redesigning of an explainable AI technique and its explanation user interface for clinical Decision Support Systems (DSS). We first present an XAI technique that meets the technical requirements of the healthcare domain: sequential, ontology-linked patient data, and multi-label classification tasks. We demonstrate its applicability to explain a clinical DSS, and we design a first prototype of an explanation user interface. Next, we test such a prototype with healthcare providers and collect their feedback with a two-fold outcome: First, we obtain evidence that explanations increase users’ trust in the XAI system, and second, we obtain useful insights on the perceived deficiencies of their interaction with the system, so we can re-design a better, more human-centered explanation interface.
@article{PBF2023, author = {Panigutti, Cecilia and Beretta, Andrea and Fadda, Daniele and Giannotti, Fosca and Pedreschi, Dino and Perotti, Alan and Rinzivillo, Salvatore}, doi = {10.1145/3587271}, issn = {2160-6463}, journal = {ACM Transactions on Interactive Intelligent Systems}, line = {1,3}, month = dec, number = {4}, open_access = {Gold}, pages = {1–35}, publisher = {Association for Computing Machinery (ACM)}, title = {Co-design of Human-centered, Explainable AI for Clinical Decision Support}, visible_on_website = {YES}, volume = {13}, year = {2023} }

2022

Privacy Risk of Global Explainers

Francesca Naretto, Anna Monreale, and Fosca Giannotti

Sep 2022

RESEARCH LINE
5

Abs Bib HTML

In this paper we propose to study a methodology that enables the evaluation of the privacy risk exposure of global explainers based on an interpretable classifier that imitates the global reasoning of a black-box classifier. The idea is to verify if the layer of interpretability added by the interpretable model can jeopardize the privacy protection of the training data used for learning the black-box classifier. In order to address this problem, we exploit a well-known attack model called membership inference attack (MIA). We then compute the privacy risk change ΔR due to the introduction of the global explainer c. The preliminary experimental results suggest that global explainers based on decision trees introduce a higher risk of privacy, increasing the percentage of records identified as members of the training dataset used to train the original black-box classifiers. These results suggest that in order to provide Trustworthy AI, it becomes fundamental to consider the relationship between different ethical values to identify possible values like transparency and privacy that may be in contrast, and studying solutions that enable the simultaneous satisfaction of more than one value.
@inbook{NMG2022, address = {Amsterdam, the Netherlands}, author = {Naretto, Francesca and Monreale, Anna and Giannotti, Fosca}, booktitle = {HHAI2022: Augmenting Human Intellect}, doi = {10.3233/faia220206}, issn = {1879-8314}, line = {5}, month = sep, open_access = {Gold}, pages = {249 - 251}, publisher = {IOS Press}, title = {Privacy Risk of Global Explainers}, visible_on_website = {YES}, year = {2022} }
Understanding the impact of explanations on advice-taking: a user study for AI-based clinical Decision Support Systems

Cecilia Panigutti, Andrea Beretta, Fosca Giannotti, and Dino Pedreschi

In CHI Conference on Human Factors in Computing Systems , Apr 2022

RESEARCH LINE
4

Abs Bib HTML

The field of eXplainable Artificial Intelligence (XAI) focuses on providing explanations for AI systems’ decisions. XAI applications to AI-based Clinical Decision Support Systems (DSS) should increase trust in the DSS by allowing clinicians to investigate the reasons behind its suggestions. In this paper, we present the results of a user study on the impact of advice from a clinical DSS on healthcare providers’ judgment in two different cases: the case where the clinical DSS explains its suggestion and the case it does not. We examined the weight of advice, the behavioral intention to use the system, and the perceptions with quantitative and qualitative measures. Our results indicate a more significant impact of advice when an explanation for the DSS decision is provided. Additionally, through the open-ended questions, we provide some insights on how to improve the explanations in the diagnosis forecasts for healthcare assistants, nurses, and doctors.
@inproceedings{PBP2022, author = {Panigutti, Cecilia and Beretta, Andrea and Giannotti, Fosca and Pedreschi, Dino}, booktitle = {CHI Conference on Human Factors in Computing Systems}, collection = {CHI ’22}, doi = {10.1145/3491102.3502104}, line = {4}, month = apr, open_access = {Gold}, pages = {1–9}, publisher = {ACM}, series = {CHI ’22}, title = {Understanding the impact of explanations on advice-taking: a user study for AI-based clinical Decision Support Systems}, visible_on_website = {YES}, year = {2022} }
Counterfactual explanations and how to find them: literature review and benchmarking

Riccardo Guidotti

Data Mining and Knowledge Discovery, Apr 2022

RESEARCH LINE
1

Abs Bib HTML

Interpretable machine learning aims at unveiling the reasons behind predictions returned by uninterpretable classifiers. One of the most valuable types of explanation consists of counterfactuals. A counterfactual explanation reveals what should have been different in an instance to observe a diverse outcome. For instance, a bank customer asks for a loan that is rejected. The counterfactual explanation consists of what should have been different for the customer in order to have the loan accepted. Recently, there has been an explosion of proposals for counterfactual explainers. The aim of this work is to survey the most recent explainers returning counterfactual explanations. We categorize explainers based on the approach adopted to return the counterfactuals, and we label them according to characteristics of the method and properties of the counterfactuals returned. In addition, we visually compare the explanations, and we report quantitative benchmarking assessing minimality, actionability, stability, diversity, discriminative power, and running time. The results make evident that the current state of the art does not provide a counterfactual explainer able to guarantee all these properties simultaneously.
@article{G2022, address = {Netherlands}, author = {Guidotti, Riccardo}, doi = {10.1007/s10618-022-00831-6}, issn = {1573-756X}, journal = {Data Mining and Knowledge Discovery}, line = {1}, month = apr, number = {5}, open_access = {Gold}, pages = {2770–2824}, publisher = {Springer Science and Business Media LLC}, title = {Counterfactual explanations and how to find them: literature review and benchmarking}, visible_on_website = {YES}, volume = {38}, year = {2022} }
Interpretable Latent Space to Enable Counterfactual Explanations

Francesco Bodria, Riccardo Guidotti, Fosca Giannotti, and Dino Pedreschi

Apr 2022

RESEARCH LINE
1

Abs Bib HTML

Many dimensionality reduction methods have been introduced to map a data space into one with fewer features and enhance machine learning models’ capabilities. This reduced space, called latent space, holds properties that allow researchers to understand the data better and produce better models. This work proposes an interpretable latent space that preserves the similarity of data points and supports a new way of learning a classification model that allows prediction and explanation through counterfactual examples. We demonstrate with extensive experiments the effectiveness of the latent space with respect to different metrics in comparison with several competitors, as well as the quality of the achieved counterfactual explanations.
@inbook{BGG2023c, address = {Montpellier, France}, author = {Bodria, Francesco and Guidotti, Riccardo and Giannotti, Fosca and Pedreschi, Dino}, booktitle = {Discovery Science}, doi = {10.1007/978-3-031-18840-4_37}, isbn = {9783031188404}, issn = {1611-3349}, line = {1}, open_access = {NO}, pages = {525–540}, publisher = {Springer Nature Switzerland}, title = {Interpretable Latent Space to Enable Counterfactual Explanations}, visible_on_website = {YES}, year = {2022} }
Stable and actionable explanations of black-box models through factual and counterfactual rules

Riccardo Guidotti, Anna Monreale, Salvatore Ruggieri, Francesca Naretto, Franco Turini, and 2 more authors

Data Mining and Knowledge Discovery, Nov 2022

RESEARCH LINE
1

2

Abs Bib HTML

Recent years have witnessed the rise of accurate but obscure classification models that hide the logic of their internal decision processes. Explaining the decision taken by a black-box classifier on a specific input instance is therefore of striking interest. We propose a local rule-based model-agnostic explanation method providing stable and actionable explanations. An explanation consists of a factual logic rule, stating the reasons for the black-box decision, and a set of actionable counterfactual logic rules, proactively suggesting the changes in the instance that lead to a different outcome. Explanations are computed from a decision tree that mimics the behavior of the black-box locally to the instance to explain. The decision tree is obtained through a bagging-like approach that favors stability and fidelity: first, an ensemble of decision trees is learned from neighborhoods of the instance under investigation; then, the ensemble is merged into a single decision tree. Neighbor instances are synthetically generated through a genetic algorithm whose fitness function is driven by the black-box behavior. Experiments show that the proposed method advances the state-of-the-art towards a comprehensive approach that successfully covers stability and actionability of factual and counterfactual explanations.
@article{GMR2022, author = {Guidotti, Riccardo and Monreale, Anna and Ruggieri, Salvatore and Naretto, Francesca and Turini, Franco and Pedreschi, Dino and Giannotti, Fosca}, doi = {10.1007/s10618-022-00878-5}, issn = {1573-756X}, journal = {Data Mining and Knowledge Discovery}, line = {1,2}, month = nov, number = {5}, open_access = {Gold}, pages = {2825–2862}, publisher = {Springer Science and Business Media LLC}, title = {Stable and actionable explanations of black-box models through factual and counterfactual rules}, visible_on_website = {YES}, volume = {38}, year = {2022} }
Explainable AI for Time Series Classification: A Review, Taxonomy and Research Directions

Andreas Theissler, Francesco Spinnato, Udo Schlegel, and Riccardo Guidotti

IEEE Access, Nov 2022

RESEARCH LINE
1

Abs Bib HTML

Time series data is increasingly used in a wide range of fields, and it is often relied on in crucial applications and high-stakes decision-making. For instance, sensors generate time series data to recognize different types of anomalies through automatic decision-making systems. Typically, these systems are realized with machine learning models that achieve top-tier performance on time series classification tasks. Unfortunately, the logic behind their prediction is opaque and hard to understand from a human standpoint. Recently, we observed a consistent increase in the development of explanation methods for time series classification justifying the need to structure and review the field. In this work, we (a) present the first extensive literature review on Explainable AI (XAI) for time series classification, (b) categorize the research field through a taxonomy subdividing the methods into time points-based, subsequences-based and instance-based, and (c) identify open research directions regarding the type of explanations and the evaluation of explanations and interpretability.
@article{TSS2022, address = { New York, USA}, author = {Theissler, Andreas and Spinnato, Francesco and Schlegel, Udo and Guidotti, Riccardo}, doi = {10.1109/access.2022.3207765}, issn = {2169-3536}, journal = {IEEE Access}, line = {1}, open_access = {Gold}, pages = {100700–100724}, publisher = {Institute of Electrical and Electronics Engineers (IEEE)}, title = {Explainable AI for Time Series Classification: A Review, Taxonomy and Research Directions}, visible_on_website = {YES}, volume = {10}, year = {2022} }
Transparent Latent Space Counterfactual Explanations for Tabular Data

Francesco Bodria, Riccardo Guidotti, Fosca Giannotti, and Dino Pedreschi

In 2022 IEEE 9th International Conference on Data Science and Advanced Analytics (DSAA) , Oct 2022

RESEARCH LINE
1

Abs Bib HTML

Artificial Intelligence decision-making systems have dramatically increased their predictive performance in recent years, beating humans in many different specific tasks. However, with increased performance has come an increase in the complexity of the black-box models adopted by the AI systems, making them entirely obscure for the decision process adopted. Explainable AI is a field that seeks to make AI decisions more transparent by producing explanations. In this paper, we propose T-LACE, an approach able to retrieve post-hoc counterfactual explanations for a given pre-trained black-box model. T-LACE exploits the similarity and linearity proprieties of a custom-created transparent latent space to build reliable counterfactual explanations. We tested T-LACE on several tabular datasets and provided qualitative evaluations of the generated explanations in terms of similarity, robustness, and diversity. Comparative analysis against various state-of-the-art counterfactual explanation methods shows the higher effectiveness of our approach.
@inproceedings{BGG2023b, address = {Shenzhen, China}, author = {Bodria, Francesco and Guidotti, Riccardo and Giannotti, Fosca and Pedreschi, Dino}, booktitle = {2022 IEEE 9th International Conference on Data Science and Advanced Analytics (DSAA)}, doi = {10.1109/dsaa54385.2022.10032407}, line = {1}, month = oct, open_access = {NO}, pages = {1–10}, publisher = {IEEE}, title = {Transparent Latent Space Counterfactual Explanations for Tabular Data}, visible_on_website = {YES}, year = {2022} }
User-driven counterfactual generator: a human centered exploration

Beretta I; Cappuccio E; Marchiori Manerba M

Dec 2022

RESEARCH LINE
1

3

Abs Bib HTML

In this paper, we critically examine the limitations of the techno-solutionist approach to explanations in the context of counterfactual generation, reaffirming interactivity as a core value in the explanation interface between the model and the user.
@misc{BCM2022, author = {M, Beretta I; Cappuccio E; Marchiori Manerba}, line = {1,3}, month = dec, title = {User-driven counterfactual generator: a human centered exploration}, year = {2022} }

2021

GLocalX - From Local to Global Explanations of Black Box AI Models

Mattia Setzu, Riccardo Guidotti, Anna Monreale, Franco Turini, Dino Pedreschi, and 1 more author

Artificial Intelligence, May 2021

RESEARCH LINE
1

4

Abs Bib HTML

Artificial Intelligence (AI) has come to prominence as one of the major components of our society, with applications in most aspects of our lives. In this field, complex and highly nonlinear machine learning models such as ensemble models, deep neural networks, and Support Vector Machines have consistently shown remarkable accuracy in solving complex tasks. Although accurate, AI models often are “black boxes” which we are not able to understand. Relying on these models has a multifaceted impact and raises significant concerns about their transparency. Applications in sensitive and critical domains are a strong motivational factor in trying to understand the behavior of black boxes. We propose to address this issue by providing an interpretable layer on top of black box models by aggregating “local” explanations. We present GLocalX, a “local-first” model agnostic explanation method. Starting from local explanations expressed in form of local decision rules, GLocalX iteratively generalizes them into global explanations by hierarchically aggregating them. Our goal is to learn accurate yet simple interpretable models to emulate the given black box, and, if possible, replace it entirely. We validate GLocalX in a set of experiments in standard and constrained settings with limited or no access to either data or local explanations. Experiments show that GLocalX is able to accurately emulate several models with simple and small models, reaching state-of-the-art performance against natively global solutions. Our findings show how it is often possible to achieve a high level of both accuracy and comprehensibility of classification models, even in complex domains with high-dimensional data, without necessarily trading one property for the other. This is a key requirement for a trustworthy AI, necessary for adoption in high-stakes decision making applications.
@article{SGM2021, author = {Setzu, Mattia and Guidotti, Riccardo and Monreale, Anna and Turini, Franco and Pedreschi, Dino and Giannotti, Fosca}, doi = {10.1016/j.artint.2021.103457}, issn = {0004-3702}, journal = {Artificial Intelligence}, line = {1,4}, month = may, open_access = {Gold}, pages = {103457}, publisher = {Elsevier BV}, title = {GLocalX - From Local to Global Explanations of Black Box AI Models}, visible_on_website = {YES}, volume = {294}, year = {2021} }
Matrix Profile-Based Interpretable Time Series Classifier

Riccardo Guidotti, and Matteo D’Onofrio

Frontiers in Artificial Intelligence, Oct 2021

RESEARCH LINE
1

Abs Bib HTML

Time series classification (TSC) is a pervasive and transversal problem in various fields ranging from disease diagnosis to anomaly detection in finance. Unfortunately, the most effective models used by Artificial Intelligence (AI) systems for TSC are not interpretable and hide the logic of the decision process, making them unusable in sensitive domains. Recent research is focusing on explanation methods to pair with the obscure classifier to recover this weakness. However, a TSC approach that is transparent by design and is simultaneously efficient and effective is even more preferable. To this aim, we propose an interpretable TSC method based on the patterns, which is possible to extract from the Matrix Profile (MP) of the time series in the training set. A smart design of the classification procedure allows obtaining an efficient and effective transparent classifier modeled as a decision tree that expresses the reasons for the classification as the presence of discriminative subsequences. Quantitative and qualitative experimentation shows that the proposed method overcomes the state-of-the-art interpretable approaches.
@article{GD2021, author = {Guidotti, Riccardo and D’Onofrio, Matteo}, doi = {10.3389/frai.2021.699448}, issn = {2624-8212}, journal = {Frontiers in Artificial Intelligence}, line = {1}, month = oct, open_access = {Gold}, publisher = {Frontiers Media SA}, title = {Matrix Profile-Based Interpretable Time Series Classifier}, visible_on_website = {YES}, volume = {4}, year = {2021} }
Exemplars and Counterexemplars Explanations for Image Classifiers, Targeting Skin Lesion Labeling

Carlo Metta, Riccardo Guidotti, Yuan Yin, Patrick Gallinari, and Salvatore Rinzivillo

In 2021 IEEE Symposium on Computers and Communications (ISCC) , Sep 2021

RESEARCH LINE
1

Abs Bib HTML

Explainable AI consists in developing mechanisms allowing for an interaction between decision systems and humans by making the decisions of the formers understandable. This is particularly important in sensitive contexts like in the medical domain. We propose a use case study, for skin lesion diagnosis, illustrating how it is possible to provide the practitioner with explanations on the decisions of a state of the art deep neural network classifier trained to characterize skin lesions from examples. Our framework consists of a trained classifier onto which an explanation module operates. The latter is able to offer the practitioner exemplars and counterexemplars for the classification diagnosis thus allowing the physician to interact with the automatic diagnosis system. The exemplars are generated via an adversarial autoencoder. We illustrate the behavior of the system on representative examples.
@inproceedings{MGY2021, author = {Metta, Carlo and Guidotti, Riccardo and Yin, Yuan and Gallinari, Patrick and Rinzivillo, Salvatore}, booktitle = {2021 IEEE Symposium on Computers and Communications (ISCC)}, doi = {10.1109/iscc53001.2021.9631485}, line = {1}, month = sep, open_access = {NO}, pages = {1–7}, publisher = {IEEE}, title = {Exemplars and Counterexemplars Explanations for Image Classifiers, Targeting Skin Lesion Labeling}, visible_on_website = {YES}, year = {2021} }
FairLens: Auditing black-box clinical decision support systems

Cecilia Panigutti, Alan Perotti, André Panisson, Paolo Bajardi, and Dino Pedreschi

Information Processing & Management, Sep 2021

RESEARCH LINE
1

4

Abs Bib HTML

Highlights: We present a pipeline to detect and explain potential fairness issues in Clinical DSS. We study and compare different multi-label classification disparity measures. We explore ICD9 bias in MIMIC-IV, an openly available ICU benchmark dataset
@article{PPB2021, author = {Panigutti, Cecilia and Perotti, Alan and Panisson, André and Bajardi, Paolo and Pedreschi, Dino}, doi = {10.1016/j.ipm.2021.102657}, issn = {0306-4573}, journal = {Information Processing & Management}, line = {1,4}, month = sep, number = {5}, open_access = {Gold}, pages = {102657}, publisher = {Elsevier BV}, title = {FairLens: Auditing black-box clinical decision support systems}, visible_on_website = {YES}, volume = {58}, year = {2021} }

2020

Predicting and Explaining Privacy Risk Exposure in Mobility Data

Francesca Naretto, Roberto Pellungrini, Anna Monreale, Franco Maria Nardini, and Mirco Musolesi

Sep 2020

RESEARCH LINE
4

5

Abs Bib HTML

Mobility data is a proxy of different social dynamics and its analysis enables a wide range of user services. Unfortunately, mobility data are very sensitive because the sharing of people’s whereabouts may arise serious privacy concerns. Existing frameworks for privacy risk assessment provide tools to identify and measure privacy risks, but they often (i) have high computational complexity; and (ii) are not able to provide users with a justification of the reported risks. In this paper, we propose expert, a new framework for the prediction and explanation of privacy risk on mobility data. We empirically evaluate privacy risk on real data, simulating a privacy attack with a state-of-the-art privacy risk assessment framework. We then extract individual mobility profiles from the data for predicting their risk. We compare the performance of several machine learning algorithms in order to identify the best approach for our task. Finally, we show how it is possible to explain privacy risk prediction on real data, using two algorithms: Shap, a feature importance-based method and Lore, a rule-based method. Overall, expert is able to provide a user with the privacy risk and an explanation of the risk itself. The experiments show excellent performance for the prediction task.
@inbook{NPM2020, author = {Naretto, Francesca and Pellungrini, Roberto and Monreale, Anna and Nardini, Franco Maria and Musolesi, Mirco}, booktitle = {Discovery Science}, doi = {10.1007/978-3-030-61527-7_27}, isbn = {9783030615277}, issn = {1611-3349}, line = {4,5}, open_access = {NO}, pages = {403–418}, publisher = {Springer International Publishing}, title = {Predicting and Explaining Privacy Risk Exposure in Mobility Data}, visible_on_website = {YES}, year = {2020} }
Doctor XAI: an ontology-based approach to black-box sequential data classification explanations

Cecilia Panigutti, Alan Perotti, and Dino Pedreschi

In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency , Jan 2020

RESEARCH LINE
1

3

4

Abs Bib HTML

Several recent advancements in Machine Learning involve blackbox models: algorithms that do not provide human-understandable explanations in support of their decisions. This limitation hampers the fairness, accountability and transparency of these models; the field of eXplainable Artificial Intelligence (XAI) tries to solve this problem providing human-understandable explanations for black-box models. However, healthcare datasets (and the related learning tasks) often present peculiar features, such as sequential data, multi-label predictions, and links to structured background knowledge. In this paper, we introduce Doctor XAI, a model-agnostic explainability technique able to deal with multi-labeled, sequential, ontology-linked data. We focus on explaining Doctor AI, a multilabel classifier which takes as input the clinical history of a patient in order to predict the next visit. Furthermore, we show how exploiting the temporal dimension in the data and the domain knowledge encoded in the medical ontology improves the quality of the mined explanations.
@inproceedings{PPP2020, author = {Panigutti, Cecilia and Perotti, Alan and Pedreschi, Dino}, booktitle = {Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency}, collection = {FAT* ’20}, doi = {10.1145/3351095.3372855}, line = {1,3,4}, month = jan, open_access = {NO}, pages = {629–639}, publisher = {ACM}, series = {FAT* ’20}, title = {Doctor XAI: an ontology-based approach to black-box sequential data classification explanations}, visible_on_website = {YES}, year = {2020} }
Data-Agnostic Local Neighborhood Generation

Riccardo Guidotti, and Anna Monreale

In 2020 IEEE International Conference on Data Mining (ICDM) , Nov 2020

RESEARCH LINE
1

Abs Bib HTML

Synthetic data generation has been widely adopted in software testing, data privacy, imbalanced learning, machine learning explanation, etc. In such contexts, it is important to generate data samples located within “local” areas surrounding specific instances. Local synthetic data can help the learning phase of predictive models, and it is fundamental for methods explaining the local behavior of obscure classifiers. The contribution of this paper is twofold. First, we introduce a method based on generative operators allowing the synthetic neighborhood generation by applying specific perturbations on a given input instance. The key factor consists in performing a data transformation that makes applicable to any type of data, i.e., data-agnostic. Second, we design a framework for evaluating the goodness of local synthetic neighborhoods exploiting both supervised and unsupervised methodologies. A deep experimentation shows the effectiveness of the proposed method.
@inproceedings{GM2020, author = {Guidotti, Riccardo and Monreale, Anna}, booktitle = {2020 IEEE International Conference on Data Mining (ICDM)}, doi = {10.1109/icdm50108.2020.00122}, issn = {2374-8486}, line = {1}, month = nov, open_access = {NO}, pages = {1040–1045}, publisher = {IEEE}, title = {Data-Agnostic Local Neighborhood Generation}, visible_on_website = {YES}, year = {2020} }

2019

Factual and Counterfactual Explanations for Black Box Decision Making

Riccardo Guidotti, Anna Monreale, Fosca Giannotti, Dino Pedreschi, Salvatore Ruggieri, and 1 more author

IEEE Intelligent Systems, Nov 2019

RESEARCH LINE
1

4

Abs Bib HTML

The rise of sophisticated machine learning models has brought accurate but obscure decision systems, which hide their logic, thus undermining transparency, trust, and the adoption of artificial intelligence (AI) in socially sensitive and safety-critical contexts. We introduce a local rule-based explanation method, providing faithful explanations of the decision made by a black box classifier on a specific instance. The proposed method first learns an interpretable, local classifier on a synthetic neighborhood of the instance under investigation, generated by a genetic algorithm. Then, it derives from the interpretable classifier an explanation consisting of a decision rule, explaining the factual reasons of the decision, and a set of counterfactuals, suggesting the changes in the instance features that would lead to a different outcome. Experimental results show that the proposed method outperforms existing approaches in terms of the quality of the explanations and of the accuracy in mimicking the black box.
@article{GMG2019, author = {Guidotti, Riccardo and Monreale, Anna and Giannotti, Fosca and Pedreschi, Dino and Ruggieri, Salvatore and Turini, Franco}, doi = {10.1109/mis.2019.2957223}, issn = {1941-1294}, journal = {IEEE Intelligent Systems}, line = {1,4}, month = nov, number = {6}, open_access = {Gold}, pages = {14–23}, publisher = {Institute of Electrical and Electronics Engineers (IEEE)}, title = {Factual and Counterfactual Explanations for Black Box Decision Making}, visible_on_website = {YES}, volume = {34}, year = {2019} }