Ethics and Legal

Explainability is one of the most important ethical and legal values identified by EU regulations (GDPR, AI act). However, when we design a trustworthy AI System it is important to take into consideration also the other ethical and legal values such as privacy, fairness, safety, robustness, etc. This requires analyzing the interplay among the different values and understanding whether they are in contrast or not.

In this direction, the work of Line 5 has started the investigation of the interplay between explainability and privacy, and explainability and fairness under different viewpoints. In particular, the research of this line tries to answer the following questions:

Can explanation methods, introducing a level of transparency in the whole decision process, might jeopardize individual privacy of people represented in the training data? Can explanation methods help in understanding the reason for possible ethical risks associated with the use of AI systems (e.g., privacy violations and biased behaviors)? Can explanation methods be fundamental for discovering other ethical issues like unfair behavior of AI systems? Starting from these questions, we have scientifically contributed with the following studies. Concerning the ability of explanation methods to explain some ethical risks such as privacy and biases, in [NPM2020,NPN2020] we propose EXPERT, an EXplainable Privacy ExposuRe predicTion framework. It is a tool that exploits Explainability as a tool for increasing privacy user awareness. We applied EXPERT on the privacy risk prediction and explanation of both tabular data [NPM2020] and sequential data [NPN2020]. In the first setting we considered the mobility context where for each user we have the historical movements (a spatio-temporal trajectory). In this context, EXPERT using the privacy risk exposure module extracts from human mobility data an individual mobility profile describing the mobility behavior of any user. Second, for each user it simulates a privacy attack and quantifies the associated privacy risk. Third, it uses the mobility profiles of the users with their associated privacy risks to train a ML model. For a new user, along with the prediction of risk, EXPERT also provides an explanation of the predicted risk generated by the risk explanation module. EXPERT exploits two state-of-the-art explanation techniques, i.e., SHAP and LORE [GMG2019]. In the second setting [NPN2020] the prediction phase is not based on a mobility profile but on the trajectory itself and given the absence of features we used only SHAP as an explanation method. In [PPB2021] we also present FairLens, a framework able to detect and explain potential bias issues in Clinical Decision Support Systems (DSS). FairLens allows testing the clinical DSS before its deployment, i.e., before handling it to final decision-makers such as physicians and nurses. FairLens takes bias analysis a step further by explaining the reasons behind the poor model performance on specific groups; in particular, it uses GlocalX to explain which elements in the patients’ clinical histories are influencing the misclassification.

Concerning the use of explainability as a means for discovering unfair behaviors, in [MG2021], we propose FairShades, a model-agnostic approach for auditing the outcomes of abusive language detection systems. FairShades combines explainability and fairness evaluation within a proactive pipeline to identify unintended biases and sensitive categories toward which the black box model under assessment is most discriminative. It is a task-specific approach for abusive language detection: it can be used to test the fairness of any abusive language detection system working on any textual dataset. However, its ideal application is on sentences containing protected identities, i.e., expressions referring to nationality, gender, etc., as the primary scope is to uncover biases and not explain the reasons for the prediction.

Research line people

Riccardo
Guidotti

Assitant Professor
University of Pisa

R.LINE 1 ▪ 3 ▪ 4 ▪ 5

Franco
Turini

Full Professor
University of Pisa

R.LINE 1 ▪ 2 ▪ 5

Salvo
Rinzivillo

Researcher
ISTI - CNR Pisa

R.LINE 1 ▪ 3 ▪ 4 ▪ 5

Andrea
Beretta

Researcher
ISTI - CNR Pisa

R.LINE 1 ▪ 4 ▪ 5

Anna
Monreale

Associate Professor
University of Pisa

R.LINE 1 ▪ 4 ▪ 5

Cecilia
Panigutti

Phd Student
Scuola Normale

R.LINE 1 ▪ 4 ▪ 5

Roberto
Pellungrini

Researcher
University of Pisa

R.LINE 5

Francesca
Naretto

Post Doctoral Researcher
Scuola Normale

R.LINE 1 ▪ 3 ▪ 4 ▪ 5

Marta
Marchiori Manerba

Phd Student
University of Pisa

R.LINE 1 ▪ 2 ▪ 5

Clara
Punzi

Phd Student
Scuola Normale

R.LINE 1 ▪ 5

Andrea
Pugnana

Researcher
Scuola Normale

R.LINE 2

António Maria
Lage De Sousa Leitão

Phd Student
Scuola Normale

R.LINE 1

Line 5 - Publications

2025

Embracing Diversity: A Multi-Perspective Approach with Soft Labels

Benedetta Muscato, Praveen Bushipaka, Gizem Gezici, Lucia Passaro, Fosca Giannotti, and 1 more author

Sep 2025

RESEARCH LINE
4

5

Abs Bib HTML

In subjective tasks like stance detection, diverse human perspectives are often simplified into a single ground truth through label aggregation i.e. majority voting, potentially marginalizing minority viewpoints. This paper presents a Multi-Perspective framework for stance detection that explicitly incorporates annotation diversity by using soft labels derived from both human and large language model (LLM) annotations. Building on a stance detection dataset focused on controversial topics, we augment it with document summaries and new LLM-generated labels. We then compare two approaches: a baseline using aggregated hard labels, and a multi-perspective model trained on disaggregated soft labels that capture annotation distributions. Our findings show that multi-perspective models consistently outperform traditional baselines (higher F1-scores), with lower model confidence, reflecting task subjectivity. This work highlights the importance of modeling disagreement and promotes a shift toward more inclusive, perspective-aware NLP systems.
@inbook{MBG2025, author = {Muscato, Benedetta and Bushipaka, Praveen and Gezici, Gizem and Passaro, Lucia and Giannotti, Fosca and Cucinotta, Tommaso}, booktitle = {HHAI 2025}, doi = {10.3233/faia250654}, isbn = {9781643686110}, issn = {1879-8314}, line = {4,5}, month = sep, open_access = {Gold}, pages = {370--384}, publisher = {IOS Press}, title = {Embracing Diversity: A Multi-Perspective Approach with Soft Labels}, visible_on_website = {YES}, year = {2025} }
SafeGen: safeguarding privacy and fairness through a genetic method

Martina Cinquini, Marta Marchiori Manerba, Federico Mazzoni, Francesca Pratesi, and Riccardo Guidotti

Machine Learning, Sep 2025

RESEARCH LINE
5

Abs Bib HTML

To ensure that Machine Learning systems produce unharmful outcomes, pursuing a joint optimization of performance and ethical profiles such as privacy and fairness is crucial. However, jointly optimizing these two ethical dimensions while maintaining predictive accuracy remains a fundamental challenge. Indeed, privacy-preserving techniques may worsen fairness and restrain the model’s ability to learn accurate statistical patterns, while data mitigation techniques may inadvertently compromise privacy. Aiming to bridge this gap, we propose safeGen, a preprocessing fairness enhancing and privacy-preserving method for tabular data. SafeGen employs synthetic data generation through a genetic algorithm to ensure that sensitive attributes are protected while maintaining the necessary statistical properties. We assess our method across multiple datasets, comparing it against state-of-the-art privacy-preserving and fairness approaches through a threefold evaluation: privacy preservation, fairness enhancement, and generated data plausibility. Through extensive experiments, we demonstrate that SafeGen consistently achieves strong anonymization while preserving or improving dataset fairness across several benchmarks. Additionally, through hybrid privacy-fairness constraints and the use of a genetic synthesizer, SafeGen ensures the plausibility of synthetic records while minimizing discrimination. Our findings demonstrate that modeling fairness and privacy within a unified generative method yields significantly better outcomes than addressing these constraints separately, reinforcing the importance of integrated approaches when multiple ethical objectives must be simultaneously satisfied.
@article{CMM2025, author = {Cinquini, Martina and Marchiori Manerba, Marta and Mazzoni, Federico and Pratesi, Francesca and Guidotti, Riccardo}, doi = {10.1007/s10994-025-06835-9}, issn = {1573-0565}, journal = {Machine Learning}, line = {5}, month = sep, number = {10}, open_access = {Gold}, publisher = {Springer Science and Business Media LLC}, title = {SafeGen: safeguarding privacy and fairness through a genetic method}, visible_on_website = {YES}, volume = {114}, year = {2025} }
A Bias Injection Technique to Assess the Resilience of Causal Discovery Methods

Martina Cinquini, Karima Makhlouf, Sami Zhioua, Catuscia Palamidessi, and Riccardo Guidotti

IEEE Access, Sep 2025

RESEARCH LINE
2

5

Abs Bib HTML

Causal discovery (CD) algorithms are increasingly applied to socially and ethically sensitive domains. However, their evaluation under realistic conditions remains challenging due to the scarcity of real-world datasets annotated with ground-truth causal structures. Whereas synthetic data generators support controlled benchmarking, they often overlook forms of bias, such as dependencies involving sensitive attributes, which may significantly affect the observed distribution and compromise the trustworthiness of downstream analysis. This paper introduces a novel synthetic data generation framework that enables controlled bias injection while preserving the causal relationships specified in a ground-truth causal graph. The framework aims to evaluate the reliability of CD methods by examining the impact of varying bias levels and outcome binarization thresholds. Experimental results show that even moderate bias levels can lead CD approaches to fail to correctly infer causal links, particularly those connecting sensitive attributes to decision outcomes. These findings underscore the need for expert validation and highlight the limitations of current CD methods in fairness-critical applications. Our proposal thus provides an essential tool for benchmarking and improving CD algorithms in biased, real-world data settings.
@article{CMZ2025, author = {Cinquini, Martina and Makhlouf, Karima and Zhioua, Sami and Palamidessi, Catuscia and Guidotti, Riccardo}, doi = {10.1109/access.2025.3573201}, issn = {2169-3536}, journal = {IEEE Access}, line = {2,5}, open_access = {Gold}, pages = {97376–97391}, publisher = {Institute of Electrical and Electronics Engineers (IEEE)}, title = {A Bias Injection Technique to Assess the Resilience of Causal Discovery Methods}, visible_on_website = {YES}, volume = {13}, year = {2025} }
Differentially Private FastSHAP for Federated Learning Model Explainability

Valerio Bonsignori, Luca Corbucci, Francesca Naretto, and Anna Monreale

In 2025 International Joint Conference on Neural Networks (IJCNN) , Jun 2025

RESEARCH LINE
1

5

Abs Bib HTML

Explaining the reasoning behind black-box model predictions while preserving user privacy is a significant challenge. This becomes even more complex in Federated Learning, where legal constraints restrict the data that clients can share with external entities. In this paper, we introduce Fastshap++, a method that adapts Fastshap to explain Federated Learning trained models. Unlike existing approaches, Fastshap++ mitigates client privacy risks by incorporating Differential Privacy into the explanation process and preventing the exchange of sensitive information between clients and external entities. We evaluate the effectiveness of Fastshap++ testing it on three different datasets, and comparing the explanations with those produced by a centralized explainer with access to clients’ training data. Lastly, we study the impact of varying levels of Differential Privacy to analyse the trade-offs between privacy and the quality of the explanations.
@inproceedings{BCN2025, address = {Rome, Italy}, author = {Bonsignori, Valerio and Corbucci, Luca and Naretto, Francesca and Monreale, Anna}, booktitle = {2025 International Joint Conference on Neural Networks (IJCNN)}, doi = {10.1109/ijcnn64981.2025.11227553}, line = {1,5}, month = jun, pages = {1–8}, publisher = {IEEE}, title = {Differentially Private FastSHAP for Federated Learning Model Explainability}, visible_on_website = {YES}, year = {2025} }
Balancing Fairness and Interpretability in Clustering with FairParTree

Cristiano Landi, Alessio Cascione, Marta Marchiori Manerba, and Riccardo Guidotti

Oct 2025

RESEARCH LINE
1

5

Abs Bib HTML

The revolution involving Machine Learning has transformed data analytics, making algorithms important in decision-making processes across various domains, even in sensitive scenarios. Indeed, traditional clustering algorithms often lack interpretability and exhibit biases, leading to discriminatory practices and opaque decision-making. To overcome these limitations, we introduce FairParTree, a fair and interpretable clustering algorithm that integrates fairness constraints directly into the clustering process, ensuring that the resulting clusters do not disproportionately disadvantage any particular group. By leveraging the structure of decision trees, FairParTree enhances the interpretability of clustering results by providing clear and understandable motivations for cluster assignments through rule-based explanations. We evaluate FairParTree against state-of-the-art competitors. Through extensive experiments, we show that it maintains strong performances w.r.t. fairness, interpretability, and clustering quality across different dataset sizes, thus positioning itself as a competitive, fair, and interpretable clustering algorithm.
@inbook{LCM2025, author = {Landi, Cristiano and Cascione, Alessio and Manerba, Marta Marchiori and Guidotti, Riccardo}, booktitle = {Explainable Artificial Intelligence}, doi = {10.1007/978-3-032-08324-1_5}, isbn = {9783032083241}, issn = {1865-0937}, line = {1,5}, month = oct, open_access = {Gold}, pages = {104–127}, publisher = {Springer Nature Switzerland}, title = {Balancing Fairness and Interpretability in Clustering with FairParTree}, visible_on_website = {YES}, year = {2025} }
Towards Building a Trustworthy RAG-Based Chatbot for the Italian Public Administration

Chandana Sree Mala, Christian Maio, Mattia Proietti, Gizem Gezici, Fosca Giannotti, and 3 more authors

Sep 2025

RESEARCH LINE
3

5

Abs Bib HTML

Building a Trustworthy Retrieval-Augmented Generation (RAG) chatbot for Italy’s public sector presents challenges that go beyond selecting an appropriate Large Language Model. A major issue is the retrieval phase, where Italian text embedders often underperform compared to English and multilingual counterparts, hindering precise identification and contextualization of critical information. Regulatory constraints further complicate matters by disallowing closed source or cloud based models, forcing reliance on on-premise or fully open source solutions that may not fully address the linguistic complexities of Italian documents. In our study, we evaluate three embedding approaches using a publicly available Italian dataset: a monolingual Italian approach, a translation based method leveraging English only embedders with backward reference mapping, and a multilingual framework applied to both original and translated texts. Our methodology involves chunking documents into coherent segments, embedding them in a high dimensional semantic space, and measuring retrieval accuracy via top-k similarity searches. Our results indicate that the translation based approach significantly improves retrieval performance over Italian specific models, suggesting that bilingual mapping can effectively address both domain specific challenges and regulatory constraints in developing RAG pipelines for public administration.
@inbook{MDP2025, author = {Mala, Chandana Sree and di Maio, Christian and Proietti, Mattia and Gezici, Gizem and Giannotti, Fosca and Melacci, Stefano and Lenci, Alessandro and Gori, Marco}, booktitle = {HHAI 2025}, doi = {10.3233/faia250637}, isbn = {9781643686110}, issn = {1879-8314}, line = {3,5}, month = sep, open_access = {Gold}, pages = {196--204}, publisher = {IOS Press}, title = {Towards Building a Trustworthy RAG-Based Chatbot for the Italian Public Administration}, visible_on_website = {YES}, year = {2025} }
Perspectives in Play: A Multi-Perspective Approach for More Inclusive NLP Systems

Benedetta Muscato, Lucia Passaro, Gizem Gezici, and Fosca Giannotti

In Proceedings of the Thirty-Fourth International Joint Conference on Artificial Intelligence , Sep 2025

RESEARCH LINE
4

5

Abs Bib HTML

In the realm of Natural Language Processing (NLP), common approaches for handling human disagreement consist of aggregating annotators’ viewpoints to establish a single ground truth. However, prior studies show that disregarding individual opinions can lead to the side-effect of under-representing minority perspectives, especially in subjective tasks, where annotators may systematically disagree because of their preferences. Recognizing that labels reflect the diverse backgrounds, life experiences, and values of individuals, this study proposes a new multi-perspective approach using soft labels to encourage the development of the next generation of perspective-aware models—more inclusive and pluralistic. We conduct an extensive analysis across diverse subjective text classification tasks including hate speech, irony, abusive language, and stance detection, to highlight the importance of capturing human disagreements, often overlooked by traditional aggregation methods. Results show that the multi-perspective approach not only better approximates human label distributions, as measured by Jensen-Shannon Divergence (JSD), but also achieves superior classification performance (higher F1-scores), outperforming traditional approaches. However, our approach exhibits lower confidence in tasks like irony and stance detection, likely due to the inherent subjectivity present in the texts. Lastly, leveraging Explainable AI (XAI), we explore model uncertainty and uncover meaningful insights into model predictions. All implementation details are available at our github repo.
@inproceedings{MPG2025, author = {Muscato, Benedetta and Passaro, Lucia and Gezici, Gizem and Giannotti, Fosca}, booktitle = {Proceedings of the Thirty-Fourth International Joint Conference on Artificial Intelligence}, collection = {IJCAI-2025}, doi = {10.24963/ijcai.2025/1092}, line = {4,5}, month = sep, open_access = {Gold}, pages = {9827–9835}, publisher = {International Joint Conferences on Artificial Intelligence Organization}, series = {IJCAI-2025}, title = {Perspectives in Play: A Multi-Perspective Approach for More Inclusive NLP Systems}, visible_on_website = {YES}, year = {2025} }
FeDa4Fair: Client-Level Federated Datasets for Fairness Evaluation

Xenia Heilmann, Luca Corbucci, Mattia Cerrato, and Anna Monreale

Dec 2025

RESEARCH LINE
5

Abs Bib

Federated Learning (FL) enables collaborative model training across multiple clients without sharing clients’ private data. However, fairness remains a key concern, as biases in local clients’ datasets can impact the entire federated system. Heterogeneous data distributions across clients may lead to models that are fairer for some clients than others. Although several fairness-enhancing solutions are present in the literature, most focus on mitigating bias for a single sensitive attribute, typically binary, overlooking the diverse and sometimes conflicting fairness needs of different clients. This limited perspective can limit the effectiveness of fairness interventions for the different clients. To support more robust and reproducible fairness research in FL, we aim to enable a consistent benchmarking of fairness-aware FL methods at both the global and client levels. In this paper, we contribute in three ways: (1) We introduce FeDa4Fair, a library to generate tabular datasets tailored to evaluating fair FL methods under heterogeneous client bias; (2) we release four bias-heterogeneous datasets and corresponding benchmarks to compare fairness mitigation methods in a controlled environment; (3) we provide ready-to-use functions for evaluating fairness outcomes for these datasets.
@misc{HCC2025, author = {Heilmann, Xenia and Corbucci, Luca and Cerrato, Mattia and Monreale, Anna}, line = {5}, month = dec, title = {FeDa4Fair: Client-Level Federated Datasets for Fairness Evaluation}, year = {2025} }
Evaluating the Privacy Exposure of Interpretable Global and Local Explainers.

Francesca Naretto, Anna Monreale, and Fosca Giannotti

Dec 2025

RESEARCH LINE
1

5

Abs Bib HTML

During the last few years, the abundance of data has significantly boosted the performance of Machine Learning models, integrating them into several aspects of daily life. However, the rise of powerful Artificial Intelligence tools has introduced ethical and legal complexities. This paper proposes a computational framework to analyze the ethical and legal dimensions of Machine Learning models, focusing specifically on privacy concerns and interpretability. In fact, recently, the research community proposed privacy attacks able to reveal whether a record was part of the black-box training set or inferring variable values by accessing and querying a Machine Learning model. These attacks highlight privacy vulnerabilities and prove that GDPR regulation might be violated by making data or Machine Learning models accessible. At the same time, the complexity of these models, often labelled as “black-boxes”, has made the development of explanation methods indispensable to enhance trust and facilitate their acceptance and adoption in high-stake scenarios.
@misc{NMG2025, author = {Naretto, Francesca and Monreale, Anna and Giannotti, Fosca}, line = {1,5}, month = dec, publisher = {Trans. Data Priv. 18 (2), 67-93}, title = {Evaluating the Privacy Exposure of Interpretable Global and Local Explainers.}, year = {2025} }

2024

The ALTAI checklist as a tool to assess ethical and legal implications for a trustworthy AI development in education

Andrea Fedele, Clara Punzi, and Stefano Tramacere

Computer Law & Security Review, Jul 2024

RESEARCH LINE
5

Abs Bib HTML

The rapid proliferation of Artificial Intelligence (AI) applications in various domains of our lives has prompted a need for a shift towards a human-centered and trustworthy approach to AI. In this study we employ the Assessment List for Trustworthy Artificial Intelligence (ALTAI) checklist to evaluate the trustworthiness of Artificial Intelligence for Student Performance Prediction (AI4SPP), an AI-powered system designed to detect students at risk of school failure. We strongly support the ethical and legal development of AI and propose an implementation design where the user can choose to have access to each level of a three-tier outcome bundle: the AI prediction alone, the prediction along with its confidence level, and, lastly, local explanations for each grade prediction together with the previous two information. AI4SPP aims to raise awareness among educators and students regarding the factors contributing to low school performance, thereby facilitating the implementation of interventions not only to help students, but also to address biases within the school community. However, we also emphasize the ethical and legal concerns that could arise from a misuse of the AI4SPP tool. First of all, the collection and analysis of data, which is essential for the development of AI models, may lead to breaches of privacy, thus causing particularly adverse consequences in the case of vulnerable individuals. Furthermore, the system’s predictions may be influenced by unacceptable discrimination based on gender, ethnicity, or socio-economic background, leading to unfair actions. The ALTAI checklist serves as a valuable self-assessment tool during the design phase of AI systems, by means of which commonly overlooked weaknesses can be highlighted and addressed. In addition, the same checklist plays a crucial role throughout the AI system life cycle. Continuous monitoring of sensitive features within the dataset, alongside survey assessments to gauge users’ responses to the systems, is essential for gathering insights and intervening accordingly. We argue that adopting a critical approach to AI development is essential for societal progress, believing that it can evolve and accelerate over time without impeding openness to new technologies. By aligning with ethical principles and legal requirements, AI systems can make significant contributions to education while mitigating potential risks and ensuring a fair and inclusive learning environment.
@article{FPT2024, author = {Fedele, Andrea and Punzi, Clara and Tramacere, Stefano}, doi = {10.1016/j.clsr.2024.105986}, issn = {2212-473X}, journal = {Computer Law & Security Review}, line = {5}, month = jul, open_access = {Gold}, pages = {105986}, publisher = {Elsevier BV}, title = {The ALTAI checklist as a tool to assess ethical and legal implications for a trustworthy AI development in education}, visible_on_website = {YES}, volume = {53}, year = {2024} }
FairBelief - Assessing Harmful Beliefs in Language Models

Mattia Setzu, Marta Marchiori Manerba, Pasquale Minervini, and Debora Nozza

In Proceedings of the 4th Workshop on Trustworthy Natural Language Processing (TrustNLP 2024) , Jul 2024

RESEARCH LINE
5

Abs Bib HTML

Language Models (LMs) have been shown to inherit undesired biases that might hurt minorities and underrepresented groups if such systems were integrated into real-world applications without careful fairness auditing.This paper proposes FairBelief, an analytical approach to capture and assess beliefs, i.e., propositions that an LM may embed with different degrees of confidence and that covertly influence its predictions. With FairBelief, we leverage prompting to study the behavior of several state-of-the-art LMs across different previously neglected axes, such as model scale and likelihood, assessing predictions on a fairness dataset specifically designed to quantify LMs’ outputs’ hurtfulness.Finally, we conclude with an in-depth qualitative assessment of the beliefs emitted by the models.We apply FairBelief to English LMs, revealing that, although these architectures enable high performances on diverse natural language processing tasks, they show hurtful beliefs about specific genders. Interestingly, training procedure and dataset, model scale, and architecture induce beliefs of different degrees of hurtfulness.
@inproceedings{SMM2024, author = {Setzu, Mattia and Marchiori Manerba, Marta and Minervini, Pasquale and Nozza, Debora}, booktitle = {Proceedings of the 4th Workshop on Trustworthy Natural Language Processing (TrustNLP 2024)}, doi = {10.18653/v1/2024.trustnlp-1.3}, line = {5}, open_access = {Gold}, pages = {27–39}, publisher = {Association for Computational Linguistics}, title = {FairBelief - Assessing Harmful Beliefs in Language Models}, visible_on_website = {YES}, year = {2024} }
Social Bias Probing: Fairness Benchmarking for Language Models

Marta Marchiori Manerba, Karolina Stanczak, Riccardo Guidotti, and Isabelle Augenstein

In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing , Jul 2024

RESEARCH LINE
5

Abs Bib HTML

While the impact of social biases in language models has been recognized, prior methods for bias evaluation have been limited to binary association tests on small datasets, limiting our understanding of bias complexities. This paper proposes a novel framework for probing language models for social biases by assessing disparate treatment, which involves treating individuals differently according to their affiliation with a sensitive demographic group. We curate SoFa, a large-scale benchmark designed to address the limitations of existing fairness collections. SoFa expands the analysis beyond the binary comparison of stereotypical versus anti-stereotypical identities to include a diverse range of identities and stereotypes. Comparing our methodology with existing benchmarks, we reveal that biases within language models are more nuanced than acknowledged, indicating a broader scope of encoded biases than previously recognized. Benchmarking LMs on SoFa, we expose how identities expressing different religions lead to the most pronounced disparate treatments across all models. Finally, our findings indicate that real-life adversities faced by various groups such as women and people with disabilities are mirrored in the behavior of these models. (arXiv)
@inproceedings{MSG2024, author = {Marchiori Manerba, Marta and Stanczak, Karolina and Guidotti, Riccardo and Augenstein, Isabelle}, booktitle = {Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing}, doi = {10.18653/v1/2024.emnlp-main.812}, line = {5}, open_access = {Gold}, pages = {14653–14671}, publisher = {Association for Computational Linguistics}, title = {Social Bias Probing: Fairness Benchmarking for Language Models}, visible_on_website = {YES}, year = {2024} }
Mapping the landscape of ethical considerations in explainable AI research

Luca Nannini, Marta Marchiori Manerba, and Isacco Beretta

Ethics and Information Technology, Jun 2024

RESEARCH LINE
1

5

Abs Bib HTML

With its potential to contribute to the ethical governance of AI, explainable AI (XAI) research frequently asserts its relevance to ethical considerations. Yet the substantiation of these claims with rigorous ethical analysis and reflection remains largely unexamined. This contribution endeavours to scrutinise the relationship between XAI and ethical considerations. We conduct a systematic review of the XAI literature by applying a multi-stage filtering process and then developing a taxonomy for classifying the depth and quality of ethical engagement in the field. Our findings show that although a growing body of XAI research references ethical issues, the majority does so at a superficial level. We identify nine ethical themes (e.g., autonomy, justice, transparency) and show how they are engaged in the literature. We discuss trends, deficits, and future directions for integrating ethical considerations meaningfully into XAI research.
@article{NMB2024, author = {Nannini, Luca and Marchiori Manerba, Marta and Beretta, Isacco}, doi = {10.1007/s10676-024-09773-7}, issn = {1572-8439}, journal = {Ethics and Information Technology}, line = {1,5}, month = jun, number = {3}, open_access = {Gold}, publisher = {Springer Science and Business Media LLC}, title = {Mapping the landscape of ethical considerations in explainable AI research}, visible_on_website = {YES}, volume = {26}, year = {2024} }
Analysis of exposome and genetic variability suggests stress as a major contributor for development of pancreatic ductal adenocarcinoma

Giulia Peduzzi, Alessio Felici, Roberto Pellungrini, Francesca Giorgolo, Riccardo Farinella, and 7 more authors

Digestive and Liver Disease, Jun 2024

RESEARCH LINE
5

Abs Bib HTML

Background: Current knowledge on pancreatic ductal adenocarcinoma (PDAC) risk factors is limited and no study has comprehensively tested the exposome alongside genetic variability for disease susceptibility. We analyzed 347 exposure variables and a polygenic risk score in UK Biobank data (816 PDAC cases, 302,645 controls). Fifty-two associations passed Bonferroni correction. Known risk factors such as smoking, pancreatitis, diabetes, heavy alcohol use and high BMI were confirmed. Novel associations include mobile phone usage intensity and multiple stress-related lifestyle factors. PRS was associated with PDAC risk but no gene‚Äìenvironment interactions were detected. Conclusion: Stressful lifestyle and sedentary behavior may play a major role in PDAC susceptibility independently of genetics.
@article{PFP2023, author = {Peduzzi, Giulia and Felici, Alessio and Pellungrini, Roberto and Giorgolo, Francesca and Farinella, Riccardo and Gentiluomo, Manuel and Spinelli, Andrea and Capurso, Gabriele and Monreale, Anna and Canzian, Federico and Calderisi, Marco and Campa, Daniele}, doi = {10.1016/j.dld.2023.10.015}, issn = {1590-8658}, journal = {Digestive and Liver Disease}, line = {5}, month = jun, number = {6}, open_access = {Gold}, pages = {1054–1063}, publisher = {Elsevier BV}, title = {Analysis of exposome and genetic variability suggests stress as a major contributor for development of pancreatic ductal adenocarcinoma}, visible_on_website = {YES}, volume = {56}, year = {2024} }
Multi-Perspective Stance Detection

Benedetta Muscato, Praveen Bushipaka, Gizem Gezici, Lucia Passaro, and Fosca Giannotti

Dec 2024

RESEARCH LINE
4

5

Abs Bib HTML

Subjective NLP tasks usually rely on human annotations provided by multiple annotators, whose judgments may vary due to their diverse backgrounds and life experiences. Traditional methods often aggregate multiple annotations into a single ground truth, disregarding the diversity in perspectives that arises from annotator disagreement. In this preliminary study, we examine the effect of including multiple annotations on model accuracy in classification. Our methodology investigates the performance of perspectiveaware classification models in stance detection task and further inspects if annotator disagreement affects the model confidence. The results show that multi-perspective approach yields better classification performance outperforming the baseline which uses the single label. This entails that designing more inclusive perspective-aware AI models is not only an essential first step in implementing responsible and ethical AI, but it can also achieve superior results than using the traditional approaches.
@misc{MBG2024bb, address = {Aachen, Germany}, author = {Muscato, Benedetta and Bushipaka, Praveen and Gezici, Gizem and Passaro, Lucia and Giannotti, Fosca}, line = {4,5}, month = dec, title = {Multi-Perspective Stance Detection}, year = {2024} }
Beyond Headlines: A Corpus of Femicides News Coverage in Italian Newspapers

Eleonora Cappuccio, Benedetta Muscato, Laura Pollacci, Marta Marchiori Manerba, Clara Punzi, and 5 more authors

Dec 2024

RESEARCH LINE
4

5

Abs Bib HTML

How newspapers cover news significantly impacts how facts are understood, perceived, and processed by the public. This is especially crucial when serious crimes are reported, e.g., in the case of femicides, where the description of the perpetrator and the victim builds a strong, often polarized opinion of this severe societal issue. This paper presents FMNews, a new dataset of articles reporting femicides extracted from Italian newspapers. Our core contribution aims to promote the development of a deeper framing and awareness of the phenomenon through an original resource available and accessible to the research community, facilitating further analyses on the topic. The paper also provides a preliminary study of the resulting collection through several example use cases and scenarios.
@misc{CMP2024, address = {Aachen, Germany}, author = {Cappuccio, Eleonora and Muscato, Benedetta and Pollacci, Laura and Manerba, Marta Marchiori and Punzi, Clara and Mala, Chandana Sree and Lalli, Margherita and Gezici, Gizem and Natilli, Michela and Giannotti, Fosca}, line = {4,5}, month = dec, title = {Beyond Headlines: A Corpus of Femicides News Coverage in Italian Newspapers}, year = {2024} }
A survey on the impact of AI-based recommenders on human behaviours: methodologies, outcomes and future directions

Luca Pappalardo, Emanuele Ferragina, Salvatore Citraro, Giuliano Cornacchia, Mirco Nanni, and 9 more authors

Dec 2024

RESEARCH LINE
3

4

5

Abs Bib HTML

Recommendation systems and assistants (in short, recommenders) are ubiquitous in online platforms and influence most actions of our day-to-day lives, suggesting items or providing solutions based on users’ preferences or requests. This survey analyses the impact of recommenders in four human-AI ecosystems: social media, online retail, urban mapping and generative AI ecosystems. Its scope is to systematise a fast-growing field in which terminologies employed to classify methodologies and outcomes are fragmented and unsystematic. We follow the customary steps of qualitative systematic review, gathering 144 articles from different disciplines to develop a parsimonious taxonomy of: methodologies employed (empirical, simulation, observational, controlled), outcomes observed (concentration, model collapse, diversity, echo chamber, filter bubble, inequality, polarisation, radicalisation, volume), and their level of analysis (individual, item, model, and systemic). We systematically discuss all findings of our survey substantively and methodologically, highlighting also potential avenues for future research. This survey is addressed to scholars and practitioners interested in different human-AI ecosystems, policymakers and institutional stakeholders who want to understand better the measurable outcomes of recommenders, and tech companies who wish to obtain a systematic view of the impact of their recommenders.
@misc{PFC2024, author = {Pappalardo, Luca and Ferragina, Emanuele and Citraro, Salvatore and Cornacchia, Giuliano and Nanni, Mirco and Rossetti, Giulio and Gezici, Gizem and Giannotti, Fosca and Lalli, Margherita and Gambetta, Daniele and Mauro, Giovanni and Morini, Virginia and Pansanella, Valentina and Pedreschi, Dino}, doi = {10.48550/arXiv.2407.01630}, line = {3,4,5}, month = dec, publisher = {arXiv}, title = {A survey on the impact of AI-based recommenders on human behaviours: methodologies, outcomes and future directions}, year = {2024} }
The ethical impact assessment of selling life insurance to titanic passengers

Gezici, Gizem; Mannari, Chiara; Orlandi, and Lorenzo

Dec 2024

RESEARCH LINE
5

Abs Bib

The Artificial Intelligence Act (AIA) is a uniform legal framework to ensure that AI systems within the European Union (EU) are safe and comply with existing law on fundamental rights and constitutional values. The AIA adopts a risk-based approach with the aim of intending to regulate AI systems, especially categorised as high-risk, which have significant harmful impacts on the health, safety and fundamental rights of persons in the Union. The AIA is founded on the Ethics Guidelines of the High-Level Expert Group for Trustworthy AI, which are grounded in fundamental rights and reflect four ethical imperatives in order to ensure ethical and robust AI. While we acknowledge that ethics is not law, we advocate that the analysis of ethical risks can assist us in complying with laws, thereby facilitating the implementation of the AIA requirements. Thus, we first design an AI-driven Decision Support System for individual risk prediction in the insurance domain (categorised as high-risk by the AIA) based on the Titanic case, which is a popular benchmark dataset in machine learning. We then fulfill an ethical impact assessment of the Titanic case study, relying on the four ethical imperatives of respect for human autonomy, prevention of harm, fairness, and explicability, declared by the High-Level Expert Group for Trustworthy AI. In the context of this ethical impact assessment, we also refer to the questions in the ALTAI checklist. Our discussions regarding the ethical impact assessment in the insurance domain demonstrate that ethical principles can intersect but also create tensions (intriguingly, only in this particular context), for which there is no definitive solution. When tensions arise, which may result in unavoidable trade-offs, these trade-offs should be addressed in a rational and methodical manner, paying special attention to the context of the current case study being evaluated.
@misc{GMO2024, author = {Gezici and Mannari, Gizem; and Orlandi, Chiara; and Lorenzo}, line = {5}, month = dec, title = {The ethical impact assessment of selling life insurance to titanic passengers}, year = {2024} }
XAI in healthcare

Gezici G.; Metta C; Beretta A.; Pellungrini R.; Rinzivillo S.; Pedreschi D.; Giannotti F.

Dec 2024

RESEARCH LINE
4

5

Abs Bib HTML

The evolution of Explainable Artificial Intelligence (XAI) within healthcare represents a crucial turn towards more transparent, understandable, and patient-centric AI applications. The main objective is not only to increase the accuracy of AI models but also, and more importantly, to establish user trust in decision support systems through improving their interpretability. This extended abstract outlines the ongoing efforts and advancements of our lab addressing the challenges brought up by complex AI systems in healthcare domain. Currently, there are four main projects: Prostate Imaging Cancer AI, Liver Transplantation & Diabetes, Breast Cancer, and Doctor XAI, and ABELE.
@misc{GMB2024, author = {F., Gezici G.; Metta C; Beretta A.; Pellungrini R.; Rinzivillo S.; Pedreschi D.; Giannotti}, line = {4,5}, month = dec, title = {XAI in healthcare}, year = {2024} }
Interpretable and Fair Mechanisms for Abstaining Classifiers

Daphne Lenders, Andrea Pugnana, Roberto Pellungrini, Toon Calders, Dino Pedreschi, and 1 more author

Dec 2024

RESEARCH LINE
1

5

Abs Bib HTML

Abstaining classifiers have the option to refrain from providing a prediction for instances that are difficult to classify. The abstention mechanism is designed to trade off the classifier‚Äôs performance on the accepted data while ensuring a minimum number of predictions. In this setting, fairness concerns often arise when the abstention mechanism reduces errors only for majority groups, increasing disparities across demographic groups. We introduce Interpretable and Fair Abstaining Classifier (IFAC), an algorithm that can reject predictions based on uncertainty and unfairness. By rejecting potentially unfair predictions, our method reduces disparities across groups of the non-rejected data. The unfairness-based rejections rely on interpretable rule-based fairness checks and situation testing, enabling transparent review and decision-making.
@misc{LPP2024, author = {Lenders, Daphne and Pugnana, Andrea and Pellungrini, Roberto and Calders, Toon and Pedreschi, Dino and Giannotti, Fosca}, doi = {[75,46,72,75,50,73,78]. }, line = {1,5}, month = dec, title = {Interpretable and Fair Mechanisms for Abstaining Classifiers}, year = {2024} }

2023

Effects of Route Randomization on Urban Emissions

Giuliano Cornacchia, Mirco Nanni, Dino Pedreschi, and Luca Pappalardo

SUMO Conference Proceedings, Jun 2023

RESEARCH LINE
4

5

Abs Bib HTML

Routing algorithms typically suggest the fastest path or slight variation to reach a user’s desired destination. Although this suggestion at the individual level is undoubtedly advantageous for the user, from a collective point of view, the aggregation of all single suggested paths may result in an increasing impact (e.g., in terms of emissions). In this study, we use SUMO to simulate the effects of incorporating randomness into routing algorithms on emissions, their distribution, and travel time in the urban area of Milan (Italy). Our results reveal that, given the common practice of routing towards the fastest path, a certain level of randomness in routes reduces emissions and travel time. In other words, the stronger the random component in the routes, the more pronounced the benefits upon a certain threshold. Our research provides insight into the potential advantages of considering collective outcomes in routing decisions and highlights the need to explore further the relationship between route randomization and sustainability in urban transportation.
@article{CNP2023, author = {Cornacchia, Giuliano and Nanni, Mirco and Pedreschi, Dino and Pappalardo, Luca}, doi = {10.52825/scp.v4i.217}, issn = {2750-4425}, journal = {SUMO Conference Proceedings}, line = {4,5}, month = jun, open_access = {Gold}, pages = {75–87}, publisher = {TIB Open Publishing}, title = {Effects of Route Randomization on Urban Emissions}, visible_on_website = {YES}, volume = {4}, year = {2023} }
EXPHLOT: EXplainable Privacy Assessment for Human LOcation Trajectories

Francesca Naretto, Roberto Pellungrini, Salvatore Rinzivillo, and Daniele Fadda

Jun 2023

RESEARCH LINE
1

3

5

Abs Bib HTML

Human mobility data play a crucial role in understanding mobility patterns and developing analytical services across domains such as urban planning, transportation, and public health. However, due to the sensitive nature of this data, identifying privacy risks is essential before deciding to release it publicly. Recent work has proposed using machine learning models for predicting privacy risk on raw mobility trajectories and using SHAP for risk explanation. However, applying SHAP to mobility data results in explanations of limited use both for privacy experts and end-users. In this work, we present EXPHLOT, a novel version of the Expert privacy risk prediction and explanation framework specifically tailored for human mobility data. We leverage state-of-the-art algorithms in time series classification to improve risk prediction while reducing computation time. We also devise an entropy-based mask to efficiently compute SHAP values and develop a module for interactive analysis and visualization of SHAP values over a map, empowering users with an intuitive understanding of privacy risk.
@inbook{NPR2023, author = {Naretto, Francesca and Pellungrini, Roberto and Rinzivillo, Salvatore and Fadda, Daniele}, booktitle = {Discovery Science}, doi = {10.1007/978-3-031-45275-8_22}, isbn = {9783031452758}, issn = {1611-3349}, line = {1,3,5}, open_access = {Gold}, pages = {325–340}, publisher = {Springer Nature Switzerland}, title = {EXPHLOT: EXplainable Privacy Assessment for Human LOcation Trajectories}, visible_on_website = {YES}, year = {2023} }
Exposing Racial Dialect Bias in Abusive Language Detection: Can Explainability Play a Role?

Marta Marchiori Manerba, and Virginia Morini

Jun 2023

RESEARCH LINE
5

Abs Bib HTML

Biases can arise and be introduced during each phase of a supervised learning pipeline, eventually leading to harm. Within the task of automatic abusive language detection, this matter becomes particularly severe since unintended bias towards sensitive topics such as gender, sexual orientation, or ethnicity can harm underrepresented groups. The role of the datasets used to train these models is crucial to address these challenges. In this contribution, we investigate whether explainability methods can expose racial dialect bias attested within a popular dataset for abusive language detection. Through preliminary experiments, we found that pure explainability techniques cannot effectively uncover biases within the dataset under analysis: the rooted stereotypes are often more implicit and complex to retrieve.
@inbook{MM2023, address = {Cham, Switzerland}, author = {Manerba, Marta Marchiori and Morini, Virginia}, booktitle = {Machine Learning and Principles and Practice of Knowledge Discovery in Databases}, doi = {10.1007/978-3-031-23618-1_32}, isbn = {9783031236181}, issn = {1865-0937}, line = {5}, open_access = {NO}, pages = {483–497}, publisher = {Springer Nature Switzerland}, title = {Exposing Racial Dialect Bias in Abusive Language Detection: Can Explainability Play a Role?}, visible_on_website = {YES}, year = {2023} }

2022

Privacy Risk of Global Explainers

Francesca Naretto, Anna Monreale, and Fosca Giannotti

Sep 2022

RESEARCH LINE
5

Abs Bib HTML

In this paper we propose to study a methodology that enables the evaluation of the privacy risk exposure of global explainers based on an interpretable classifier that imitates the global reasoning of a black-box classifier. The idea is to verify if the layer of interpretability added by the interpretable model can jeopardize the privacy protection of the training data used for learning the black-box classifier. In order to address this problem, we exploit a well-known attack model called membership inference attack (MIA). We then compute the privacy risk change ΔR due to the introduction of the global explainer c. The preliminary experimental results suggest that global explainers based on decision trees introduce a higher risk of privacy, increasing the percentage of records identified as members of the training dataset used to train the original black-box classifiers. These results suggest that in order to provide Trustworthy AI, it becomes fundamental to consider the relationship between different ethical values to identify possible values like transparency and privacy that may be in contrast, and studying solutions that enable the simultaneous satisfaction of more than one value.
@inbook{NMG2022, address = {Amsterdam, the Netherlands}, author = {Naretto, Francesca and Monreale, Anna and Giannotti, Fosca}, booktitle = {HHAI2022: Augmenting Human Intellect}, doi = {10.3233/faia220206}, issn = {1879-8314}, line = {5}, month = sep, open_access = {Gold}, pages = {249 - 251}, publisher = {IOS Press}, title = {Privacy Risk of Global Explainers}, visible_on_website = {YES}, year = {2022} }

Assessing Trustworthy AI in Times of COVID-19: Deep Learning for Predicting a Multiregional Score Conveying the Degree of Lung Compromise in COVID-19 Patients

Himanshi Allahabadi, Julia Amann, Isabelle Balot, Andrea Beretta, Charles Binkley, and 52 more authors

IEEE Transactions on Technology and Society, Dec 2022

RESEARCH LINE

Abs Bib HTML

This article’s main contributions are twofold: 1) to demonstrate how to apply the general European Union’s High-Level Expert Group’s (EU HLEG) guidelines for trustworthy AI in practice for the domain of healthcare and 2) to investigate the research question of what does “trustworthy AI” mean at the time of the COVID-19 pandemic. To this end, we present the results of a post-hoc self-assessment to evaluate the trustworthiness of an AI system for predicting a multiregional score conveying the degree of lung compromise in COVID-19 patients, developed and verified by an interdisciplinary team with members from academia, public hospitals, and industry in time of pandemic. The AI system aims to help radiologists to estimate and communicate the severity of damage in a patient’s lung from Chest X-rays. It has been experimentally deployed in the radiology department of the ASST Spedali Civili clinic in Brescia, Italy, since December 2020 during pandemic time. The methodology we have applied for our post-hoc assessment, called Z-Inspection®, uses sociotechnical scenarios to identify ethical, technical, and domain-specific issues in the use of the AI system in the context of the pandemic.

@article{AAB2022,
  author = {Allahabadi, Himanshi and Amann, Julia and Balot, Isabelle and Beretta, Andrea and Binkley, Charles and Bozenhard, Jonas and Bruneault, Frederick and Brusseau, James and Candemir, Sema and Cappellini, Luca Alessandro and Chakraborty, Subrata and Cherciu, Nicoleta and Cociancig, Christina and Coffee, Megan and Ek, Irene and Espinosa-Leal, Leonardo and Farina, Davide and Fieux-Castagnet, Genevieve and Frauenfelder, Thomas and Gallucci, Alessio and Giuliani, Guya and Golda, Adam and van Halem, Irmhild and Hildt, Elisabeth and Holm, Sune and Kararigas, Georgios and Krier, Sebastien A. and Kuhne, Ulrich and Lizzi, Francesca and Madai, Vince I. and Markus, Aniek F. and Masis, Serg and Mathez, Emilie Wiinblad and Mureddu, Francesco and Neri, Emanuele and Osika, Walter and Ozols, Matiss and Panigutti, Cecilia and Parent, Brendan and Pratesi, Francesca and Moreno-Sanchez, Pedro A. and Sartor, Giovanni and Savardi, Mattia and Signoroni, Alberto and Sormunen, Hanna-Maria and Spezzatti, Andy and Srivastava, Adarsh and Stephansen, Annette F. and Theng, Lau Bee and Tithi, Jesmin Jahan and Tuominen, Jarno and Umbrello, Steven and Vaccher, Filippo and Vetter, Dennis and Westerlund, Magnus and Wurth, Renee and Zicari, Roberto V.},
  doi = {10.1109/tts.2022.3195114},
  issn = {2637-6415},
  journal = {IEEE Transactions on Technology and Society},
  line = {4,5},
  month = dec,
  number = {4},
  open_access = {Gold},
  pages = {272–289},
  publisher = {Institute of Electrical and Electronics Engineers (IEEE)},
  title = {Assessing Trustworthy AI in Times of COVID-19: Deep Learning for Predicting a Multiregional Score Conveying the Degree of Lung Compromise in COVID-19 Patients},
  visible_on_website = {YES},
  volume = {3},
  year = {2022}
}

Evaluating the Privacy Exposure of Interpretable Global Explainers

Francesca Naretto, Anna Monreale, and Fosca Giannotti

In 2022 IEEE 4th International Conference on Cognitive Machine Intelligence (CogMI) , Dec 2022

RESEARCH LINE
5

Abs Bib HTML

In recent years we are witnessing the diffusion of AI systems based on powerful Machine Learning models which find application in many critical contexts such as medicine, financial market and credit scoring. In such a context it is particularly important to design Trustworthy AI systems while guaranteeing transparency, with respect to their decision reasoning and privacy protection. Although many works in the literature addressed the lack of transparency and the risk of privacy exposure of Machine Learning models, the privacy risks of explainers have not been appropriately studied. This paper presents a methodology for evaluating the privacy exposure raised by interpretable global explainers able to imitate the original black-box classifier. Our methodology exploits the well-known Membership Inference Attack. The experimental results highlight that global explainers based on interpretable trees lead to an increase in privacy exposure.
@inproceedings{NMG2022b, author = {Naretto, Francesca and Monreale, Anna and Giannotti, Fosca}, booktitle = {2022 IEEE 4th International Conference on Cognitive Machine Intelligence (CogMI)}, doi = {10.1109/cogmi56440.2022.00012}, line = {5}, month = dec, open_access = {NO}, pages = {13–19}, publisher = {IEEE}, title = {Evaluating the Privacy Exposure of Interpretable Global Explainers}, visible_on_website = {YES}, year = {2022} }
Investigating Debiasing Effects on Classification and Explainability

Marta Marchiori Manerba, and Riccardo Guidotti

In Proceedings of the 2022 AAAI/ACM Conference on AI, Ethics, and Society , Jul 2022

RESEARCH LINE
1

5

Abs Bib HTML

During each stage of a dataset creation and development process, harmful biases can be accidentally introduced, leading to models that perpetuates marginalization and discrimination of minorities, as the role of the data used during the training is critical. We propose an evaluation framework that investigates the impact on classification and explainability of bias mitigation preprocessing techniques used to assess data imbalances concerning minorities’ representativeness and mitigate the skewed distributions discovered. Our evaluation focuses on assessing fairness, explainability and performance metrics. We analyze the behavior of local model-agnostic explainers on the original and mitigated datasets to examine whether the proxy models learned by the explainability techniques to mimic the black-boxes disproportionately rely on sensitive attributes, demonstrating biases rooted in the explainers. We conduct several experiments about known biased datasets to demonstrate our proposal’s novelty and effectiveness for evaluation and bias detection purposes.
@inproceedings{MG2022, author = {Marchiori Manerba, Marta and Guidotti, Riccardo}, booktitle = {Proceedings of the 2022 AAAI/ACM Conference on AI, Ethics, and Society}, collection = {AIES ’22}, doi = {10.1145/3514094.3534170}, line = {1,5}, month = jul, open_access = {NO}, pages = {468–478}, publisher = {ACM}, series = {AIES ’22}, title = {Investigating Debiasing Effects on Classification and Explainability}, visible_on_website = {YES}, year = {2022} }

2021

Trustworthy AI

Raja Chatila, Virginia Dignum, Michael Fisher, Fosca Giannotti, Katharina Morik, and 2 more authors

Jul 2021

RESEARCH LINE
5

Abs Bib HTML

Modern AI systems have become of widespread use in almost all sectors with a strong impact on our society. However, the very methods on which they rely, based on Machine Learning techniques for processing data to predict outcomes and to make decisions, are opaque, prone to bias and may produce wrong answers. Objective functions optimized in learning systems are not guaranteed to align with the values that motivated their definition. Properties such as transparency, verifiability, explainability, security, technical robustness and safety, are key to build operational governance frameworks, so that to make AI systems justifiably trustworthy and to align their development and use with human rights and values.
@inbook{CDF2021, author = {Chatila, Raja and Dignum, Virginia and Fisher, Michael and Giannotti, Fosca and Morik, Katharina and Russell, Stuart and Yeung, Karen}, booktitle = {Reflections on Artificial Intelligence for Humanity}, doi = {10.1007/978-3-030-69128-8_2}, isbn = {9783030691288}, issn = {1611-3349}, line = {5}, open_access = {NO}, pages = {13–39}, publisher = {Springer International Publishing}, title = {Trustworthy AI}, visible_on_website = {YES}, year = {2021} }
FairShades: Fairness Auditing via Explainability in Abusive Language Detection Systems

Marta Marchiori Manerba, and Riccardo Guidotti

In 2021 IEEE Third International Conference on Cognitive Machine Intelligence (CogMI) , Dec 2021

RESEARCH LINE
1

5

Abs Bib HTML

At every stage of a supervised learning process, harmful biases can arise and be inadvertently introduced, ultimately leading to marginalization, discrimination, and abuse towards minorities. This phenomenon becomes particularly impactful in the sensitive real-world context of abusive language detection systems, where non-discrimination is difficult to assess. In addition, given the opaqueness of their internal behavior, the dynamics leading a model to a certain decision are often not clear nor accountable, and significant problems of trust could emerge. A robust value-oriented evaluation of models’ fairness is therefore necessary. In this paper, we present FairShades, a model-agnostic approach for auditing the outcomes of abusive language detection systems. Combining explainability and fairness evaluation, FairShades can identify unintended biases and sensitive categories towards which models are most discriminative. This objective is pursued through the auditing of meaningful counterfactuals generated within CheckList framework. We conduct several experiments on BERT-based models to demonstrate our proposal’s novelty and effectiveness for unmasking biases.
@inproceedings{MG2021, author = {Manerba, Marta Marchiori and Guidotti, Riccardo}, booktitle = {2021 IEEE Third International Conference on Cognitive Machine Intelligence (CogMI)}, doi = {10.1109/cogmi52975.2021.00014}, line = {1,5}, month = dec, open_access = {NO}, pages = {34–43}, publisher = {IEEE}, title = {FairShades: Fairness Auditing via Explainability in Abusive Language Detection Systems}, visible_on_website = {YES}, year = {2021} }
Explainable AI Within the Digital Transformation and Cyber Physical Systems: XAI Methods and Applications

Dec 2021

RESEARCH LINE
1

5

Abs Bib HTML

This book presents Explainable Artificial Intelligence (XAI), which aims at producing explainable models that enable human users to understand and appropriately trust the obtained results. The authors discuss the challenges involved in making machine learning-based AI explainable. Firstly, that the explanations must be adapted to different stakeholders (end-users, policy makers, industries, utilities etc.) with different levels of technical knowledge (managers, engineers, technicians, etc.) in different application domains. Secondly, that it is important to develop an evaluation framework and standards in order to measure the effectiveness of the provided explanations at the human and the technical levels. This book gathers research contributions aiming at the development and/or the use of XAI techniques in order to address the aforementioned challenges in different applications such as healthcare, finance, cybersecurity, and document summarization. It allows highlighting the benefits and requirements of using explainable models in different application domains in order to provide guidance to readers to select the most adapted models to their specified problem and conditions. Includes recent developments of the use of Explainable Artificial Intelligence (XAI) in order to address the challenges of digital transition and cyber-physical systems; Provides a textual scientific description of the use of XAI in order to address the challenges of digital transition and cyber-physical systems; Presents examples and case studies in order to increase transparency and understanding of the methodological concepts.
@book{GMP2021, author = {}, doi = {10.1007/978-3-030-76409-8}, isbn = {9783030764098}, line = {1,5}, open_access = {NO}, pages = {9-31}, publisher = {Springer International Publishing}, title = {Explainable AI Within the Digital Transformation and Cyber Physical Systems: XAI Methods and Applications}, visible_on_website = {YES}, year = {2021} }

2020

Prediction and Explanation of Privacy Risk on Mobility Data with Neural Networks

Francesca Naretto, Roberto Pellungrini, Franco Maria Nardini, and Fosca Giannotti

Dec 2020

RESEARCH LINE
4

5

Abs Bib HTML

The analysis of privacy risk for mobility data is a fundamental part of any privacy-aware process based on such data. Mobility data are highly sensitive. Therefore, the correct identification of the privacy risk before releasing the data to the public is of utmost importance. However, existing privacy risk assessment frameworks have high computational complexity. To tackle these issues, some recent work proposed a solution based on classification approaches to predict privacy risk using mobility features extracted from the data. In this paper, we propose an improvement of this approach by applying long short-term memory (LSTM) neural networks to predict the privacy risk directly from original mobility data. We empirically evaluate privacy risk on real data by applying our LSTM-based approach. Results show that our proposed method based on a LSTM network is effective in predicting the privacy risk with results in terms of F1 of up to 0.91. Moreover, to explain the predictions of our model, we employ a state-of-the-art explanation algorithm, Shap. We explore the resulting explanation, showing how it is possible to provide effective predictions while explaining them to the end-user.
@inbook{NPN2020, author = {Naretto, Francesca and Pellungrini, Roberto and Nardini, Franco Maria and Giannotti, Fosca}, booktitle = {ECML PKDD 2020 Workshops}, doi = {10.1007/978-3-030-65965-3_34}, isbn = {9783030659653}, issn = {1865-0937}, line = {4,5}, open_access = {NO}, pages = {501–516}, publisher = {Springer International Publishing}, title = {Prediction and Explanation of Privacy Risk on Mobility Data with Neural Networks}, visible_on_website = {YES}, year = {2020} }
Predicting and Explaining Privacy Risk Exposure in Mobility Data

Francesca Naretto, Roberto Pellungrini, Anna Monreale, Franco Maria Nardini, and Mirco Musolesi

Dec 2020

RESEARCH LINE
4

5

Abs Bib HTML

Mobility data is a proxy of different social dynamics and its analysis enables a wide range of user services. Unfortunately, mobility data are very sensitive because the sharing of people’s whereabouts may arise serious privacy concerns. Existing frameworks for privacy risk assessment provide tools to identify and measure privacy risks, but they often (i) have high computational complexity; and (ii) are not able to provide users with a justification of the reported risks. In this paper, we propose expert, a new framework for the prediction and explanation of privacy risk on mobility data. We empirically evaluate privacy risk on real data, simulating a privacy attack with a state-of-the-art privacy risk assessment framework. We then extract individual mobility profiles from the data for predicting their risk. We compare the performance of several machine learning algorithms in order to identify the best approach for our task. Finally, we show how it is possible to explain privacy risk prediction on real data, using two algorithms: Shap, a feature importance-based method and Lore, a rule-based method. Overall, expert is able to provide a user with the privacy risk and an explanation of the risk itself. The experiments show excellent performance for the prediction task.
@inbook{NPM2020, author = {Naretto, Francesca and Pellungrini, Roberto and Monreale, Anna and Nardini, Franco Maria and Musolesi, Mirco}, booktitle = {Discovery Science}, doi = {10.1007/978-3-030-61527-7_27}, isbn = {9783030615277}, issn = {1611-3349}, line = {4,5}, open_access = {NO}, pages = {403–418}, publisher = {Springer International Publishing}, title = {Predicting and Explaining Privacy Risk Exposure in Mobility Data}, visible_on_website = {YES}, year = {2020} }
Rischi etico-legali dell’Intelligenza Artificiale

Monreale Anna

Dec 2020

RESEARCH LINE
5

Abs Bib HTML

Nowadays, a wide variety of personal data, describing directly or indirectly individuals’ activities, are collected and available, due to the spread of different apps, sensors and mobile devices. The availability of such data opens unprecedented opportunities of developing AI systems exploiting those data to provide a wide range of benefits in different domains. Unfortunately, in a critical domain like healthcare and justice the development and the application of AI systems can rise many ethical and legal concerns. This article discusses the ethical and legal implication in terms of privacy and transparency, also providing an overview on the different approaches to ethics across the world.
@misc{M2020, author = {Anna, Monreale}, issn = {2037-6677}, line = {5}, month = dec, title = {Rischi etico-legali dell’Intelligenza Artificiale}, year = {2020} }