PhD thesis : Explaining “black box” AI algorithms through their training examples

about the role

Global context

Recent advances in machine learning have led to new AI applications promising increased automation of new tasks to enhance operational efficiency or relieve professionals from less interesting tasks. These AIs often rely on over-parameterized models (known as deep learning), whose parameters are learned from vast training datasets. As a result, they are often described as 'opaque': it is impossible for users or data scientists to precisely understand the process that leads to decisions once the system is in production. Meanwhile, regulators (AI Act), business users, and end users are demanding more transparency, creating a need for explainability of these algorithms.
In recent years, many explainability methods have emerged for various data modalities, each with its advantages and disadvantages. Among them, example-based explainability methods identify training examples that influence an AI's decisions [1]. This provides a meaningful explanation for non-experts, similar to how people reason: 'I think it is XXX because I have seen a similar case in the past.' These methods have, for instance, been recently applied to state-of-the-art language models [2].

Scientific goal

The thesis work will consist of a literature review of the theoretical, algorithmic, and practical advancements of these methods in order to derive new research avenues, as well as to propose new methods in partnership with supervisors (at Orange Innovation and academic institutions). We have identified a number of challenges to address, as well as references that tackle these issues:

• Scaling methods that can be resource-intensive (memory, algorithmic complexity), based on recent approximate algorithms and proposing new methods [3].
• The formalization of how to appropriately assign importance to each training example when the prediction made by the model relies on multiple training examples [4].
• The identification of groups of examples that lead to undesirable behaviors, such as unfair predictions towards certain groups of instances, mislabeled examples, or anomalies [5].
• The issue of usability regarding the provision of these explanations to real users, rather than just data science experts. Here, there will be an opportunity to engage with expert ergonomists at Innovation on the reception of technical objects in work conditions [6]."

During the thesis, the candidate will also be expected to publish their new scientific contributions and present them at conferences, as well as at internal seminars and the host university.