Our research is centered on Explainable AI (XAI), wherein we aim to establish mathematical foundations for elucidating model predictions.
The overarching goal is to discern the limitations and possibilities associated with explanations generated by AI models.
By focusing on the development of robust methods, our work contributes to the broader objective of enhancing transparency, accountability, and trustworthiness in AI systems.
The specific applications of this research span diverse fields such as medicine, healthcare, law, and ethics.
Example from Our Research:
One of the challenges in machine learning, particularly in medical applications, is accurately interpreting feature importance scores, especially when complex dependencies exist in the data.
A key situation that can lead to unexpected outcomes is the presence of a collider variable, also known as an inverted fork.
For instance, consider the example depicted in the attached Figure. Smoking (which is unobserved) increases the likelihood of both Cancer and Gum chewing. Additionally, doctors recommend chewing gum to patients with Earache. When building predictive models for Cancer using features like Gum and Earache, but without knowledge of Smoking, misleading conclusions may arise.
- Scenario 1: A model using only Earache as a feature would assign no importance to Earache since it is independent of Cancer.
- Scenario 2: A model using only Gum would show Gum as an important predictor of Cancer due to their correlation, even though this association arises from the unobserved variable (Smoking).
- Scenario 3: In a model using both Earache and Gum, feature importance scores might misleadingly indicate that Earache is predictive of Cancer. This is because conditioning on Gum (the collider) creates a spurious association between Earache and Cancer.
These kinds of situations, known in causality literature, are not easily resolved without additional information about the underlying causal mechanisms.
In medical contexts, such erroneous interpretations could lead to incorrect conclusions, such as associating Earache with cancer risk or wrongly attributing gum chewing as a preventive factor for cancer.
Our research explores both the theoretical underpinnings of these phenomena and provides empirical results to help mitigate such risks when applying machine learning models in healthcare