In 2021 I launched a special session at the International Conference on Brain Informatics with a long, slightly provocative title: Explainable Artificial Intelligence for Unveiling the Brain: From the Black-Box to the Glass-Box. We simply called it XAIB. Five editions later, I want to look back at who joined us, what we discussed, and how my own thinking about explainability changed along the way.
2021: why the black box is a problem
The first edition took place on 18 September 2021. The starting point was simple. Machine learning was already everywhere in brain research, from neuroimage analysis to the automatic diagnosis of neurodegenerative diseases. Yet clinicians had good reasons not to trust predictions they could not understand.
I could not have hoped for a better opening speaker than Rich Caruana from Microsoft Research. The title of his talk, Friends Don’t Let Friends Deploy Black-Box Models: The Importance of Intelligible Machine Learning for Healthcare, still sums up the spirit of XAIB. Michele Ferrante from the US National Institute of Mental Health discussed funding opportunities and research directions for XAI, computational neuroscience and computational psychiatry. Dimitris Pinotsis, from City, University of London and MIT, showed how machine learning can be combined with biophysical modelling to address problems in computational psychiatry.
From the very first edition, XAIB was about more than tools. It was about how explainability connects to scientific understanding and to the way research is funded and translated. (Watch the 2021 recording.)
2022: explainability meets neurodegeneration
The second edition, on 15 July 2022, focused firmly on clinical applications. Mónica Hernández from the University of Zaragoza used explainable AI to understand why the top three methods of the TADPOLE challenge performed so well in forecasting Alzheimer’s disease diagnosis. Bojan Bogdanovic from Ss. Cyril and Methodius University in Skopje presented an explainable machine learning analysis of Alzheimer’s disease. Antonio Parziale from the University of Salerno discussed handwriting analysis and machine learning for neurodegenerative disorders.
All three talks shared an idea that I find essential. The value of explainability is not just to justify a prediction, but to learn something new about the disease, and sometimes to question hypotheses we take for granted. (Watch the 2022 recording.)
2023: widening the lens
In 2023 XAIB ran as a joint session with Recent Advances in Artificial Intelligence for Brain Research. It was our best-attended edition, and the topics widened accordingly.
Parul Verma from UCSF presented graph-based computational models of whole-brain functional dynamics, including their alterations in Alzheimer’s disease. Malhar Patel from Rhino Health introduced a federated computing platform that keeps clinical data behind hospital firewalls. Sanjay Ghosh, then at UCSF, presented a deep multi-task framework for tinnitus classification and severity prediction from structural MRI.
This edition reminded me that trustworthy AI is not only about explanations. It also depends on mechanistic understanding and on how we access and protect clinical data.
2024: from interpretability to robustness
The fourth edition took place on 15 December 2024. I co-chaired it with Sanjay Ghosh, by then Assistant Professor at IIT Kharagpur, and Chiara Camastra chaired the session.
Christoph Molnar gave the invited talk Supervised ML in Science – From Interpretability to Robustness. It put words to something many of us had been feeling: interpretability only has value if what we interpret is stable and scientifically meaningful. Sanjay asked the matching question for deep learning: how far are we, really, from explainable deep models? Contributed papers covered explainable CNNs for brain disease diagnosis, falsifiable models of visual flicker fusion, and the acceptance of EEG devices. (See the 2024 program.)
2025: explainability meets uncertainty
The fifth edition, on 11 November 2025, was co-organized with Vincenzo Dentamaro from the University of Bari Aldo Moro and moderated again by Chiara Camastra. For the first time, the call explicitly included uncertainty quantification, robustness and calibration alongside explainability.
Valeriy Manokhin opened with a keynote on conformal prediction for trustworthy, explainable AI. Vincenzo Dentamaro presented deterministic explainability with the EVIDENCE and MuPAX theories. Felice Franchini argued for deterministic and reproducible explanations in genomic data. Contributed talks covered augmented reality for tremor assessment, explainable multiscale learning of brain networks, and speech reconstruction from intracranial EEG. (Watch the 2025 recording.)
As I said in my closing remarks, explainability, uncertainty quantification and reproducibility are not parallel tracks. They are complementary paths toward trustworthy AI. This edition closed a circle for me. An explanation tells you why a model made a prediction, and uncertainty tells you how much to trust it. In the clinic you need both. This idea is also at the core of ICeX, the framework I developed for brain age estimation.
Five lessons
Looking back across the five editions, these are the points I keep returning to:
- Explainability is a property of the whole pipeline, not of a plot. Weak validation or unstable models produce explanations that look convincing and are wrong.
- The goal is knowledge, not justification. The most valuable talks used explainability to learn something about the brain or about a disease.
- Interpretable-by-design models deserve more credit. Rich Caruana made this case in our very first session, and it has aged very well.
- Uncertainty is part of the explanation. Without it, explainability tells only half the story.
- Communities are built one edition at a time. Sanjay Ghosh joined as an invited speaker in 2023, co-chaired in 2024 and presented again in 2025. Mónica Hernández spoke in 2022 and has since become a close collaborator. That continuity is what I am proudest of.
Thank you
XAIB would not have existed without our invited speakers, the authors who submitted their work, the co-chairs Sanjay Ghosh and Vincenzo Dentamaro, Chiara Camastra, who chaired the last two editions, the program committee members, and the Brain Informatics community that hosted us every year.
The field has moved from “can we open the black box?” to “can we trust what we see inside, and how much?” To me, that is real progress. Let’s keep unveiling the brain, one transparent model at a time.
Leave a Reply