* External authors




Explainable Inference on Sequential Data via Memory-Tracking

Biagio La Rosa*

Roberto Capobianco

Daniele Nardi*

* External authors




In this paper we present a novel mechanism to get explanations that allow to better understand network predictions when dealing with sequential data. Specifically, we adopt memory-based networks — Differential Neural Computers — to exploit their capability of storing data in memory and reusing it for inference. By tracking both the memory access at prediction time, and the information stored by the network at each step of the input sequence, we can retrieve the most relevant input steps associated to each prediction. We validate our approach (1) on a modified T-maze, which is a non-Markovian discrete control task evaluating an algorithm’s ability to correlate events far apart in history, and (2) on the Story Cloze Test, which is a commonsense reasoning framework for evaluating story understanding that requires a system to choose the correct ending to a four-sentence story. Our results show that we are able to explain agent’s decisions in (1) and to reconstruct the most relevant sentences used by the network to select the story ending in (2). Additionally, we show not only that by removing those sentences the network prediction changes, but also that the same are sufficient to reproduce the inference.

Related Publications

Outracing Champion Gran Turismo Drivers with Deep Reinforcement Learning

Nature, 2022
Pete Wurman, Samuel Barrett, Kenta Kawamoto, James MacGlashan, Kaushik Subramanian, Thomas J. Walsh, Roberto Capobianco, Alisa Devlic, Franziska Eckert, Florian Fuchs, Leilani Gilpin, Piyush Khandelwal, Varun Kompella, Hao Chih Lin, Patrick MacAlpine, Declan Oller, Takuma Seno, Craig Sherstan, Michael D. Thomure, Houmehr Aghabozorgi, Leon Barrett, Rory Douglas, Dion Whitehead Amago, Peter Dürr, Peter Stone, Michael Spranger, Hiroaki Kitano

Many potential applications of artificial intelligence involve making real-time decisions in physical systems while interacting with humans. Automobile racing represents an extreme example of these conditions; drivers must execute complex tactical manoeuvres to pass or block…

Planetary Environment Prediction Using Generative Modeling

AIAA SciTech Forum, 2022
Shrijit Singh*, Shreyansh Daftry*, Roberto Capobianco

Planetary rovers have a limited sensory horizon and operate in environments where limited information about the surrounding terrain is available. The rough and unknown nature of the terrain in planetary environments potentially leads to scenarios where the rover gets stuckan…

Tafl-ES: Exploring Evolution Strategies for Asymmetrical Board Games

AIxIA, 2021
Roberto Gallotta*, Roberto Capobianco

NeuroEvolution Strategies (NES) are a subclass of Evolution Strategies (ES). While their application to games and board games have been studied in the past [11], current state of the art in most of the games is still held by classic RL models, such as AlphaGo Zero [16]. This…

  • HOME
  • Publications
  • Explainable Inference on Sequential Data via Memory-Tracking


Shape the Future of AI with Sony AI

We want to hear from those of you who have a strong desire
to shape the future of AI.