Deep Active Inference with Generative Actions and Diversity-Based Action Choice

EasyChair Preprint 14808, version 2

Abstract

The literature of Deep Active Inference, implementing the generative, biologically inspired Active Inference framework with the Deep Learning approach, often makes use of a hidden state transition model to generate current hidden states. It also usually leverages the Monte Carlo family methods to choose the agent's next action that minimizes the Expected Free Energy. The action identification typically uses either a stochastic sampling process or a learning of sampled actions by a 'habit' model. In this work, the goal is to explore an approach based on the learning and generation of actions as a result of hidden state transitions. The corresponding generative model, along with the variational form of the Free Energy and the Expected Free Energy, are formulated for an environment represented as a Partially Observable Markov Decision Process, and the model architecture is also presented. We also suggest a novel approach for the action choice: the generated action minimizing the Expected Free Energy is chosen based on the diversity of the expected risk relatively to that of its originating action set. The Active Inference agent is also equipped with top-down, selective, context-dependent attention mechanisms to control its behavior. Experiments have been conducted by addressing the continuous versions of Mountain Car and Inverted Pendulum problems. The results show the ability of the agent to learn and solve both problems with promising performance, requiring noticeable changes only on high-level attention parameters. This work highlights that this approach of action generation, choice and planning by Active Inference agents might represent a worthy alternative to usual methods, noticeably for considerations of computational efficiency and bio-mimetism.

Keyphrases: Action Policy, Active Inference, Deep Active Inference, deep learning, generative model, planning, top-down attention

@booklet{EasyChair:14808,
  author    = {Yacine Benabderrahmane and Godefroy Clair and Jérémy Dufourmantelle and Claire Ky},
  title     = {Deep Active Inference with Generative Actions and Diversity-Based Action Choice},
  howpublished = {EasyChair Preprint 14808},
  year      = {EasyChair, 2024}}