主动推理

Overview

In biophysics and cognitive science, the free energy principle offers a formal account of the representational capacities of physical systems: that is, why things that exist look as if they track properties of the systems to which they are coupled. It establishes that the dynamics of physical systems minimize surprisal (the negative log probability of some outcome); or equivalently, free energy (its variational upper bound). The principle is used in Bayesian accounts of brain function and some approaches to artificial intelligence; it is formally related to variational Bayesian methods. Karl Friston introduced it as a description of embodied perception-action loops in neuroscience.

The free energy principle models the behavior of one system that is distinct from, but coupled to, another system (e.g., an embedding environment), where the degrees of freedom that implement the interface between the two is cast as a Markov blanket.

Background

The notion that self-organizing biological systems — like a cell or brain — can be understood as minimizing variational free energy is based upon Helmholtz's work on unconscious inference and subsequent treatments in psychology and machine learning. Variational free energy is a functional of observations and a probability density over their hidden causes. This variational density is defined in relation to a probabilistic model that generates predicted observations from hypothesized causes. In this setting, free energy provides an approximation to Bayesian model evidence. Therefore, its minimization can be seen as a Bayesian inference process. When a system actively makes observations to minimize free energy, it implicitly performs active inference and maximizes the evidence for its model of the world.

However, free energy is also an upper bound on the self-information of outcomes, where the long-term average of surprise is entropy.

Action and perception

Active inference applies the techniques of approximate Bayesian inference to infer the causes of sensory data under a generative model of how those data are caused and then uses these inferences to guide action. Bayes' rule characterizes the probabilistically optimal inversion of such a causal model, but applying it is typically computationally intractable, leading to the use of approximate methods. In active inference, the leading class of such approximate methods are variational methods, for both practical and theoretical reasons: practical, as they often lead to simple inference procedures; and theoretical, because they are related to underlying physical principles.

These variational methods proceed by minimizing an upper bound on the divergence between the Bayes-optimal inference (or posterior) and its approximation. This upper bound is the free energy.

Free energy minimization

Self-organization

Free energy minimization has been proposed as a hallmark of self-organizing systems when cast as random dynamical systems. This formulation rests on a Markov blanket (comprising action and sensory states) that separates internal and external states. If internal states and action minimize free energy, then they place an upper bound on the entropy of sensory states: （公式略）

This is because the long-term average of surprise is entropy. This bound resists a natural tendency to disorder — of the sort associated with the second law of thermodynamics (in closed systems with detailed balance) and fluctuation theorems (in open systems with non-equilibrium steady state solutions).

Bayesian inference

All Bayesian inference can be cast in terms of free energy minimization. When free energy is minimized with respect to internal states, the Kullback–Leibler divergence between the variational and posterior density over hidden states is minimized.

Neuroscience

Free energy minimization provides a useful way to formulate normative (Bayes optimal) models of inference and learning under uncertainty and therefore subscribes to the Bayesian brain hypothesis. The neuronal processes described by free energy minimization depend on the nature of hidden states: Ψ =X× Θ × Π that can comprise time-dependent variables, time-invariant parameters and the precision (i.e., inverse variance or temperature) of random fluctuations. Minimizing variables, parameters, and precision correspond to inference, learning, and the encoding of uncertainty (i.e., attention), respectively.

Perceptual inference and categorization

Free energy minimization formalizes the notion of unconscious inference in perception and provides a normative (Bayesian) theory of neuronal processing. The associated process theory of neuronal dynamics is based on minimizing free energy through gradient descent. This corresponds to generalized Bayesian filtering (where ~ denotes a variable in generalized coordinates of motion and D is a derivative matrix operator): （公式略）

Active inference

When gradient descent is applied to action a=-∂ ₐF(s,μ), motor control can be understood in terms of classical reflex arcs that are engaged by descending (corticospinal) predictions. This provides a formalism that generalizes the equilibrium point solution — to the degrees of freedom problem — to movement trajectories.

Active inference under discrete models usually considers decision-making under uncertainty in terms of selecting actions from policies that minimize expected free energy. While variational free energy measures the fit between an agent's generative model and current sensory data, expected free energy (EFE) evaluates future outcomes under different possible action policies. In this setting, future outcomes become random variables and expected free energy as an expectation under the posterior predictive density q_(μ)(ψ | a), conditioned on action: （公式略）

Dual Formulations of EFE

Expected free energy can be decomposed in two equivalent ways, revealing how it balances the imperatives for motivated behavior:

1. Risk and Ambiguity