Mean field particle methods

Mean field particle methods are a broad class of interacting type Monte Carlo algorithms for simulating from a sequence of probability distributions satisfying a nonlinear evolution equation^[1][2][3][4] These flows of probability measures can always be interpreted as the distributions of the random states of a Markov process whose transition probabilities depends on the distributions of the current random states.^[1][2] A natural way to simulate these sophisticated nonlinear Markov processes is to sample a large number of copies of the process, replacing in the evolution equation the unknown distributions of the random states by the sampled empirical measures. In contrast with traditional Monte Carlo and Markov chain Monte Carlo methods these mean field particle techniques rely on sequential interacting samples. The terminology mean field reflects the fact that each of the samples (a.k.a. particles, individuals, walkers, agents, creatures, or phenotypes) interacts with the empirical measures of the process. When the size of the system tends to infinity, these random empirical measures converge to the deterministic distribution of the random states of the nonlinear Markov chain, so that the statistical interaction between particles vanishes. In other words, starting with a chaotic configuration based on independent copies of initial state of the nonlinear Markov chain model, the chaos propagates at any time horizon as the size the system tends to infinity; that is, finite blocks of particles reduces to independent copies of the nonlinear Markov process. This result is called the propagation of chaos property.^[5][6][7] The terminology "propagation of chaos" originated with the work of Mark Kac in 1976 on a colliding mean field kinetic gas model^[8]

History

The theory of mean field interacting particle models had certainly started by the mid-1960s, with the work of Henry P. McKean Jr. on Markov interpretations of a class of nonlinear parabolic partial differential equations arising in fluid mechanics.^[5][9] The mathematical foundations of these classes of models were developed from the mid-1980s to the mid-1990s by several mathematicians, including Werner Braun, Klaus Hepp,^[10] Karl Oelschläger,^[11][12][13] Gérard Ben Arous and Marc Brunaud,^[14] Donald Dawson, Jean Vaillancourt^[15] and Jürgen Gärtner,^[16][17] Christian Léonard,^[18] Sylvie Méléard, Sylvie Roelly,^[6] Alain-Sol Sznitman^[7][19] and Hiroshi Tanaka^[20] for diffusion type models; F. Alberto Grünbaum,^[21] Tokuzo Shiga, Hiroshi Tanaka,^[22] Sylvie Méléard and Carl Graham^[23][24][25] for general classes of interacting jump-diffusion processes.

We also quote an earlier pioneering article by Theodore E. Harris and Herman Kahn, published in 1951, using mean field but heuristic-like genetic methods for estimating particle transmission energies.^[26] Mean field genetic type particle methods are also used as heuristic natural search algorithms (a.k.a. metaheuristic) in evolutionary computing. The origins of these mean field computational techniques can be traced to 1950 and 1954 with the work of Alan Turing on genetic type mutation-selection learning machines^[27] and the articles by Nils Aall Barricelli at the Institute for Advanced Study in Princeton, New Jersey.^[28][29] The Australian geneticist Alex Fraser also published in 1957 a series of papers on the genetic type simulation of artificial selection of organisms.^[30]

Quantum Monte Carlo, and more specifically Diffusion Monte Carlo methods can also be interpreted as a mean field particle approximation of Feynman-Kac path integrals.^{[3][4][31][32][33][34][35]} The origins of Quantum Monte Carlo methods are often attributed to Enrico Fermi and Robert Richtmyer who developed in 1948 a mean field particle interpretation of neutron-chain reactions,^[36] but the first heuristic-like and genetic type particle algorithm (a.k.a. Resampled or Reconfiguration Monte Carlo methods) for estimating ground state energies of quantum systems (in reduced matrix models) is due to Jack H. Hetherington in 1984^[35] In molecular chemistry, the use of genetic heuristic-like particle methods (a.k.a. pruning and enrichment strategies) can be traced back to 1955 with the seminal work of Marshall. N. Rosenbluth and Arianna. W. Rosenbluth.^[37]

The first pioneering articles on the applications of these heuristic-like particle methods in nonlinear filtering problems were the independent studies of Neil Gordon, David Salmon and Adrian Smith (bootstrap filter),^[38] Genshiro Kitagawa (Monte Carlo filter) ,^[39] and the one by Himilcon Carvalho, Pierre Del Moral, André Monin and Gérard Salut^[40] published in the 1990s. The term interacting "particle filters" was first coined in 1996 by Del Moral.^[41] Particle filters were also developed in signal processing in the early 1989-1992 by P. Del Moral, J.C. Noyer, G. Rigal, and G. Salut in the LAAS-CNRS in a series of restricted and classified research reports with STCAN (Service Technique des Constructions et Armes Navales), the IT company DIGILOG, and the LAAS-CNRS (the Laboratory for Analysis and Architecture of Systems) on RADAR/SONAR and GPS signal processing problems.^{[42][43][44][45][46][47]}

The foundations and the first rigorous analysis on the convergence of genetic type models and mean field Feynman-Kac particle methods are due to Pierre Del Moral^[48][49] in 1996. Branching type particle methods with varying population sizes were also developed in the end of the 1990s by Dan Crisan, Jessica Gaines and Terry Lyons,^[50][51][52] and by Dan Crisan, Pierre Del Moral and Terry Lyons.^[53] The first uniform convergence results with respect to the time parameter for mean field particle models were developed in the end of the 1990s by Pierre Del Moral and Alice Guionnet^[54][55] for interacting jump type processes, and by Florent Malrieu for nonlinear diffusion type processes.^[56]

New classes of mean field particle simulation techniques for Feynman-Kac path-integration problems includes genealogical tree based models,^[2][3][57] backward particle models,^[2][58] adaptive mean field particle models,^[59] island type particle models,^[60][61] and particle Markov chain Monte Carlo methods^[62][63]

Applications

In physics, and more particularly in statistical mechanics, these nonlinear evolution equations are often used to describe the statistical behavior of microscopic interacting particles in a fluid or in some condensed matter. In this context, the random evolution of a virtual fluid or a gas particle is represented by McKean-Vlasov diffusion processes, reaction–diffusion systems, or Boltzmann type collision processes.^{[11][12][13][25][64]} As its name indicates, the mean field particle model represents the collective behavior of microscopic particles weakly interacting with their occupation measures. The macroscopic behavior of these many-body particle systems is encapsulated in the limiting model obtained when the size of the population tends to infinity. Boltzmann equations represent the macroscopic evolution of colliding particles in rarefied gases, while McKean Vlasov diffusions represent the macroscopic behavior of fluid particles and granular gases.

In computational physics and more specifically in quantum mechanics, the ground state energies of quantum systems is associated with the top of the spectrum of Schrödinger's operators. The Schrödinger equation is the quantum mechanics version of the Newton's second law of motion of classical mechanics (the mass times the acceleration is the sum of the forces). This equation represents the wave function (a.k.a. the quantum state) evolution of some physical system, including molecular, atomic of subatomic systems, as well as macroscopic systems like the universe.^[65] The solution of the imaginary time Schrödinger equation (a.k.a. the heat equation) is given by a Feynman-Kac distribution associated with a free evolution Markov process (often represented by Brownian motions) in the set of electronic or macromolecular configurations and some potential energy function. The long time behavior of these nonlinear semigroups is related to top eigenvalues and ground state energies of Schrödinger's operators.^{[3][32][33][34][35][66]} The genetic type mean field interpretation of these Feynman-Kac models are termed Resample Monte Carlo, or Diffusion Monte Carlo methods. These branching type evolutionary algorithms are based on mutation and selection transitions. During the mutation transition, the walkers evolve randomly and independently in a potential energy landscape on particle configurations. The mean field selection process (a.k.a. quantum teleportation, population reconfiguration, resampled transition) is associated with a fitness function that reflects the particle absorption in an energy well. Configurations with low relative energy are more likely to duplicate. In molecular chemistry, and statistical physics Mean field particle methods are also used to sample Boltzmann-Gibbs measures associated with some cooling schedule, and to compute their normalizing constants (a.k.a. free energies, or partition functions).^{[2][67][68][69]}

In computational biology, and more specifically in population genetics, spatial branching processes with competitive selection and migration mechanisms can also represented by mean field genetic type population dynamics models.^[4][70] The first moments of the occupation measures of a spatial branching process are given by Feynman-Kac distribution flows.^[71][72] The mean field genetic type approximation of these flows offers a fixed population size interpretation of these branching processes.^[2][3][70] Extinction probabilities can be interpreted as absorption probabilities of some Markov process evolving in some absorbing environment. These absorption models are represented by Feynman-Kac models.^{[73][74][75][76]} The long time behavior of these processes conditioned on non-extinction can be expressed in an equivalent way by quasi-invariant measures, Yaglom limits,^[77] or invariant measures of nonlinear normalized Feynman-Kac flows.^{[2][3][54][55][66][78]}

In computer sciences, and more particularly in artificial intelligence these mean field type genetic algorithms are used as random search heuristics that mimic the process of evolution to generate useful solutions to complex optimization problems.^[79][80][81] These stochastic search algorithms belongs to the class of Evolutionary models. The idea is to propagate a population of feasible candidate solutions using mutation and selection mechanisms. The mean field interaction between the individuals is encapsulated in the selection and the cross-over mechanisms.

In mean field games and multi-agent interacting systems theories, mean field particle processes are used to represent the collective behavior of complex systems with interacting individuals.^{[82][83][84][85][86][87][88][89]} In this context, the mean field interaction is encapsulated in the decision process of interacting agents. The limiting model as the number of agents tends to infinity is sometimes called the continuum model of agents^[90]

In information theory, and more specifically in statistical machine learning and signal processing, mean field particle methods are used to sample sequentially from the conditional distributions of some random process with respect to a sequence of observations or a cascade of rare events.^{[2][3][70][91]} In discrete time nonlinear filtering problems, the conditional distributions of the random states of a signal given partial and noisy observations satisfy a nonlinear updating-prediction evolution equation. The updating step is given by Bayes' rule, and the prediction step is a Chapman-Kolmogorov transport equation. The mean field particle interpretation of these nonlinear filtering equations is a genetic type selection-mutation particle algorithm^[48] During the mutation step, the particles evolve independently of one another according to the Markov transitions of the signal . During the selection stage, particles with small relative likelihood values are killed, while the ones with high relative values are multiplied.^[92][93] These mean field particle techniques are also used to solve multiple-object tracking problems, and more specifically to estimate association measures^{[2][70][70][94]}

The continuous time version of these particle models are mean field Moran type particle interpretations of the robust optimal filter evolution equations or the Kushner-Stratonotich stochastic partial differential equation.^[4][31][93] These genetic type mean field particle algorithms also termed Particle Filters and Sequential Monte Carlo methods are extensively and routinely used in operation research and statistical inference .^[95][96][97] The term "particle filters" was first coined in 1996 by Del Moral,^[41] and the term "sequential Monte Carlo" by Liu and Chen in 1998. Subset simulation and Monte Carlo splitting^[98] techniques are particular instances of genetic particle schemes and Feynman-Kac particle models equipped with Markov chain Monte Carlo mutation transitions^{[67][99][100]}

Illustrations of the Mean field simulation method

Countable state space models

To motivate the mean field simulation algorithm we start with S a finite or countable state space and let P(S) denote the set of all probability measures on S. Consider a sequence of probability distributions $(\eta_0, \eta_1, \cdots)$ on S satisfying an evolution equation:

$\eta_{n+1}=\Phi(\eta_n)$

(1)

for some, possibly nonlinear, mapping $\Phi: P(S) \to P(S).$ These distributions are given by vectors

$\eta_n=(\eta_n(x))_{x\in S},$

that satisfy:

$0 \leqslant \eta_n(x) \leqslant 1, \qquad \sum\nolimits_{x\in S}\eta_n(x)=1.$

Therefore, $\Phi$ is a mapping from the $(s-1)$-unit simplex into itself, where s stands for the cardinality of the set S. When s is too large, solving equation (1) is intractable or computationally very costly. One natural way to approximate these evolution equations is to reduce sequentially the state space using a mean field particle model. One of the simplest mean field simulation scheme is defined by the Markov chain

$\xi^{(N)}_n=\left(\xi^{(N,1)}_n, \cdots, \xi^{(N,N)}_n \right)$

on the product space $S^N$, starting with N independent random variables with probability distribution $\eta_0$ and elementary transitions

$\mathbf{P} \left( \left. \xi^{(N,1)}_{n+1}=y^1,\cdots,\xi^{(N,N)}_{n+1}=y^N \right |\xi^{(N)}_n\right)=\prod_{i=1}^N \Phi\left(\eta_n^N\right)\left(y^i\right),$

with the empirical measure

$\eta^N_n=\frac{1}{N}\sum_{j=1}^N1_{\xi^{(N,j)}_n}$

where $1_x$ is the indicator function of the state x.

In other words, given $\xi^{(N)}_n$ the samples $\xi^{(N)}_{n+1}$ are independent random variables with probability distribution $\Phi\left(\eta_n^N\right)$. The rationale behind this mean field simulation technique is the following: We expect that when $\eta_{n}^N$ is a good approximation of $\eta_n$, then $\Phi\left(\eta_n^N\right)$ is an approximation of $\Phi\left(\eta_n\right)=\eta_{n+1}$. Thus, since $\eta_{n+1}^N$ is the empirical measure of N conditionally independent random variables with common probability distribution $\Phi\left(\eta_n^N\right)$, we expect $\eta_{n+1}^N$ to be a good approximation of $\eta_{n+1}$.

Another strategy is to find a collection

$K_{\eta_n}=\left(K_{\eta_n}(x,y)\right)_{x,y\in S}$

of stochastic matrices indexed by $\eta_n\in P(S)$ such that

$\sum_{x\in S}\eta_n(x)K_{\eta_n}(x,y)=\Phi(\eta_n)(y)=\eta_{n+1}(y)$

(2)

This formula allows us to interpret the sequence $(\eta_0, \eta_1, \cdots)$ as the probability distributions of the random states $\left(\overline{X}_0, \overline{X}_1, \cdots \right)$ of the nonlinear Markov chain model with elementary transitions

$\mathbf{P} \left ( \left.\overline{X}_{n+1}=y \right| \overline{X}_n=x \right )=K_{\eta_n}(x,y), \qquad \text{Law}(\overline{X}_n)=\eta_n.$

A collection of Markov transitions $K_{\eta_n}$ satisfying the equation (1) is called a McKean interpretation of the sequence of measures $\eta_n$. The mean field particle interpretation of (2) is now defined by the Markov chain

$\xi^{(N)}_n=\left(\xi^{(N,1)}_n, \cdots, \xi^{(N,N)}_n \right)$

on the product space $S^N$, starting with N independent random copies of $X_0$ and elementary transitions

$\mathbf{P}\left( \left. \xi^{(N,1)}_{n+1}=y^1,\cdots,\xi^{(N,N)}_{n+1}=y^N \right |\xi^{(N)}_n\right)=\prod_{i=1}^N K_{n+1,\eta_n^N}\left(\xi^{(N,i)}_n,y^i\right),$

with the empirical measure

$\eta^N_n=\frac{1}{N}\sum_{j=1}^N1_{\xi^{(N,j)}_n}$

Under some weak regularity conditions^[2] on the mapping $\Phi$ for any function $f: S\to \mathbf{R}$, we have the almost sure convergence

$\frac{1}{N}\sum_{j=1}^N f\left(\xi^{(N,j)}_n\right)\to_{N\uparrow\infty}E\left(f(\overline{X}_n)\right)=\sum_{x\in S}\eta_n(x)f(x)$

These nonlinear Markov processes and their mean field particle interpretation can be extended to time non homogeneous models on general measurable state spaces.^[2]

Feynman-Kac models

To illustrate the abstract models presented above, we consider a stochastic matrix $M=(M(x,y))_{x,y\in S}$ and some function $G : S \to (0,1)$. We associate with these two objects the mapping

$\begin{cases} \Phi : P(S) \to P(S) \\ (\eta_n(x))_{x\in S} \mapsto \left(\Phi(\eta_n)(y)\right)_{y\in S} \end{cases} \qquad \Phi(\eta_n)(y)=\sum_{x\in S} \Psi_{G}(\eta_n)(x)M(x,y)$

and the Boltzmann-Gibbs measures $\Psi_{G}(\eta_n)(x)$ defined by

$\Psi_{G}(\eta_n)(x)=\frac{\eta_n(x)G(x)}{\sum_{z\in S}\eta_n(z)G(z)}.$

We denote by $K_{\eta_n}=\left(K_{\eta_n}(x,y)\right)_{x,y\in S}$ the collection of stochastic matrices indexed by $\eta_n\in P(S)$ given by

$K_{\eta_n}(x,y)=\epsilon G(x) M(x,y)+(1-\epsilon G(x)) \Phi(\eta_n)(y)$

for some parameter $\epsilon \in [0,1]$. It is readily checked that the equation (2) is satisfied. In addition, we can also show (cf. for instance^[3]) that the solution of (1) is given by the Feynman-Kac formula

$\eta_n(x) =\frac{E\left(1_x(X_n)\prod_{p=0}^{n-1} G(X_p)\right)}{E\left(\prod_{p=0}^{n-1} G(X_p) \right)},$

with a Markov chain $X_n$ with initial distribution $\eta_0$ and Markov transition M.

For any function $f : S\to \mathbf{R}$ we have

$\eta_n(f):=\sum_{x\in S}\eta_n(x)f(x) =\frac{E\left(f(X_n)\prod_{p=0}^{n-1}G(X_p)\right)}{E\left(\prod_{p=0}^{n-1} G(X_p)\right)}$

If $G(x)=1$ is the unit function and $\epsilon=1$, then we have

$K_{\eta_n}(x,y)=M(x,y)=\mathbf{P} \left( \left. X_{n+1}=y \right | X_n=x\right), \qquad \eta_n(x) =E\left(1_x(X_n)\right)=\mathbf{P}(X_n=x).$

And the equation (2) reduces to the Chapman-Kolmogorov equation

$\eta_{n+1}(y)=\sum_{x\in S}\eta_n(x)M(x,y) \qquad \Leftrightarrow \qquad \mathbf{P}\left(X_{n+1}=y\right) =\sum_{x\in S} \mathbf{P}(X_{n+1}=y|X_n=x) \mathbf{P}\left(X_n=x\right)$

The mean field particle interpretation of this Feynman-Kac model is defined by sampling sequentially N conditionally independent random variables $\xi^{(N,i)}_{n+1}$ with probability distribution

$K_{n+1,\eta_n^N}\left(\xi^{(N,i)}_n,y\right)=\epsilon G\left(\xi^{(N,i)}_n\right) M\left(\xi^{(N,i)}_n,y\right)+\left(1-\epsilon G\left(\xi^{(N,i)}_n\right)\right) \sum_{j=1}^N \frac{G\left(\xi^{(N,j)}_n\right)}{\sum_{k=1}^N G\left(\xi^{(N,k)}_n\right)} M\left(\xi^{(N,j)}_n,y\right)$

In other words, with a probability $\epsilon G\left(\xi^{(N,i)}_n\right)$ the particle $\xi^{(N,i)}_n$ evolves to a new state $\xi^{(N,i)}_{n+1}=y$ randomly chosen with the probability distribution $M\left(\xi^{(N,i)}_n,y\right)$; otherwise, $\xi^{(N,i)}_n$ jumps to a new location $\xi^{(N,j)}_{n}$ randomly chosen with a probability proportional to $G\left(\xi^{(N,j)}_n\right)$ and evolves to a new state $\xi^{(N,i)}_{n+1}=y$ randomly chosen with the probability distribution $M\left(\xi^{(N,j)}_n, y\right).$ If $G(x)=1$ is the unit function and $\epsilon=1$, the interaction between the particle vanishes and the particle model reduces to a sequence of independent copies of the Markov chain $X_n$. When $\epsilon=0$ the mean field particle model described above reduces to a simple mutation-selection genetic algorithm with fitness function G and mutation transition M. These nonlinear Markov chain models and their mean field particle interpretation can be extended to time non homogeneous models on general measurable state spaces (including transition states, path spaces and random excursion spaces) and continuous time models.^[1][2][3]

Gaussian nonlinear state space models

We consider a sequence of real valued random variables $\left (\overline{X}_0, \overline{X}_1, \cdots \right)$ defined sequentially by the equations

$\overline{X}_{n+1}=E\left(a\left(\overline{X}_n\right)\right) b \left (\overline{X}_n \right )+c \left (\overline{X}_n \right )+\sigma W_n$

(3)

with a collection $W_n$ of independent standard Gaussian random variables, a positive parameter σ, some functions $a,b,c: \mathbf{R} \to \mathbf{R},$ and some standard Gaussian initial random state $\overline{X}_0$. We let $\eta_n$ be the probability distribution of the random state $\overline{X}_n$; that is, for any bounded measurable function f, we have

$E\left(f(\overline{X}_n)\right)=\int_{\mathbf{R}} f(x) \eta_n(dx),$

with

$\mathbf{P} \left (\overline{X}_n\in dx \right )=\eta_n(dx)$

The integral is the Lebesgue integral, and dx stands for an infinitesimal neighborhood of the state x. The Markov transition of the chain is given for any bounded measurable functions f by the formula

$E\left( \left. f \left (\overline{X}_{n+1} \right ) \right |\overline{X}_n=x\right)=\int_{\mathbf{R}} K_{\eta_n}(x,dy) f(y),$

with

$K_{\eta_n}(x,dy)=\mathbf{P} \left ( \left.\overline{X}_{n+1}\in dy\right | \overline{X}_n=x \right )=\frac{1}{\sqrt{2\pi}\sigma} \exp{\left\{-\frac{1}{2\sigma^2}\left(y-\left[b(x)\int_{\mathbf{R}} a(z) \eta_n(dz)+c(x)\right]\right)^2\right\}} dy$

Using the tower property of conditional expectations we prove that the probability distributions $\eta_n$ satisfy the nonlinear equation

$\int_{\mathbf{R}} \eta_{n+1}(dy) f(y)=\int_{\mathbf{R}}\left[\int_{\mathbf{R}} \eta_n(dx)K_{\eta_n}(x,dy)\right] f(y)$

for any bounded measurable functions f. This equation is sometimes written in the more synthetic form

$\eta_{n+1} =\Phi\left(\eta_n\right)= \eta_nK_{\eta_n}\quad\Leftrightarrow\quad\eta_{n+1}(dy)= \left(\eta_nK_{\eta_n}\right)(dy) =\int_{x\in \mathbf{R}}\eta_n(dx)K_{\eta_n}(x,dy)$

The mean field particle interpretation of this model is defined by the Markov chain

$\xi^{(N)}_n=\left(\xi^{(N,1)}_n, \cdots, \xi^{(N,N)}_n \right)$

on the product space $\mathbf{R}^N$ by

$\xi^{(N,i)}_{n+1}=\left(\frac{1}{N}\sum_{j=1}^N a\left(\xi^{(N,i)}_n\right)\right) b\left(\xi^{(N,i)}_n\right)+c\left(\xi^{(N,i)}_n\right)+\sigma W^i_n\qquad 1\leqslant i\leqslant N$

where

$\xi^{(N)}_0= \left(\xi^{(N,1)}_0, \cdots, \xi^{(N,N)}_0\right), \qquad \left( W^1_n, \cdots, W^N_n\right)$

stand for N independent copies of $\overline{X}_0$ and $W_n; n \geqslant 1,$ respectively. For regular models (for instance for bounded Lipschitz functions a, b, c) we have the almost sure convergence

$\frac{1}{N}\sum_{j=1}^N f\left(\xi^{(N,i)}_n\right)=\int_{\mathbf{R}} f(y) \eta^N_n(dy) \to_{N\uparrow\infty} E\left(f(\overline{X}_n)\right) = \int_{\mathbf{R}}f(y)\eta_n(dy),$

with the empirical measure

$\eta^N_n=\frac{1}{N}\sum_{j=1}^N \delta_{\xi^{(N,i)}_n}$

for any bounded measurable functions f (cf. for instance ^[2]). In the above display, $\delta_x$ stands for the Dirac measure at the state x.

Continuous time mean field models

We consider a standard Brownian motion $\overline{W}_{t_n}$ (a.k.a. Wiener Process) evaluated on a time mesh sequence $t_0=0<t_1<\cdots<t_n<\cdots$ with a given time step $t_n-t_{n-1}=h$. We choose $c(x)=x$ in equation (1), we replace $b(x)$ and σ by $b(x) \times h$ and $\sigma \times \sqrt{h}$, and we write $\overline{X}_{t_n}$ instead of $\overline{X}_n$ the values of the random states evaluated at the time step $t_n.$ Recalling that $\left(\overline{W}_{t_{n+1}}-\overline{W}_{t_n}\right)$ are independent centered Gaussian random variables with variance $t_n-t_{n-1} = h,$ the resulting equation can be rewritten in the following form

$\overline{X}_{t_{n+1}}-\overline{X}_{t_n}=E\left(a\left(\overline{X}_{t_n}\right)\right)b(\overline{X}_{t_n})h+\sigma\left(\overline{W}_{t_{n+1}}-\overline{W}_{t_n}\right)$

(4)

When h → 0, the above equation converge to the nonlinear diffusion process

$d\overline{X}_{t}=E\left(a\left(\overline{X}_{t}\right)\right)b(\overline{X}_{t})dt+\sigma d\overline{W}_{t}$

The mean field continuous time model associated with these nonlinear diffusions is the (interacting) diffusion process $\xi^{(N)}_t=\left(\xi^{(N,i)}_t\right)_{1\leqslant i\leqslant N}$ on the product space $\mathbf{R}^N$ defined by

$d\xi^{(N,i)}_{t}=\left(\frac{1}{N}\sum_{j=1}^N a\left(\xi^{(N,i)}_t\right)\right)b\left(\xi^{(N,i)}_t\right)+\sigma d\overline{W}_{t}^i\qquad 1\leqslant i\leqslant N$

where

$\xi^{(N)}_0= \left(\xi^{(N,1)}_0, \cdots, \xi^{(N,N)}_0\right), \qquad \left( \overline{W}_{t}^1, \cdots, \overline{W}_t^N\right)$

are N independent copies of $\overline{X}_0$ and $\overline{W}_t.$ For regular models (for instance for bounded Lipschitz functions a, b) we have the almost sure convergence

$\frac{1}{N}\sum_{j=1}^N f\left(\xi^{(N,i)}_t\right)=\int_{\mathbf{R}} f(y) \eta^N_t(dy)\to_{N\uparrow\infty} E\left(f(\overline{X}_t)\right)=\int_{\mathbf{R}} f(y) \eta_t(dy)$,

with $\eta_t=\text{Law}\left(\overline{X}_{t}\right),$ and the empirical measure

$\eta^N_t=\frac{1}{N}\sum_{j=1}^N \delta_{\xi^{(N,i)}_t}$

for any bounded measurable functions f (cf. for instance.^[7]). These nonlinear Markov processes and their mean field particle interpretation can be extended to interacting jump-diffusion processes^{[1][2][23][25]}

References

1. Kolokoltsov, Vassili (2010). Nonlinear Markov processes. Cambridge Univ. Press. p. 375.
2. Del Moral, Pierre (2013). Mean field simulation for Monte Carlo integration. Chapman & Hall/CRC Press. p. 626. Monographs on Statistics & Applied Probability
3. Del Moral, Pierre (2004). Feynman-Kac formulae. Genealogical and interacting particle approximations. Springer. p. 575. Series: Probability and Applications
4. Del Moral, Pierre; Miclo, Laurent (2000). Branching and Interacting Particle Systems Approximations of Feynman-Kac Formulae with Applications to Non-Linear Filtering. (PDF). Lecture Notes in Mathematics 1729. pp. 1–145. doi:10.1007/bfb0103798.
5. McKean, Henry, P. (1967). "Propagation of chaos for a class of non-linear parabolic equations". Lecture Series in Differential Equations, Catholic Univ. 7: 41–57.
6. Méléard, Sylvie; Roelly, Sylvie (1987). "A propagation of chaos result for a system of particles with moderate interaction" (PDF). Stoch. Proc. and Appl. 26: 317–332. doi:10.1016/0304-4149(87)90184-0.
7. Sznitman, Alain-Sol (1991). Topics in propagation of chaos. Springer, Berlin. pp. 164–251. Saint-Flour Probability Summer School, 1989
8. Kac, Mark (1976). Probability and Related Topics in Physical Sciences. Topics in Physical Sciences. American Mathematical Society, Providence, Rhode Island.
9. McKean, Henry, P. (1966). "A class of Markov processes associated with nonlinear parabolic equations" (PDF). Proc. Natl. Acad. Sci. USA 56 (6): 1907–1911. Bibcode:1966PNAS...56.1907M. doi:10.1073/pnas.56.6.1907. PMC 220210. PMID 16591437.
10. Braun, Werner; Hepp, Klaus (1977). "The Vlasov dynamics and its fluctuations in the 1 limit of interacting classical particles.". Communications in Mathematical Physics 56: 101–113. Bibcode:1977CMaPh..56..101B. doi:10.1007/bf01611497.
11. Oelschläger, Karl (1984). "A martingale approach to the law of large numbers for weakly interacting stochastic processes". Ann. Probab. 12: 458–479. doi:10.1214/aop/1176993301.
12. Oelschläger, Karl (1989). "On the derivation of reaction-diffusion equations as limit of dynamics of systems of moderately interacting stochastic processes". Prob. Th. Rel. Fields 82: 565–586.
13. Oelschläger, Karl (1990). "Large systems of interacting particles and porous medium equation". J. Differential Equations 88: 294–346. Bibcode:1990JDE....88..294O. doi:10.1016/0022-0396(90)90101-t.
14. Ben Arous, Gérard; Brunaud, Marc (1990). "Méthode de Laplace: Etude variationnelle des fluctuations de diffusions de type "champ moyen"". Stochastics 31, 79–144, (1990) 31: 79–144. doi:10.1080/03610919008833649.
15. Dawson, Donald; Vaillancourt, Jean (1995). "Nonlinear Differential Equations and Applications". Nonlinear Differential Equations and Applications 2 (2): 199–229. doi:10.1007/bf01295311.
16. Dawson, Donald; Gartner, Jurgen (1987). "Large deviations from the McKean-Vlasov limit for weakly interacting diffusions". Stochastics 20: 247–308. doi:10.1080/17442508708833446.
17. Gartner, Jurgen (1988). "J. GÄRTNER, On the McKean-Vlasov limit for interacting diffusions". Math. Nachr. 137: 197–248. doi:10.1002/mana.19881370116.
18. Léonard, Christian (1986). "Une loi des grands nombres pour des systèmes de diffusions avec interaction et à coefficients non bornés". Ann. I.H.P. 22: 237–262.
19. Sznitman, Alain-Sol (1984). "Nonlinear reflecting diffusion process, and the propagation of chaos and fluctuations associated". J. Funct. Anal. 36: 311–336.
20. Tanaka, Hiroshi (1984). "Tanaka, H.: Limit theorems for certain diffusion processes with interaction". Proceeding of the Taniguchi International Symposium on Stochastic Analysis: 469–488.
21. Grunbaum., F. Alberto (1971). "Propagation of chaos for the Boltzmann equation". Archive for Rational Mechanics and Analysis 42: 323–345. Bibcode:1971ArRMA..42..323G. doi:10.1007/BF00250440.
22. Shiga, Tokuzo; Tanaka, Hiroshi (1985). "Central limit theorem for a system of Markovian particles with mean field interactions". Zeitschrift für Wahrscheinlichkeitstheorie und Verwandte Gebiete 69 (3).
23. Graham, Carl (1992). "Non linear diffusions with jumps". Ann. I.H.P. 28 (3): 393–402.
24. Méléard, Sylvie (1996). "Asymptotic behaviour of some interacting particle systems; McKean-Vlasov and Boltzmann models". Lecture Notes in Mathematics, Springer 1627: 42–95. doi:10.1007/bfb0093177.
25. Graham, Carl; Méléard, Sylvie (1997). "Stochastic particle approximations for generalized Boltzmann models and convergence estimates.". Annals of Probability 25 (1): 115–132. doi:10.1214/aop/1024404281.
26. Herman, Kahn; Harris, Theodore, E. (1951). "Estimation of particle transmission by random sampling" (PDF). Natl. Bur. Stand. Appl. Math. Ser. 12: 27–30.
27. Turing, Alan M. "Computing machinery and intelligence". Mind LIX (238): 433–460. doi:10.1093/mind/LIX.236.433.
28. Barricelli, Nils Aall (1954). "Esempi numerici di processi di evoluzione". Methodos: 45–68.
29. Barricelli, Nils Aall (1957). "Symbiogenetic evolution processes realized by artificial methods". Methodos: 143–182.
30. Fraser, Alex (1957). "Simulation of genetic systems by automatic digital computers. I. Introduction". Aust. J. Biol. Sci. 10: 484–491.
31. Del Moral, Pierre; Miclo, Laurent (2000). "A Moran particle system approximation of Feynman-Kac formulae.". Stochastic Processes and their Applications 86 (2): 193–216. doi:10.1016/S0304-4149(99)00094-0.
32. Del Moral, Pierre (2003). "Particle approximations of Lyapunov exponents connected to Schrödinger operators and Feynman-Kac semigroups" (PDF). ESAIM Probability & Statistics 7: 171–208. doi:10.1051/ps:2003001.
33. Assaraf, Roland; Caffarel, Michel; Khelif, Anatole (2000). "Diffusion Monte Carlo Methods with a fixed number of walkers" (PDF). Phys. Rev. E 61: 4566–4575. Bibcode:2000PhRvE..61.4566A. doi:10.1103/physreve.61.4566.
34. Caffarel, Michel; Ceperley, David; Kalos, Malvin (1993). "Comment on Feynman-Kac Path-Integral Calculation of the Ground-State Energies of Atoms". Phys. Rev. Lett. 71: 2159. Bibcode:1993PhRvL..71.2159C. doi:10.1103/physrevlett.71.2159.
35. Hetherington, Jack, H. (1984). "Observations on the statistical iteration of matrices". Phys. Rev. A. 30 (2713): 2713–2719. Bibcode:1984PhRvA..30.2713H. doi:10.1103/PhysRevA.30.2713.
36. Fermi, Enrique; Richtmyer, Robert, D. (1948). "Note on census-taking in Monte Carlo calculations" (PDF). LAM 805 (A). Declassified report Los Alamos Archive
37. Rosenbluth, Marshall, N.; Rosenbluth, Arianna, W. (1955). "Monte-Carlo calculations of the average extension of macromolecular chains". J. Chem. Phys 23: 356–359. Bibcode:1955JChPh..23..356R. doi:10.1063/1.1741967.
38. Gordon, N. J.; Salmond, D. J.; Smith, A. F. M. (1993). "Novel approach to nonlinear/non-Gaussian Bayesian state estimation". IEE Proceedings F on Radar and Signal Processing 140 (2): 107–113. doi:10.1049/ip-f-2.1993.0015. Retrieved 2009-09-19.
39. Kitagawa, G. (1996). "Monte carlo filter and smoother for non-Gaussian nonlinear state space models". Journal of Computational and Graphical Statistics 5 (1): 1–25. doi:10.2307/1390750. JSTOR 1390750.
40. Carvalho, Himilcon; Del Moral, Pierre; Monin, André; Salut, Gérard (July 1997). "Optimal Non-linear Filtering in GPS/INS Integration." (PDF). IEEE-Trans. on Aerospace and electronic systems 33 (3).
41. Del Moral, Pierre (1996). "Non Linear Filtering: Interacting Particle Solution." (PDF). Markov Processes and Related Fields 2 (4): 555–580.
42. P. Del Moral, G. Rigal, and G. Salut. Estimation and nonlinear optimal control : An unified framework for particle solutions
    LAAS-CNRS, Toulouse, Research Report no. 91137, DRET-DIGILOG- LAAS/CNRS contract, April (1991).
43. P. Del Moral, G. Rigal, and G. Salut. Nonlinear and non Gaussian particle filters applied to inertial platform repositioning.
    LAAS-CNRS, Toulouse, Research Report no. 92207, STCAN/DIGILOG-LAAS/CNRS Convention STCAN no. A.91.77.013, (94p.) September (1991).
44. P. Del Moral, G. Rigal, and G. Salut. Estimation and nonlinear optimal control : Particle resolution in filtering and estimation. Experimental results.
    Convention DRET no. 89.34.553.00.470.75.01, Research report no.2 (54p.), January (1992).
45. P. Del Moral, G. Rigal, and G. Salut. Estimation and nonlinear optimal control : Particle resolution in filtering and estimation. Theoretical results
    Convention DRET no. 89.34.553.00.470.75.01, Research report no.3 (123p.), October (1992).
46. P. Del Moral, J.-Ch. Noyer, G. Rigal, and G. Salut. Particle filters in radar signal processing : detection, estimation and air targets recognition.
    LAAS-CNRS, Toulouse, Research report no. 92495, December (1992).
47. P. Del Moral, G. Rigal, and G. Salut. Estimation and nonlinear optimal control : Particle resolution in filtering and estimation.
    Studies on: Filtering, optimal control, and maximum likelihood estimation. Convention DRET no. 89.34.553.00.470.75.01. Research report no.4 (210p.), January (1993).
48. Del Moral, Pierre (1996). "Non Linear Filtering: Interacting Particle Solution." (PDF). Markov Processes and Related Fields 2 (4): 555–580.
49. Del Moral, Pierre (1998). "Measure Valued Processes and Interacting Particle Systems. Application to Non Linear Filtering Problems". Annals of Applied Probability (Publications du Laboratoire de Statistique et Probabilités, 96-15 (1996) ed.) 8 (2): 438–495. doi:10.1214/aoap/1028903535.
50. Crisan, Dan; Gaines, Jessica; Lyons, Terry (1998). "Convergence of a branching particle method to the solution of the Zakai". SIAM Journal on Applied Mathematics 58 (5): 1568–1590. doi:10.1137/s0036139996307371.
51. Crisan, Dan; Lyons, Terry (1997). "Nonlinear filtering and measure-valued processes". Probability Theory and Related Fields 109 (2): 217–244. doi:10.1007/s004400050131.
52. Crisan, Dan; Lyons, Terry (1999). "A particle approximation of the solution of the Kushner–Stratonovitch equation". Probability Theory and Related Fields 115 (4): 549–578. doi:10.1007/s004400050249.
53. Crisan, Dan; Del Moral, Pierre; Lyons, Terry (1999). "Discrete filtering using branching and interacting particle systems" (PDF). Markov Processes and Related Fields 5 (3): 293–318.
54. Del Moral, Pierre; Guionnet, Alice (2001). "On the stability of interacting processes with applications to filtering and genetic algorithms". Annales de l'Institut Henri Poincaré 37 (2): 155–194. Bibcode:2001AnIHP..37..155D. doi:10.1016/s0246-0203(00)01064-5.
55. Del Moral, Pierre; Guionnet, Alice (1999). "On the stability of Measure Valued Processes with Applications to filtering". C.R. Acad. Sci. Paris 39 (1): 429–434.
56. Malrieu, Florent (2001). "Logarithmic Sobolev inequalities for some nonlinear PDE’s". Stochastic Process. Appl 95 (1): 109–132. doi:10.1016/s0304-4149(01)00095-3.
57. Del Moral, Pierre; Miclo, Laurent (2001). "Genealogies and Increasing Propagations of Chaos for Feynman-Kac and Genetic Models". Annals of Applied Probability 11 (4): 1166–1198.
58. Del Moral, Pierre; Doucet, Arnaud; Singh, Sumeetpal, S. (2010). "A Backward Particle Interpretation of Feynman-Kac Formulae" (PDF). M2AN 44 (5): 947–976. doi:10.1051/m2an/2010048.
59. Del Moral, Pierre; Doucet, Arnaud; Jasra, Ajay (2012). "On Adaptive Resampling Procedures for Sequential Monte Carlo Methods" (PDF). Bernoulli 18 (1): 252–278. doi:10.3150/10-bej335.
60. Vergé, Christelle; Dubarry, Cyrille; Del Moral, Pierre; Moulines, Eric (2013). "On parallel implementation of Sequential Monte Carlo methods: the island particle model". Statistics and Computing 25: 243–260. doi:10.1007/s11222-013-9429-x.
61. Chopin, Nicolas; Jacob, Pierre, E.; Papaspiliopoulos, Omiros. "SMC^2: an efficient algorithm for sequential analysis of state-space models". arXiv:1101.1528v3.
62. Andrieu, Christophe; Doucet, Arnaud; Holenstein, Roman (2010). "Particle Markov chain Monte Carlo methods". Journal Royal Statistical Society B 72 (3): 269–342. doi:10.1111/j.1467-9868.2009.00736.x.
63. Del Moral, Pierre; Patras, Frédéric; Kohn, Robert (2014). "On Feynman-Kac and particle Markov chain Monte Carlo models". arXiv:1404.5733.
64. Cercignani, Carlo; Illner, Reinhard; Pulvirenti, Mario (1994). The Mathematical Theory of Dilute Gases. Springer.
65. Schrodinger, Erwin (1926). "An Undulatory Theory of the Mechanics of Atoms and Molecules". Physical Review 28 (6): 1049–1070. Bibcode:1926PhRv...28.1049S. doi:10.1103/physrev.28.1049.
66. Del Moral, Pierre; Doucet, Arnaud (2004). "Particle Motions in Absorbing Medium with Hard and Soft Obstacles". Stochastic Analysis and Applications 22 (5): 1175–1207. doi:10.1081/SAP-200026444.
67. Del Moral, Pierre; Doucet, Arnaud; Jasra, Ajay (2006). "Sequential Monte Carlo samplers" (PDF). J. Royal Statist. Soc. B 68: 411–436. doi:10.1111/j.1467-9868.2006.00553.x.
68. Lelièvre, Tony; Rousset, Mathias; Stoltz, Gabriel (2007). "Computation of free energy differences through nonequilibrium stochastic dynamics: the reaction coordinate case.". J. Comp. Phys. 222 (2): 624–643. arXiv:cond-mat/0603426. Bibcode:2007JCoPh.222..624L. doi:10.1016/j.jcp.2006.08.003.
69. Lelièvre, Tony; Rousset, Mathias; Stoltz, Gabriel (2010). "Free energy computations: A mathematical perspective". Imperial College Press: 472.
70. Caron, François; Del Moral, Pierre; Doucet, Arnaud; Pace, Michele (2011). "On the Stability and the Approximation of Branching Distribution Flows, with Applications to Nonlinear Multiple Target Filtering". SIAM J. Control Optim. 49: 1766–1792. doi:10.1080/07362994.2011.598797.
71. Dynkin, Eugène, B. (1994). An Introduction to Branching Measure-Valued Processes. CRM Monograph Series. p. 134. ISBN 0-8218-0269-0.
72. Zoia, Andrea; Dumonteil, Eric; Mazzolo, Alain. "Discrete Feynman-Kac formulas for branching random walks" (PDF). EPL (Europhysics Letters) 98 (40012). arXiv:1202.2811. Bibcode:2012EL.....9840012Z. doi:10.1209/0295-5075/98/40012.
73. Pitman, Jim; Fitzsimmons, Patrick, J. (1999). "Kac’s moment formula and the Feynman–Kac formula for additive functionals of a Markov process". Stochastic Processes and their Applications 79 (1): 117–134. doi:10.1016/S0304-4149(98)00081-7.
74. Arendt, Wolfgang; Batty, Charles, J.K. (1993). "Absorption semigroups and Dirichlet boundary conditions" (PDF). Math. Ann. 295: 427–448. doi:10.1007/bf01444895.
75. Lant, Timothy; Thieme, Horst (2007). "Perturbation of Transition Functions and a Feynman-Kac Formula for the Incorporation of Mortality". Positivity 11 (2): 299–318. doi:10.1007/s11117-006-2044-8.
76. Takeda, Masayoshi (2008). "Some Topics connected with Gaugeability for Feynman-Kac Functionals" (PDF). RIMS Kokyuroku Bessatsu B6: 221–236.
77. Yaglom, Isaak (1947). "Certain limit theorems of the theory of branching processes". Dokl. Acad. Nauk SSSR 56: 795–798.
78. Del Moral, Pierre; Miclo, Laurent (2002). "On the Stability of Non Linear Semigroup of Feynman-Kac Type" (PDF). Annales de la Faculté des Sciences de Toulouse 11 (2): 135–175. doi:10.5802/afst.1021.
79. Kallel, Leila; Naudts, Bart; Rogers, Alex. Theoretical Aspects of Evolutionary Computing. Springer, Berlin, New York; Natural computing series. p. 497. ISBN 3540673962.
80. Del Moral, Pierre; Kallel, Leila; Rowe, John (2001). "Modeling genetic algorithms with interacting particle systems". Revista de Matematica: Teoria y Aplicaciones 8 (2): 19–77. doi:10.15517/rmta.v8i2.201. CiteSeerX: 10.1.1.87.7330.
81. Del Moral, Pierre; Guionnet, Alice (2001). "On the stability of interacting processes with applications to filtering and genetic algorithms". Annales de l'Institut Henri Poincaré 37 (2): 155–194. Bibcode:2001AnIHP..37..155D. doi:10.1016/S0246-0203(00)01064-5.
82. Aumann, Robert John (1964). "Markets with a continuum of traders". Econometrica 32 (1-2): 39–50. doi:10.2307/1913732.
83. Jovanovic, Boyan; Rosenthal, Robert W. (1988). "Anonymous sequential games". Journal of Mathematical Economics 17 (1): 77–87. doi:10.1016/0304-4068(88)90029-8.
84. Huang, Minyi.Y; Malhame, Roland P.; Caines, Peter E. (2006). "Large Population Stochastic Dynamic Games: Closed-Loop McKean–Vlasov Systems and the Nash Certainty Equivalence Principle". Special issue in honor of the 65th birthday of Tyrone Duncan,Communications in Information and Systems 6: 221–252.
85. Maynard Smith, John (1982). Evolution and the Theory of Games. Cambridge University Press, Cambridge.
86. Kolokoltsov, Vassili; Li, Jiajie; Yang, Wei. "Mean field games and nonlinear Markov processes". arXiv:1112.3744v2.
87. Lasry, Jean Michel; Lions, Pierre Louis (2007). "Mean field games". Japanese J. Math 2 (1): 229–260. doi:10.1007/s11537-007-0657-8.
88. Carmona, René; Fouque, Jean Pierre; Sun, Li-Hsien (2014). "Mean Field Games and Systemic Risk". Communications in Mathematical Sciences.
89. Budhiraja, Amarjit; Del Moral, Pierre; Rubenthaler, Sylvain (2013). "Discrete time Markovian agents interacting through a potential". ESAIM Probability & Statistics 17: 614–634. doi:10.1051/ps/2012014.
90. Aumann, Robert (1964). "Markets with a continuum of traders" (PDF). Econometrica 32 (1-2): 39–50. doi:10.2307/1913732.
91. Del Moral, Pierre; Lézaud, Pascal (2006). Branching and interacting particle interpretation of rare event probabilities. (PDF) (stochastic Hybrid Systems: Theory and Safety Critical Applications, eds. H. Blom and J. Lygeros. ed.). Springer, Berlin. pp. 277–323.
92. Crisan, Dan; Del Moral, Pierre; Lyons, Terry (1998). "Discrete Filtering Using Branching and Interacting Particle Systems." (PDF). Markov Processes and Related Fields 5 (3): 293–318.
93. Crisan, Dan; Del Moral, Pierre; Lyons, Terry (1998). "Interacting Particle Systems Approximations of the Kushner Stratonovitch Equation" (PDF). Advances in Applied Probability 31 (3): 819–838. doi:10.1239/aap/1029955206.
94. Pace, Michele; Del Moral, Pierre (2013). "Mean-Field PHD Filters Based on Generalized Feynman-Kac Flow". IEEE Journal of Selected Topics in Signal Processing. Special Issue on Multi-target tracking (7-3): 484–495. Bibcode:2013ISTSP...7..484P. doi:10.1109/JSTSP.2013.2250909.
95. Cappe, O.; Moulines, E.; Ryden, T. (2005). Inference in Hidden Markov Models. Springer.
96. Liu, J. (2001). Monte Carlo strategies in Scientific Computing. Springer.
97. Doucet, A. (2001). de Freitas, J. F. G.; Gordon, J., eds. Sequential Monte Carlo Methods in Practice. Springer.
98. Botev, Z. I.; Kroese, D. P. (2008). "Efficient Monte Carlo simulation via the generalized splitting method". Methodology and Computing in Applied Probability 10 (4): 1–16. doi:10.1007/s11009-008-9073-7.
99. Botev, Z. I.; Kroese, D. P. (2012). "Efficient Monte Carlo simulation via the generalized splitting method". Statistics and Computing 22 (1): 1–16. doi:10.1007/s11222-010-9201-4.
100. Cérou, Frédéric; Del Moral, Pierre; Furon, Teddy; Guyader, Arnaud (2012). "Sequential Monte Carlo for Rare event estimation". Statistics and Computing 22 (3): 795–808. doi:10.1007/s11222-011-9231-6.

External links

• Feynman-Kac models and interacting particle systems Theoretical aspects and a list of application domains of Feynman-Kac particle methods.
• Sequential Monte Carlo method and particle filters resources
• Interacting Particle Systems resources
• QMC in Cambridge and around the world General information about Quantum Monte Carlo.
• EVOLVER Software package for stochastic optimisation using genetic algorithms
• CASINO Quantum Monte Carlo program developed by the Theory of Condensed Matter group at the Cavendish Laboratory in Cambridge.
• Biips is a probabilistic programming software for Bayesian inference with interacting particle systems.