Linear-quadratic regulator

The theory of optimal control is concerned with operating a dynamic system at minimum cost. The case where the system dynamics are described by a set of linear differential equations and the cost is described by a quadratic function is called the LQ problem. One of the main results in the theory is that the solution is provided by the linear-quadratic regulator (LQR), a feedback controller whose equations are given below. The LQR is an important part of the solution to the LQG (Linear-Quadratic-Gaussian) problem. Like the LQR problem itself, the LQG problem is one of the most fundamental problems in control theory.

General description

This means that the settings of a (regulating) controller governing either a machine or process (like an airplane or chemical reactor) are found by using a mathematical algorithm that minimizes a cost function with weighting factors supplied by a human (engineer). The "cost" (function) is often defined as a sum of the deviations of key measurements from their desired values. In effect this algorithm finds those controller settings that minimize the undesired deviations, like deviations from desired altitude or process temperature. Often the magnitude of the control action itself is included in this sum so as to keep the energy expended by the control action itself limited.

In effect, the LQR algorithm takes care of the tedious work done by the control systems engineer in optimizing the controller. However, the engineer still needs to specify the weighting factors and compare the results with the specified design goals. Often this means that controller synthesis will still be an iterative process where the engineer judges the produced "optimal" controllers through simulation and then adjusts the weighting factors to get a controller more in line with the specified design goals.

The LQR algorithm is, at its core, just an automated way of finding an appropriate state-feedback controller. As such it is not uncommon to find that control engineers prefer alternative methods like full state feedback (also known as pole placement) to find a controller over the use of the LQR algorithm. With these the engineer has a much clearer linkage between adjusted parameters and the resulting changes in controller behavior. Difficulty in finding the right weighting factors limits the application of the LQR based controller synthesis.

Finite-horizon, continuous-time LQR

For a continuous-time linear system, defined on $t\in[t_0,t_1]$, described by

$\dot{x} = Ax + Bu$

with a quadratic cost function defined as

$J = \frac{1}{2} x^T(t_1)F(t_1)x(t_1) + \int\limits_{t_0}^{t_1} \left( x^T Q x + u^T R u + 2 x^T N u \right) dt$

the feedback control law that minimizes the value of the cost is

$u = -K x \,$

where $K$ is given by

$K = R^{-1} (B^T P(t) + N^T) \,$

and $P$ is found by solving the continuous time Riccati differential equation.

$A^T P(t) + P(t) A - (P(t) B + N) R^{-1} (B^T P(t) + N^T) + Q = - \dot{P}(t) \,$

with the boundary condition

$P(t_1) = F(t_1).$

The first order conditions for J_min are

(i) State equation

$\dot{x} = Ax + Bu$

(ii) Co-state equation

$-\dot{\lambda} = Qx + Nu + A^T \lambda$

(iii) Stationary equation

$0 = Ru + N^Tx + B^T \lambda$

(iv) Boundary conditions

$x(t_0) = x_0$

and $\lambda(t_1) = F(t_1) x(t_1)$

Infinite-horizon, continuous-time LQR

For a continuous-time linear system described by

$\dot{x} = Ax + Bu$

with a cost functional defined as

$J = \int_{0}^\infty \left( x^T Q x + u^T R u + 2 x^T N u \right) dt$

the feedback control law that minimizes the value of the cost is

$u = -K x \,$

where $K$ is given by

$K = R^{-1} (B^T P + N^T) \,$

and $P$ is found by solving the continuous time algebraic Riccati equation

$A^T P + P A - (P B + N) R^{-1} (B^T P + N^T) + Q = 0 \,$

This can be also written as

$\mathcal A^T P + P \mathcal A - P B R^{-1} B^T P + \mathcal Q = 0 \,$

with

$\mathcal A = A - B R^{-1} N^T \qquad \mathcal Q = Q - N R^{-1} N^T \,$

Finite-horizon, discrete-time LQR

For a discrete-time linear system described by ^[1]

$x_{k+1} = A x_k + B u_k \,$

with a performance index defined as

$J = \sum\limits_{k=0}^{N} \left( x_k^T Q x_k + u_k^T R u_k + 2 x_k^T N u_k \right)$

the optimal control sequence minimizing the performance index is given by

$u_k = -F_k x_{k} \,$

where

$F_k = (R + B^T P_k B)^{-1} (B^T P_k A + N^T) \,$

and $P_k$ is found iteratively backwards in time by the dynamic Riccati equation

$P_{k-1} = A^T P_k A - (A^T P_k B + N) \left( R + B^T P_k B \right)^{-1} (B^T P_k A + N^T) + Q$

from terminal condition $P_N = Q$. Note that $u_N$ is not defined, since $x$ is driven to its final state $x_N$ by $A x_{N-1} + B u_{N-1}$.

Infinite-horizon, discrete-time LQR

For a discrete-time linear system described by

$x_{k+1} = A x_k + B u_k \,$

with a performance index defined as

$J = \sum\limits_{k=0}^{\infty} \left( x_k^T Q x_k + u_k^T R u_k + 2 x_k^T N u_k \right)$

the optimal control sequence minimizing the performance index is given by

$u_k = -F x_k \,$

where

$F = (R + B^T P B)^{-1} (B^T P A + N^T) \,$

and $P$ is the unique positive definite solution to the discrete time algebraic Riccati equation (DARE)

$P = A^T P A - (A^T P B + N) \left( R + B^T P B \right)^{-1} (B^T P A + N^T) + Q$.

This can be also written as

$P = \mathcal A^T P \mathcal A - \mathcal A^T P B \left( R + B^T P B \right)^{-1} B^T P \mathcal A + \mathcal Q$

with

$\mathcal A = A - B R^{-1} N^T \qquad \mathcal Q = Q - N R^{-1} N^T$.

Note that one way to solve the algebraic Riccati equation is by iterating the dynamic Riccati equation of the finite-horizon case until it converges.

References

1. ↑ Chow, Gregory C. (1986). Analysis and Control of Dynamic Economic Systems. Krieger Publ. Co. ISBN 0-89874-969-7.

• Kwakernaak, Huibert and Sivan, Raphael (1972). Linear Optimal Control Systems. First Edition. Wiley-Interscience. ISBN 0-471-51110-2.

• Sontag, Eduardo (1998). Mathematical Control Theory: Deterministic Finite Dimensional Systems. Second Edition. Springer. ISBN 0-387-98489-5.

External links

• MATLAB function for Linear Quadratic Regulator design
• Mathematica function for Linear Quadratic Regulator design