Linear subspace

One-dimensional subspaces in the two-dimensional vector space over the finite field F5. The origin (0, 0), marked with green circles, belongs to any of six 1-subspaces, while each of 24 remaining points belongs to exactly one; a property which holds for 1-subspaces over any field and in all dimensions. All F5^2 (i.e. a 5 × 5 square) is pictured four times for a better visualization

In linear algebra and related fields of mathematics, a linear subspace (or vector subspace) is a vector space that is a subset of some other (higher-dimension) vector space. A linear subspace is usually called simply a subspace when the context serves to distinguish it from other kinds of subspaces.

Definition and useful characterization of subspace

Let K be a field (such as the real numbers), and let V be a vector space over K. As usual, we call elements of V vectors and call elements of K scalars. Ignoring the full extent of mathematical generalization, scalars can be understood simply as numbers. Suppose that W is a subset of V. If W is a vector space itself (which means that it is closed under operations of addition and scalar multiplication), with the same vector space operations as V has, then W is a subspace of V.

To use this definition, we don't have to prove that all the properties of a vector space hold for W. Instead, we can prove a theorem that gives us an easier way to show that a subset of a vector space is a subspace.

Theorem: Let V be a vector space over the field K, and let W be a subset of V. Then W is a subspace if and only if W satisfies the following three conditions:

1. The zero vector, 0, is in W.
2. If u and v are elements of W, then the sum u + v is an element of W;
3. If u is an element of W and c is a scalar from K, then the product cu is an element of W.

Proof: Firstly, property 1 ensures W is nonempty. Looking at the definition of a vector space, we see that properties 2 and 3 above assure closure of W under addition and scalar multiplication, so the vector space operations are well defined. Since elements of W are necessarily elements of V, axioms 1, 2 and 5–8 of a vector space are satisfied. By the closure of W under scalar multiplication (specifically by 0 and −1), the vector space's definitional axiom identity element of addition and axiom inverse element of addition are satisfied.

Conversely, if W is a subspace of V, then W is itself a vector space under the operations induced by V, so properties 2 and 3 are satisfied. By property 3, −w is in W whenever w is, and it follows that W is closed under subtraction as well. Since W is nonempty, there is an element x in W, and $\bold x - \bold x = {\bold 0}$ is in W, so property 1 is satisfied. One can also argue that since W is nonempty, there is an element x in W, and 0 is in the field K so $0 \bold x = {\bold 0}$ and therefore property 1 is satisfied.

Examples

Example I: Let the field K be the set R of real numbers, and let the vector space V be the real coordinate space R³. Take W to be the set of all vectors in V whose last component is 0. Then W is a subspace of V.

Proof:

1. Given u and v in W, then they can be expressed as u = (u₁, u₂, 0) and v = (v₁, v₂, 0). Then u + v = (u₁+v₁, u₂+v₂, 0+0) = (u₁+v₁, u₂+v₂, 0). Thus, u + v is an element of W, too.
2. Given u in W and a scalar c in R, if u = (u₁, u₂, 0) again, then cu = (cu₁, cu₂, c0) = (cu₁, cu₂,0). Thus, cu is an element of W too.

Example II: Let the field be R again, but now let the vector space be the Cartesian plane R². Take W to be the set of points (x, y) of R² such that x = y. Then W is a subspace of R².

Proof:

1. Let p = (p₁, p₂) and q = (q₁, q₂) be elements of W, that is, points in the plane such that p₁ = p₂ and q₁ = q₂. Then p + q = (p₁+q₁, p₂+q₂); since p₁ = p₂ and q₁ = q₂, then p₁ + q₁ = p₂ + q₂, so p + q is an element of W.
2. Let p = (p₁, p₂) be an element of W, that is, a point in the plane such that p₁ = p₂, and let c be a scalar in R. Then cp = (cp₁, cp₂); since p₁ = p₂, then cp₁ = cp₂, so cp is an element of W.

In general, any subset of the real coordinate space Rⁿ that is defined by a system of homogeneous linear equations will yield a subspace. (The equation in example I was z = 0, and the equation in example II was x = y.) Geometrically, these subspaces are points, lines, planes, and so on, that pass through the point 0.

Examples related to calculus

Example III: Again take the field to be R, but now let the vector space V be the set R^R of all functions from R to R. Let C(R) be the subset consisting of continuous functions. Then C(R) is a subspace of R^R.

Proof:

1. We know from calculus that 0 ∈ C(R) ⊂ R^R.
2. We know from calculus that the sum of continuous functions is continuous.
3. Again, we know from calculus that the product of a continuous function and a number is continuous.

Example IV: Keep the same field and vector space as before, but now consider the set Diff(R) of all differentiable functions. The same sort of argument as before shows that this is a subspace too.

Examples that extend these themes are common in functional analysis.

Properties of subspaces

A way to characterize subspaces is that they are closed under linear combinations. That is, a nonempty set W is a subspace if and only if every linear combination of (finitely many) elements of W also belongs to W. Conditions 2 and 3 for a subspace are simply the most basic kinds of linear combinations.

In a topological vector space X, a subspace W need not be closed in general, but a finite-dimensional subspace is always closed.^[1] The same is true for subspaces of finite codimension, i.e. determined by a finite number of continuous linear functionals.

Descriptions

Descriptions of subspaces include the solution set to a homogeneous system of linear equations, the subset of Euclidean space described by a system of homogeneous linear parametric equations, the span of a collection of vectors, and the null space, column space, and row space of a matrix. Geometrically (especially, over the field of real numbers and its subfields), a subspace is a flat in an n-space that passes through the origin.

A natural description of an 1-subspace is the scalar multiplication of one non-zero vector v to all possible scalar values. 1-subspaces specified by two vectors are equal if and only if one vector can be obtained from another with scalar multiplication:

$\exist c\in K: \mathbf{v}' = c\mathbf{v}\text{ (or }\mathbf{v} = \frac{1}{c}\mathbf{v}'\text{)}$

This idea is generalized for higher dimensions with linear span, but criteria for equality of k-spaces specified by sets of k vectors are not so simple.

A dual description is provided with linear functionals (usually implemented as linear equations). One non-zero linear functional F specifies its kernel subspace F = 0 of codimension 1. Subspaces of codimension 1 specified by two linear functionals are equal if and only if one functional can be obtained from another with scalar multiplication (in the dual space):

$\exist c\in K: \mathbf{F}' = c\mathbf{F}\text{ (or }\mathbf{F} = \frac{1}{c}\mathbf{F}'\text{)}$

It is generalized for higher codimensions with a system of equations. The following two subsections will present this latter description in details, and the remaining four subsections further describe the idea of linear span.

Systems of linear equations

The solution set to any homogeneous system of linear equations with n variables is a subspace in the coordinate space Kⁿ:

$\left\{ \left[\!\! \begin{array}{c} x_1 \\ x_2 \\ \vdots \\ x_n \end{array} \!\!\right] \in K^n : \begin{alignat}{6} a_{11} x_1 &&\; + \;&& a_{12} x_2 &&\; + \cdots + \;&& a_{1n} x_n &&\; = 0& \\ a_{21} x_1 &&\; + \;&& a_{22} x_2 &&\; + \cdots + \;&& a_{2n} x_n &&\; = 0& \\ \vdots\;\;\; && && \vdots\;\;\; && && \vdots\;\;\; && \vdots\,& \\ a_{m1} x_1 &&\; + \;&& a_{m2} x_2 &&\; + \cdots + \;&& a_{mn} x_n &&\; = 0& \end{alignat} \right\}.$

For example (over real or rational numbers), the set of all vectors (x, y, z) satisfying the equations

$x + 3y + 2z = 0 \;\;\;\;\text{and}\;\;\;\; 2x - 4y + 5z = 0$

is a one-dimensional subspace. More generally, that is to say that given a set of n independent functions, the dimension of the subspace in K^k will be the dimension of the null set of A, the composite matrix of the n functions.

Null space of a matrix

In a finite-dimensional space, a homogeneous system of linear equations can be written as a single matrix equation:

$A\mathbf{x} = \mathbf{0}.$

The set of solutions to this equation is known as the null space of the matrix. For example, the subspace described above is the null space of the matrix

$A = \left[ \begin{alignat}{3} 1 && 3 && 2 &\\ 2 && \;\;-4 && \;\;\;\;5 &\end{alignat} \,\right]\text{.}$

Every subspace of Kⁿ can be described as the null space of some matrix (see algorithms, below).

Linear parametric equations

The subset of Kⁿ described by a system of homogeneous linear parametric equations is a subspace:

$\left\{ \left[\!\! \begin{array}{c} x_1 \\ x_2 \\ \vdots \\ x_n \end{array} \!\!\right] \in K^n : \begin{alignat}{7} x_1 &&\; = \;&& a_{11} t_1 &&\; + \;&& a_{12} t_2 &&\; + \cdots + \;&& a_{1m} t_m & \\ x_2 &&\; = \;&& a_{21} t_1 &&\; + \;&& a_{22} t_2 &&\; + \cdots + \;&& a_{2m} t_m & \\ \vdots \,&& && \vdots\;\;\; && && \vdots\;\;\; && && \vdots\;\;\; & \\ x_n &&\; = \;&& a_{n1} t_1 &&\; + \;&& a_{n2} t_2 &&\; + \cdots + \;&& a_{nm} t_m & \\ \end{alignat} \text{ for some } t_1,\ldots,t_m\in K \right\}.$

For example, the set of all vectors (x, y, z) parameterized by the equations

$x = 2t_1 + 3t_2,\;\;\;\;y = 5t_1 - 4t_2,\;\;\;\;\text{and}\;\;\;\;z = -t_1 + 2t_2$

is a two-dimensional subspace of K³, if K is a number field (such as real or rational numbers).^[2]

Span of vectors

In linear algebra, the system of parametric equations can be written as a single vector equation:

$\begin{bmatrix} x& \\ y& \\ z& \end{bmatrix} \;=\; t_1 \!\begin{bmatrix} 2& \\ 5& \\ -1& \end{bmatrix} + t_2 \!\begin{bmatrix} 3& \\ -4& \\ 2& \end{bmatrix}.$

The expression on the right is called a linear combination of the vectors (2, 5, −1) and (3, −4, 2). These two vectors are said to span the resulting subspace.

In general, a linear combination of vectors v₁, v₂, ... , v_k is any vector of the form

$t_1 \mathbf{v}_1 + \cdots + t_k \mathbf{v}_k.$

The set of all possible linear combinations is called the span:

$\text{Span} \{ \mathbf{v}_1, \ldots, \mathbf{v}_k \} = \left\{ t_1 \mathbf{v}_1 + \cdots + t_k \mathbf{v}_k : t_1,\ldots,t_k\in K \right\} .$

If the vectors v₁, ... , v_k have n components, then their span is a subspace of Kⁿ. Geometrically, the span is the flat through the origin in n-dimensional space determined by the points v₁, ... , v_k.

Example

The xz-plane in R³ can be parameterized by the equations

$x = t_1, \;\;\; y = 0, \;\;\; z = t_2.$

As a subspace, the xz-plane is spanned by the vectors (1, 0, 0) and (0, 0, 1). Every vector in the xz-plane can be written as a linear combination of these two:

$(t_1, 0, t_2) = t_1(1,0,0) + t_2(0,0,1)\text{.}\,$

Geometrically, this corresponds to the fact that every point on the xz-plane can be reached from the origin by first moving some distance in the direction of (1, 0, 0) and then moving some distance in the direction of (0, 0, 1).

Column space and row space

A system of linear parametric equations in a finite-dimensional space can also be written as a single matrix equation:

$\mathbf{x} = A\mathbf{t}\;\;\;\;\text{where}\;\;\;\;A = \left[ \begin{alignat}{2} 2 && 3 & \\ 5 && \;\;-4 & \\ -1 && 2 & \end{alignat} \,\right]\text{.}$

In this case, the subspace consists of all possible values of the vector x. In linear algebra, this subspace is known as the column space (or image) of the matrix A. It is precisely the subspace of Kⁿ spanned by the column vectors of A.

The row space of a matrix is the subspace spanned by its row vectors. The row space is interesting because it is the orthogonal complement of the null space (see below).

Independence, basis, and dimension

The vectors u and v are a basis for this two-dimensional subspace of R³.

In general, a subspace of Kⁿ determined by k parameters (or spanned by k vectors) has dimension k. However, there are exceptions to this rule. For example, the subspace of K³ spanned by the three vectors (1, 0, 0), (0, 0, 1), and (2, 0, 3) is just the xz-plane, with each point on the plane described by infinitely many different values of t₁, t₂, t₃.

In general, vectors v₁, ... , v_k are called linearly independent if

$t_1 \mathbf{v}_1 + \cdots + t_k \mathbf{v}_k \;\ne\; u_1 \mathbf{v}_1 + \cdots + u_k \mathbf{v}_k$

for (t₁, t₂, ... , t_k) ≠ (u₁, u₂, ... , u_k).^[3] If v₁, ..., v_k are linearly independent, then the coordinates t₁, ..., t_k for a vector in the span are uniquely determined.

A basis for a subspace S is a set of linearly independent vectors whose span is S. The number of elements in a basis is always equal to the geometric dimension of the subspace. Any spanning set for a subspace can be changed into a basis by removing redundant vectors (see algorithms, below).

Example

Let S be the subspace of R⁴ defined by the equations

$x_1 = 2 x_2\;\;\;\;\text{and}\;\;\;\;x_3 = 5x_4.$

Then the vectors (2, 1, 0, 0) and (0, 0, 5, 1) are a basis for S. In particular, every vector that satisfies the above equations can be written uniquely as a linear combination of the two basis vectors:

$(2t_1, t_1, 5t_2, t_2) = t_1(2, 1, 0, 0) + t_2(0, 0, 5, 1).\,$

The subspace S is two-dimensional. Geometrically, it is the plane in R⁴ passing through the points (0, 0, 0, 0), (2, 1, 0, 0), and (0, 0, 5, 1).

Operations and relations on subspaces

Inclusion

The set-theoretical inclusion binary relation specifies a partial order on the set of all subspaces (of any dimension).

A subspace cannot lie in any subspace of lesser dimension. If dim U = k, a finite number, and U ⊂ W, then dim W = k if and only if U = W.

Intersection

In R³, the intersection of two-dimensional subspaces is one-dimensional

Given subspaces U and W of a vector space V, then their intersection U ∩ W := {v ∈ V : v is an element of both U and W} is also a subspace of V.^[4]

Proof:

1. Let v and w be elements of U ∩ W. Then v and w belong to both U and W. Because U is a subspace, then v + w belongs to U. Similarly, since W is a subspace, then v + w belongs to W. Thus, v + w belongs to U ∩ W.
2. Let v belong to U ∩ W, and let c be a scalar. Then v belongs to both U and W. Since U and W are subspaces, cv belongs to both U and W.
3. Since U and W are vector spaces, then 0 belongs to both sets. Thus, 0 belongs to U ∩ W.

For every vector space V, the set {0} and V itself are subspaces of V.^[5]

Sum

If U and W are subspaces, their sum is the subspace

$U + W = \left\{ \mathbf{u} + \mathbf{w} \colon \mathbf{u}\in U, \mathbf{w}\in W \right\}.$^[6]

For example, the sum of two lines is the plane that contains them both. The dimension of the sum satisfies the inequality

$\max(\dim U,\dim W) \leq \dim(U + W) \leq \dim(U) + \dim(W).$

Here the minimum only occurs if one subspace is contained in the other, while the maximum is the most general case. The dimension of the intersection and the sum are related:

$\dim(U+W) = \dim(U) + \dim(W) - \dim(U \cap W).$^[7]

Lattice of subspaces

The operations intersection and sum make the set of all subspaces a bounded modular lattice, where the {0} subspace, the least element, is an identity element of the sum operation, and the identical subspace V, the greatest element, is an identity element of the intersection operation.

Other

If V is an inner product space, then the orthogonal complement ⊥ of any subspace of V is again a subspace. This operation, understood as negation (¬), makes the lattice of subspaces a (possibly infinite) orthocomplemented lattice (it is not a distributive lattice).

In a pseudo-Euclidean space there are orthogonal complements too, but such operation does not form a Boolean algebra (nor a Heyting algebra) because of null subspaces, for which N ∩ N^⊥ = N ≠ {0}. The same case presents the ^⊥ operation in symplectic vector spaces.

Algorithms

Most algorithms for dealing with subspaces involve row reduction. This is the process of applying elementary row operations to a matrix until it reaches either row echelon form or reduced row echelon form. Row reduction has the following important properties:

1. The reduced matrix has the same null space as the original.
2. Row reduction does not change the span of the row vectors, i.e. the reduced matrix has the same row space as the original.
3. Row reduction does not affect the linear dependence of the column vectors.

Basis for a row space

Input An m × n matrix A.

Output A basis for the row space of A.

1. Use elementary row operations to put A into row echelon form.
2. The nonzero rows of the echelon form are a basis for the row space of A.

See the article on row space for an example.

If we instead put the matrix A into reduced row echelon form, then the resulting basis for the row space is uniquely determined. This provides an algorithm for checking whether two row spaces are equal and, by extension, whether two subspaces of Kⁿ are equal.

Subspace membership

Input A basis {b₁, b₂, ..., b_k} for a subspace S of Kⁿ, and a vector v with n components.

Output Determines whether v is an element of S

1. Create a (k + 1) × n matrix A whose rows are the vectors b₁, ... , b_k and v.
2. Use elementary row operations to put A into row echelon form.
3. If the echelon form has a row of zeroes, then the vectors {b₁, ..., b_k, v} are linearly dependent, and therefore v ∈ S .

Basis for a column space

Input An m × n matrix A

Output A basis for the column space of A

1. Use elementary row operations to put A into row echelon form.
2. Determine which columns of the echelon form have pivots. The corresponding columns of the original matrix are a basis for the column space.

See the article on column space for an example.

This produces a basis for the column space that is a subset of the original column vectors. It works because the columns with pivots are a basis for the column space of the echelon form, and row reduction does not change the linear dependence relationships between the columns.

Coordinates for a vector

Input A basis {b₁, b₂, ..., b_k} for a subspace S of Kⁿ, and a vector v ∈ S

Output Numbers t₁, t₂, ..., t_k such that v = t₁b₁ + ··· + t_kb_k

1. Create an augmented matrix A whose columns are b₁,...,b_k , with the last column being v.
2. Use elementary row operations to put A into reduced row echelon form.
3. Express the final column of the reduced echelon form as a linear combination of the first k columns. The coefficients used are the desired numbers t₁, t₂, ..., t_k. (These should be precisely the first k entries in the final column of the reduced echelon form.)

If the final column of the reduced row echelon form contains a pivot, then the input vector v does not lie in S.

Basis for a null space

Input An m × n matrix A.

Output A basis for the null space of A

1. Use elementary row operations to put A in reduced row echelon form.
2. Using the reduced row echelon form, determine which of the variables x₁, x₂, ..., x_n are free. Write equations for the dependent variables in terms of the free variables.
3. For each free variable x_i, choose a vector in the null space for which x_i = 1 and the remaining free variables are zero. The resulting collection of vectors is a basis for the null space of A.

See the article on null space for an example.

Basis for the sum and intersection of two subspaces

Given two subspaces U and W of V, a basis of the sum $U + W$ and the intersection $U \cap W$ can be calculated using the Zassenhaus algorithm

Equations for a subspace

Input A basis {b₁, b₂, ..., b_k} for a subspace S of Kⁿ

Output An (n − k) × n matrix whose null space is S.

1. Create a matrix A whose rows are b₁, b₂, ..., b_k.
2. Use elementary row operations to put A into reduced row echelon form.
3. Let c₁, c₂, ..., c_n be the columns of the reduced row echelon form. For each column without a pivot, write an equation expressing the column as a linear combination of the columns with pivots.
4. This results in a homogeneous system of n − k linear equations involving the variables c₁,...,c_n. The (n − k) × n matrix corresponding to this system is the desired matrix with nullspace S.

Example

If the reduced row echelon form of A is

$\left[ \begin{alignat}{6} 1 && 0 && -3 && 0 && 2 && 0 \\ 0 && 1 && 5 && 0 && -1 && 4 \\ 0 && 0 && 0 && 1 && 7 && -9 \\ 0 && \;\;\;\;\;0 && \;\;\;\;\;0 && \;\;\;\;\;0 && \;\;\;\;\;0 && \;\;\;\;\;0 \end{alignat} \,\right]$

then the column vectors c₁, ..., c₆ satisfy the equations

$\begin{alignat}{1} \mathbf{c}_3 &= -3\mathbf{c}_1 + 5\mathbf{c}_2 \\ \mathbf{c}_5 &= 2\mathbf{c}_1 - \mathbf{c}_2 + 7\mathbf{c}_4 \\ \mathbf{c}_6 &= 4\mathbf{c}_2 - 9\mathbf{c}_4 \end{alignat}$

It follows that the row vectors of A satisfy the equations

$\begin{alignat}{1} x_3 &= -3x_1 + 5x_2 \\ x_5 &= 2x_1 - x_2 + 7x_4 \\ x_6 &= 4x_2 - 9x_4. \end{alignat}$

In particular, the row vectors of A are a basis for the null space of the corresponding matrix.

See also

• Signal subspace
• Multilinear subspace learning
• Cyclic subspace

Textbooks

• Axler, Sheldon Jay (1997), Linear Algebra Done Right (2nd ed.), Springer-Verlag, ISBN 0-387-98259-0
• Lay, David C. (August 22, 2005), Linear Algebra and Its Applications (3rd ed.), Addison Wesley, ISBN 978-0-321-28713-7
• Meyer, Carl D. (February 15, 2001), Matrix Analysis and Applied Linear Algebra, Society for Industrial and Applied Mathematics (SIAM), ISBN 978-0-89871-454-8
• Poole, David (2006), Linear Algebra: A Modern Introduction (2nd ed.), Brooks/Cole, ISBN 0-534-99845-3
• Anton, Howard (2005), Elementary Linear Algebra (Applications Version) (9th ed.), Wiley International
• Leon, Steven J. (2006), Linear Algebra With Applications (7th ed.), Pearson Prentice Hall

External links

• Vector subspace at PlanetMath.org. .
• Gilbert Strang, MIT Linear Algebra Lecture on the Four Fundamental Subspaces at Google Video, from MIT OpenCourseWare

Notes

1. See Paul DuChateau. "Basic Facts About Hilbert Space" (pdf). Retrieved September 17, 2012. for Hilbert spaces
2. Generally, K can be any field of such characteristic that the given integer matrix has the appropriate rank in it. All fields include integers, but some integers may equal to zero in some fields.
3. This definition is often stated differently: vectors v₁, ..., v_k are linearly independent if t₁v₁ + ··· + t_kv_k ≠ 0 for (t₁, t₂, ..., t_k) ≠ (0, 0, ..., 0). The two definitions are equivalent.
4. Nering (1970, p. 21)
5. Nering (1970, p. 20)
6. Nering (1970, p. 21)
7. Nering (1970, p. 22)

References

• Nering, Evar D. (1970), Linear Algebra and Matrix Theory (2nd ed.), New York: Wiley, LCCN 76091646