Subspaces, basis and dimension

A dependent set of vectors describes its span wastefully, and removing the waste leaves something whose size turns out not to depend on how the removing was done.

The previous lesson defined the span of a set of vectors as everything reachable from it by adding and scaling, and defined independence as the absence of redundancy. This lesson names the objects those spans are, gives each one a minimal description, and proves that any two minimal descriptions have the same length.

Subspaces

A subspace of $R^{n}$ is a non-empty set $V$ of vectors that is closed under both operations: if $u$ and $v$ are in $V$ then so is $u + v$ , and if $v$ is in $V$ then so is $c v$ for every scalar $c$ . Those two conditions are all of it, and they immediately force a third: taking $c = 0$ shows every subspace contains $0$ .

The point of the definition is that a subspace is a place where the whole of linear algebra still works. You cannot fall out of it by doing arithmetic, so it can be treated as a small vector space in its own right.

Some sets qualify and some do not, and the failures are more instructive. The line through the origin in the direction $(1, 2)$ is a subspace: multiples of $(1, 2)$ add to multiples of $(1, 2)$ . The line $y = 2 x + 1$ is not, since it misses the origin, and also because $(0, 1)$ and $(1, 3)$ are on it while their sum $(1, 4)$ is not. The first quadrant, all $(x, y)$ with $x \geq 0$ and $y \geq 0$ , is closed under addition but not under scaling, since $- 1$ times $(1, 1)$ leaves it, and that single failure disqualifies it. The set of vectors of length at most one fails for the same reason: doubling escapes it. Curved sets and bounded sets are never subspaces, and neither is anything that avoids the origin.

Two families of subspaces matter, and they will turn out to be the same family. First, every span is a subspace, directly from the definition, since a combination of combinations is a combination. Second, the solution set of a homogeneous system, one whose right hand side is $0$ , is a subspace: if $u$ and $v$ both satisfy every equation with zero on the right, so does $u + v$ and so does $c u$ , because each equation is built from adding and scaling. That is why homogeneous systems behave so much better than inhomogeneous ones, and why the solution set of $x + 2 y - z = 0$ is a plane through the origin while the solution set of $x + 2 y - z = 4$ is a parallel plane that is not a subspace at all.

Example. Is the set of $(x, y, z)$ with $x + 2 y - z = 0$ a subspace of $R^{3}$ ?

Yes. If $(x_{1}, y_{1}, z_{1})$ and $(x_{2}, y_{2}, z_{2})$ both satisfy it, then adding the two equations shows the componentwise sum satisfies it too, and multiplying one equation by $c$ shows $c$ times a solution is a solution. The set is the solution set of a homogeneous equation, so it is a plane through the origin.

Now you. Is the set of $(x, y)$ with $x y = 0$ , that is, the two coordinate axes together, a subspace of $R^{2}$ ?

Answer

No. It is closed under scaling, since scaling a point on an axis keeps it on that axis, but not under addition: $(1, 0)$ and $(0, 1)$ are both in the set while $(1, 1)$ is not, because $1 \times 1 \neq 0$ . Closure under both operations is required, and a set can pass one test and fail the other.

Basis

A basis of a subspace $V$ is a set of vectors that is independent and spans $V$ . The two requirements pull in opposite directions and meet in the middle: spanning demands enough vectors to reach everything, independence forbids any that are not needed. A basis is therefore a description of $V$ with nothing missing and nothing wasted.

The standard basis of $R^{n}$ is $e_{1} = (1, 0, \dots, 0)$ , $e_{2} = (0, 1, 0, \dots, 0)$ and so on. It spans because $(a_{1}, \dots, a_{n}) = a_{1} e_{1} + \dots + a_{n} e_{n}$ , and it is independent because that same combination is zero only when every $a_{i}$ is zero. It is the basis that ordinary coordinates use, and its dominance is a convention rather than a fact about space, which is exactly the point of a later lesson.

Other bases of the same space are everywhere. In $R^{2}$ , the pair $(1, 1)$ and $(1, - 1)$ is a basis: independent, since neither is a multiple of the other, and spanning, since any $(a, b)$ is $\frac{a + b}{2} (1, 1) + \frac{a - b}{2} (1, - 1)$ . So is $(2, 1)$ and $(1, 3)$ . There are infinitely many bases of the plane, and no reason internal to the mathematics to prefer one.

The property that makes a basis worth having is uniqueness of coordinates. If $v_{1}, \dots, v_{k}$ is a basis of $V$ , every vector in $V$ can be written as a combination of them in exactly one way. Existence is the spanning half. Uniqueness comes from independence: suppose $w = a_{1} v_{1} + \dots + a_{k} v_{k}$ and also $w = b_{1} v_{1} + \dots + b_{k} v_{k}$ . Subtracting gives $(a_{1} - b_{1}) v_{1} + \dots + (a_{k} - b_{k}) v_{k} = 0$ , and independence forces every coefficient to vanish, so $a_{i} = b_{i}$ for each $i$ . The two representations were the same one.

Those unique numbers are the coordinates of $w$ with respect to the basis. Ordinary coordinates are the special case where the basis is the standard one, and the familiar statement that a point in the plane is a pair of numbers is really the statement that the plane has a basis of size two.

Example. Find a basis for the plane $x + 2 y - z = 0$ and give its dimension.

Solve the equation: $x$ is determined once $y$ and $z$ are chosen, since $x = - 2 y + z$ . Taking $y = 1$ , $z = 0$ gives $(- 2, 1, 0)$ , and taking $y = 0$ , $z = 1$ gives $(1, 0, 1)$ . Both satisfy the equation: $- 2 + 2 - 0 = 0$ and $1 + 0 - 1 = 0$ . Every solution is $y (- 2, 1, 0) + z (1, 0, 1)$ , so they span, and neither is a multiple of the other, so they are independent. The plane has a basis of two vectors.

Now you. Find a basis for the set of $(x, y, z)$ in which all three components are equal.

Answer

Such a vector is $(t, t, t) = t (1, 1, 1)$ , so the single vector $(1, 1, 1)$ spans the set, and one non-zero vector is always independent. The basis is ${(1, 1, 1)}$ and the set is a line through the origin.

Dimension is well defined

Both bases of the plane given above have two elements, and both bases in the examples had the size the geometry suggested. That is not luck, and it needs proof, because a basis is a choice and there are infinitely many choices available.

Claim. If a subspace $V$ is spanned by $n$ vectors, then any $n + 1$ vectors in $V$ are dependent.

The proof reuses the counting argument from the previous lesson. Let $s_{1}, \dots, s_{n}$ span $V$ , and take any $w_{1}, \dots, w_{n + 1}$ in $V$ . Each $w_{j}$ is a combination of the $s_{i}$ , say with coefficients $a_{1 j}, \dots, a_{n j}$ . Now ask for a dependence, $c_{1} w_{1} + \dots + c_{n + 1} w_{n + 1} = 0$ . Substituting and collecting the terms belonging to each $s_{i}$ , the combination equals $0$ whenever every one of the $n$ quantities $a_{i 1} c_{1} + \dots + a_{i, n + 1} c_{n + 1}$ is zero. That is a homogeneous system of $n$ equations in $n + 1$ unknowns. Elimination pins down at most $n$ unknowns, so at least one is free, and choosing it non-zero gives a non-trivial solution. The $w_{j}$ are dependent.

The consequence is immediate. If $B_{1}$ and $B_{2}$ are both bases of $V$ , with $n$ and $m$ elements, then $B_{1}$ spans and $B_{2}$ is independent, so $m \leq n$ by the claim, and reversing the roles gives $n \leq m$ . Hence $n = m$ : every basis of $V$ has the same number of elements, and that number is the dimension of $V$ , written $\dim V$ .

Dimension is now a fact about the space rather than about anyone's description of it. A line through the origin has dimension one, a plane through the origin has dimension two, $R^{n}$ has dimension $n$ because the standard basis has $n$ elements, and the subspace ${0}$ has dimension zero, its basis being the empty set.

Spaces that are not arrows

Nothing in the definitions of subspace, basis and dimension mentioned arrows or coordinates. They used addition, scalar multiplication and the eight rules, so they apply verbatim to every vector space, and the useful ones are frequently not sets of arrows at all.

The polynomials of degree at most five form a vector space: adding two of them gives another, scaling one gives another, and the zero polynomial is the zero vector. A basis is $1, x, x^{2}, x^{3}, x^{4}, x^{5}$ , since every such polynomial is a unique combination of these, so the dimension is six, one more than the degree. The coordinates of $3 x^{2} - 7$ in that basis are $(- 7, 0, 3, 0, 0, 0)$ , which is exactly the list of coefficients: a polynomial is a vector whose components happen to be written next to powers of $x$ .

The two by two matrices form a four-dimensional space, with a basis of the four matrices having a single entry equal to one. This is worth noticing because it means matrices are themselves vectors, and any statement proved about spans and independence applies to a collection of matrices.

The solutions of a differential equation like $y^{''} + y = 0$ form a vector space, since a sum of solutions solves it and so does a multiple. A basis is $\cos x$ and $\sin x$ , so the dimension is two, and that is the real reason a second order equation carries exactly two arbitrary constants: the solution set is a two-dimensional subspace of the space of functions, and the constants are coordinates in it. The same idea explains the phrase "general solution" in every differential equations course.

Example. What is the dimension of the space of polynomials of degree at most three that vanish at $x = 0$ ?

The full space has dimension four, with basis $1, x, x^{2}, x^{3}$ . Vanishing at zero means the constant term is zero, which removes exactly one basis element, leaving $x, x^{2}, x^{3}$ . Those three are independent and span the set, so the dimension is three. Note the pattern: one linear condition removed one dimension, exactly as one equation cut $R^{3}$ down to a plane.

Now you. Is the set of polynomials of degree at most three with $p (1) = 0$ a subspace, and if so what is its dimension?

Answer

It is a subspace: if $p (1) = 0$ and $q (1) = 0$ then $(p + q) (1) = 0$ and $(c p) (1) = 0$ . The condition $p (1) = 0$ is one linear equation on the four coefficients, so it removes one dimension, leaving three. A basis is $x - 1$ , $x^{2} - 1$ and $x^{3} - 1$ , each of which vanishes at $x = 1$ .

Trimming and extending

Two workhorse facts follow, and both are proved by the same move.

Every spanning set contains a basis. If a spanning set is dependent, one of its vectors is a combination of the others, so deleting it leaves the span unchanged. Repeat. The set is finite, so the process stops, and it stops exactly when no vector is redundant, which is independence.

Every independent set in $V$ can be extended to a basis of $V$ . If an independent set does not yet span $V$ , some $w$ in $V$ is out of reach, and adding it keeps the set independent, since a dependence involving $w$ would express $w$ in terms of the others and put it in reach. Repeat. By the claim above the set can never exceed $\dim V$ elements, so the process stops, and it stops exactly when the set spans.

Together these give a useful shortcut: in a space of dimension $d$ , any $d$ independent vectors automatically span, and any $d$ vectors that span are automatically independent. Once the count is right, one of the two conditions is enough to check.

Example. Express $(2, - 5, 3)$ in the basis $(1, - 1, 0)$ , $(1, 0, - 1)$ of the plane $x + y + z = 0$ .

First check the vector belongs there: $2 - 5 + 3 = 0$ , so it does. Solve $a (1, - 1, 0) + b (1, 0, - 1) = (2, - 5, 3)$ . The second component gives $- a = - 5$ , so $a = 5$ ; the third gives $- b = 3$ , so $b = - 3$ ; and the first is then the check, $5 - 3 = 2$ . The coordinates are $(5, - 3)$ , so a vector with three ordinary components has just two coordinates in this basis, which is the dimension of the plane it lives in.

Now you. Express $(7, 4)$ in the basis $(1, 1)$ , $(1, - 1)$ of $R^{2}$ .

Answer

Solving $a + b = 7$ and $a - b = 4$ gives $a = 5.5$ and $b = 1.5$ . Check: $5.5 (1, 1) + 1.5 (1, - 1) = (5.5 + 1.5, 5.5 - 1.5) = (7, 4)$ . The same arrow that has standard coordinates $(7, 4)$ has coordinates $(5.5, 1.5)$ here, and nothing about the arrow changed.

Honest limits

Coordinates depend entirely on the basis, and a vector does not carry its coordinates around with it. The last example is the whole warning: $(7, 4)$ and $(5.5, 1.5)$ are the same arrow described by two people using different rulers. Software and textbooks alike suppress the basis because it is almost always the standard one, and the suppression becomes a bug the moment two bases are in play, which is what the lesson on change of basis is about.

The theory above assumes a finite spanning set exists. Spaces without one, such as all continuous functions on an interval, are infinite-dimensional, and there the word basis is used in a weaker sense involving convergent infinite sums, which needs analysis rather than algebra. Fourier series are that theory in action, and nothing in this course applies to them without care.

Finally, dimension is exact and discontinuous. Three vectors that lie in a plane span a two-dimensional space; nudge one of them by $10^{- 9}$ out of the plane and the span is suddenly all of $R^{3}$ , though it is a very thin sort of three dimensions. Numerical work replaces dimension with a threshold on how thin, which the lesson on rank returns to.

There are objects, there are coordinates, and there is a well-defined notion of size. What is entirely missing is anything that moves: so far every vector has sat still. The next lesson introduces maps that respect both operations, and finds that such a map is completely pinned down by what it does to a basis.