Change of basis

A matrix is not a transformation but a description of one, written in a particular basis, and choosing a better basis can make the same transformation far easier to read.

Two facts from earlier lessons are needed here. A basis gives every vector a unique list of coordinates, and the standard basis has been used silently everywhere so far. And the matrix of a transformation is the list of the images of the basis vectors, so it depends on which basis those are. This lesson makes the dependence explicit and turns it into a tool.

Coordinates in another basis

Take the basis ${\mathbf{b}}_1=(1,1)$ and ${\mathbf{b}}_2=(-1,1)$ of ${\mathbb{R}}^2$, which is independent and therefore a basis, since neither vector is a multiple of the other. The arrow with standard coordinates $(3,1)$ has different coordinates here: solving $a(1,1)+b(-1,1)=(3,1)$ gives $a-b=3$ and $a+b=1$, so $a=2$ and $b=-1$. The same arrow is $(3,1)$ to one observer and $(2,-1)$ to the other, and nothing about the arrow changed.

The translation between the two is a matrix. Let $P=[(1,1),(-1,1)]$, whose columns are the new basis vectors written in standard coordinates. Then for any coordinate list $\mathbf{c}$ in the new basis, $P\mathbf{c}$ is a combination of the new basis vectors with weights $\mathbf{c}$, which is exactly the standard-coordinate version of the same arrow. So

and $P$ is invertible precisely because its columns form a basis. It is worth pausing on the direction, which everyone gets backwards once: the matrix whose columns are the new basis vectors converts from new coordinates to standard ones.

Here $\det P=(1)(1)-(-1)(1)=2$, so $P^{-1}=\frac{1}{2}[(1,-1),(1,1)]$ by the two by two formula. Applying it to $(3,1)$ gives $\frac{1}{2}(3+1,-3+1)=(2,-1)$, matching the direct solution.

Example. Find the coordinates of $(4,2)$ in the basis $(1,1)$, $(-1,1)$.

Apply $P^{-1}$: $\frac{1}{2}(4+2,-4+2)=(3,-1)$. Checking, $3(1,1)-1(-1,1)=(3+1,3-1)=(4,2)$.

Now you. Find the coordinates of $(0,6)$ in the same basis.

The matrix of a map in another basis

Now let $A$ be the matrix of a transformation in the standard basis, and ask for its matrix $B$ in the new one. The new matrix must do this: take new coordinates in, give new coordinates out.

There is no need to think geometrically about it. Break the job into three steps that are each already understood. Given new coordinates, first convert them to standard coordinates with $P$. Then apply $A$, which is the description that works in standard coordinates. Then convert the answer back to new coordinates with $P^{-1}$. Since matrices compose right to left,

Two matrices related this way are called similar, and the relation says they are the same transformation seen by two observers using different rulers. Every geometric fact about the transformation is shared between them, and only the numbers differ.

Take $A=[(2,1),(1,2)]$, whose action is not obvious by inspection: it sends $(1,0)$ to $(2,1)$ and $(0,1)$ to $(1,2)$, tilting both basis vectors inward. Compute $B$ with the $P$ above. First $AP$, whose columns are $A$ applied to the columns of $P$: $A(1,1)=1(2,1)+1(1,2)=(3,3)$, and $A(-1,1)=-1(2,1)+1(1,2)=(-1,1)$. Then apply $P^{-1}$ to each: $P^{-1}(3,3)=\frac{1}{2}(3+3,-3+3)=(3,0)$, and $P^{-1}(-1,1)=\frac{1}{2}(-1+1,1+1)=(0,1)$. So

which is diagonal. In the new basis the transformation is trivial to describe: it triples everything along $(1,1)$ and leaves everything along $(-1,1)$ alone. That statement is a complete description of a map that looked like four unrelated numbers a moment ago, and it is checkable directly: $A(1,1)=(3,3)$, three times $(1,1)$, and $A(-1,1)=(-1,1)$, unchanged.

The basis was not chosen at random, and how to find such a basis is the subject of the next lesson. The point here is what a good basis buys.

Example. Find the matrix of the reflection $F=[(0,1),(1,0)]$, which swaps the axes, in the basis $(1,1)$, $(-1,1)$.

$FP$ has columns $F(1,1)=(1,1)$ and $F(-1,1)=(1,-1)$. Applying $P^{-1}$: $P^{-1}(1,1)=\frac{1}{2}(1+1,-1+1)=(1,0)$ and $P^{-1}(1,-1)=\frac{1}{2}(1-1,-1-1)=(0,-1)$. So $B=[(1,0),(0,-1)]$, again diagonal. It says the reflection fixes the direction $(1,1)$, which is the mirror line itself, and negates the direction $(-1,1)$, which is perpendicular to the mirror. That is what a reflection does, stated in one line.

Now you. What is the matrix, in the standard basis, of the map that triples every vector along $(1,1)$ and leaves every vector along $(-1,1)$ alone?

Building a transformation the easy way

That last exercise is a method, and it is how awkward transformations are constructed in practice: describe the map in the basis where it is obvious, then conjugate back.

Find the matrix that reflects the plane in the line through $(2,1)$. Doing this directly requires trigonometry. Doing it by change of basis requires none. Take ${\mathbf{b}}_1=(2,1)$ along the mirror line and ${\mathbf{b}}_2=(-1,2)$ perpendicular to it, since $(2)(-1)+(1)(2)=0$. In that basis the reflection fixes the first direction and negates the second, so its matrix is $D=[(1,0),(0,-1)]$.

Now convert. $P=[(2,1),(-1,2)]$ with $\det P=4+1=5$, so $P^{-1}=\frac{1}{5}[(2,-1),(1,2)]$. Then $D{P}^{-1}=\frac{1}{5}[(2,1),(1,-2)]$, negating the second component of each column, and multiplying on the left by $P$ gives columns $\frac{1}{5}[2(2,1)+1(-1,2)]=\frac{1}{5}(3,4)$ and $\frac{1}{5}[1(2,1)-2(-1,2)]=\frac{1}{5}(4,-3)$. So

Check it against the formula derived in the lesson on transformations, $[(\cos 2\theta ,\sin 2\theta ),(\sin 2\theta ,-\cos 2\theta )]$ for a mirror at angle $\theta$. Here $\tan \theta =1/2$, and the double angle identities give $\cos 2\theta =(1-{\tan }^2\theta )/(1+{\tan }^2\theta )=0.75/1.25=0.6$ and $\sin 2\theta =2\tan \theta /(1+{\tan }^2\theta )=1/1.25=0.8$. The two routes agree exactly. Direct verification is easier still: $A(2,1)=2(0.6,0.8)+1(0.8,-0.6)=(2,1)$, the mirror line fixed, and $A(-1,2)=-1(0.6,0.8)+2(0.8,-0.6)=(1,-2)$, the perpendicular reversed.

Between two bases, neither of them standard

The standard basis has played the role of a hub so far: every conversion went through it. Nothing requires that, and the general case is a composition of two conversions already understood.

Let $U$ and $V$ be two bases, with $P_U$ and $P_V$ the matrices whose columns are their vectors in standard coordinates. To convert $U$ coordinates into $V$ coordinates, first go up to standard coordinates with $P_U$, then down into $V$ coordinates with $P_V^{-1}$. So the conversion matrix is $P_V^{-1}{P}_U$, and the same three-step reading gives the matrix of a transformation in $V$ from its matrix in $U$, by conjugating with it.

Take $U=\{(1,1),(-1,1)\}$ and $V=\{(2,1),(-1,2)\}$, both bases since neither pair is parallel. Then $P_V=[(2,1),(-1,2)]$ has determinant $5$, so $P_V^{-1}=\frac{1}{5}[(2,-1),(1,2)]$, and the conversion matrix has columns $P_V^{-1}(1,1)=\frac{1}{5}(3,1)=(0.6,0.2)$ and $P_V^{-1}(-1,1)=\frac{1}{5}(-1,3)=(-0.2,0.6)$.

Test it. The arrow with $U$ coordinates $(2,-1)$ is $2(1,1)-1(-1,1)=(3,1)$ in standard coordinates. Applying the conversion matrix to $(2,-1)$ gives $2(0.6,0.2)-1(-0.2,0.6)=(1.4,-0.2)$, and checking in $V$: $1.4(2,1)-0.2(-1,2)=(2.8+0.2,1.4-0.4)=(3,1)$. The same arrow, described three ways.

Example. Using the conversion matrix above, what are the $V$ coordinates of the arrow whose $U$ coordinates are $(1,1)$?

Apply it: $1(0.6,0.2)+1(-0.2,0.6)=(0.4,0.8)$. Checking, the arrow is $1(1,1)+1(-1,1)=(0,2)$ in standard coordinates, and $0.4(2,1)+0.8(-1,2)=(0.8-0.8,0.4+1.6)=(0,2)$.

Now you. Which matrix converts $V$ coordinates back into $U$ coordinates?

What survives a change of basis

Similar matrices describe the same map, so any quantity with a genuine geometric meaning must be the same for both. Two are easy to establish.

The determinant survives: $\det (P^{-1}AP)=\det (P^{-1})\det (A)\det (P)=\det (A)$, using the multiplication rule and $\det (P^{-1})=1/\det (P)$. This had to happen, since the determinant is a ratio of areas and a ratio does not care what units the areas are measured in. Above, $\det A=4-1=3$ and $\det B=3\times 1=3$.

The trace, the sum of the diagonal entries, also survives. It follows from the fact that $\text{tr}(XY)=\text{tr}(YX)$ for any two matrices, which is a short computation with the entry formula, since both sides sum $x_{ij}{y}_{ji}$ over all $i$ and $j$. Then $\text{tr}(P^{-1}AP)=\text{tr}(AP{P}^{-1})=\text{tr}(A)$. Above, $\text{tr}A=2+2=4$ and $\text{tr}B=3+1=4$. Unlike the determinant, the trace has no obvious geometric reading yet; the next lesson supplies one.

Rank survives too, being the dimension of the image, and so does invertibility. What does not survive is anything to do with the entries themselves: whether a matrix is diagonal, whether it is symmetric, how large its entries are. Those are properties of a description.

Example. Are $[(2,1),(1,2)]$ and $[(3,0),(0,2)]$ similar?

Compare invariants. The traces are $4$ and $5$, which already differ, so no similarity is possible. There is no need to search for a $P$.

Now you. Are $[(1,0),(1,1)]$ and $[(1,0),(0,1)]$ similar? Both have trace $2$ and determinant $1$.

Honest limits

A change of basis requires $P$ to be invertible, so its columns must be a basis. Nothing else is required of them: they need not be perpendicular and need not have length one. When they are orthonormal, $P^{-1}=P^T$ and the conversion is free, which is the practical reason orthonormal bases are worth building.

Numerically, a nearly dependent basis makes $P$ badly conditioned, and errors in the entries of $A$ are amplified by roughly the condition number of $P$ when $P^{-1}AP$ is formed. Two matrices can therefore be similar in exact arithmetic and behave very differently in floating point, which is why numerical algorithms restrict themselves to orthogonal $P$ wherever they can.

The important limit is the one this lesson has been circling. Twice now a well-chosen basis turned a matrix diagonal, which reduced a transformation to a list of stretch factors along a list of directions. Nothing so far says which matrices allow this, how to find the directions, or whether they exist at all. Since a diagonal matrix in the new basis means each new basis vector is simply scaled by the map, the directions being sought are the ones the transformation leaves on their own line. Those are the eigenvectors, and finding them is the next lesson.