Determinants and inverses

A transformation can be reversed only if nothing was lost on the way, and one number computed from a square matrix says whether anything was.

The previous lesson composed transformations and found that the projection $[(1, 0), (0, 0)]$ flattens the plane onto a line, after which no transformation can recover which height a point came from. This lesson turns that observation into a test, and the test has a geometric meaning: how much the transformation multiplies area.

What area does under a transformation

Start with the unit square, whose corners are $0$ , $e_{1}$ , $e_{2}$ and $e_{1} + e_{2}$ , and whose area is one. A linear map sends $e_{1}$ and $e_{2}$ to the columns of its matrix, and sends the fourth corner to the sum of those columns, since it preserves addition. So the unit square becomes the parallelogram spanned by the two columns.

Every other region follows, because a linear map treats the whole plane uniformly. Any shape can be approximated by small squares from a fine grid, each square becomes a small parallelogram of the same shape and size as every other, and areas add. So a single number, the area of the image of the unit square, gives the factor by which every area is multiplied. That number is the determinant.

Compute it for the matrix with columns $u = (a, c)$ and $v = (b, d)$ , taking all four entries positive for the picture. Enclose the parallelogram in the rectangle from $0$ to $(a + b, c + d)$ , of area $(a + b) (c + d)$ , and remove what is not the parallelogram: two triangles of area $\frac{1}{2} a c$ , two triangles of area $\frac{1}{2} b d$ , and two rectangles of area $b c$ . That leaves

(a + b) (c + d) - a c - b d - 2 b c = a d - b c

after cancelling. So the determinant of a two by two matrix is

\det A = a d - b c

where, in the column notation of this course, $A = [(a, c), (b, d)]$ . For $A = [(3, 1), (1, 2)]$ that is $6 - 1 = 5$ : the unit square becomes a parallelogram of area five, and a circle of area $π$ becomes an ellipse of area $5 π$ .

Check the catalogue from two lessons ago against this. A shear $[(1, 0), (k, 1)]$ has determinant $1$ , and a shear does preserve area, since it slides layers sideways without changing their height or width. A rotation $[(\cos θ, \sin θ), (- \sin θ, \cos θ)]$ has determinant $\cos^{2} θ + \sin^{2} θ = 1$ , as a rigid motion must. A scaling $[(a, 0), (0, b)]$ has determinant $a b$ , which is the area of the rectangle it makes. And the projection $[(1, 0), (0, 0)]$ has determinant $0$ , because the image parallelogram is flat.

Example. Find the determinant of $[(4, 2), (1, 3)]$ and say what it means.

Here $a = 4$ , $c = 2$ , $b = 1$ , $d = 3$ , so $\det = (4) (3) - (1) (2) = 10$ . The transformation multiplies every area by ten. A triangle of area $0.5$ becomes one of area $5$ .

Now you. Find the determinant of $[(2, 3), (4, 6)]$ .

Answer

$\det = (2) (6) - (4) (3) = 12 - 12 = 0$ . The second column is exactly twice the first, so both basis vectors land on the same line and the unit square is flattened onto it, with no area left.

The sign

The formula $a d - b c$ can come out negative, and area cannot, so the determinant is a signed area. The sign records orientation.

Going anticlockwise, $e_{1}$ comes before $e_{2}$ . If the images keep that arrangement, the determinant is positive; if the images are swapped in sense, so that the shorter turn from the first column to the second is now clockwise, the determinant is negative and the map has flipped the plane over. Reflection in the line $y = x$ has matrix $[(0, 1), (1, 0)]$ and determinant $0 \cdot 0 - 1 \cdot 1 = - 1$ : area is preserved in size, as reflection should, and the sign reports the flip. Rotations, which do not flip, have determinant $+ 1$ .

Orientation is not a curiosity. It is the difference between a left hand and a right hand, between a mould and the object it casts, and in three dimensions it is what distinguishes the two mirror forms of a chiral molecule.

The multiplication rule

Because the determinant is a scale factor, and scale factors multiply when transformations are composed, the following holds for all square $A$ and $B$ of the same size:

\det (A B) = \det (A) \det (B)

Doing $B$ then $A$ multiplies area by $\det B$ and then by $\det A$ , so the total factor is the product, and the signs behave too, since two flips restore the original orientation. Proving this from the entry formula is an unpleasant expansion; proving it from what the determinant means is one sentence.

Check it once with numbers. With $A = [(3, 1), (1, 2)]$ , determinant $5$ , and $B = [(2, 0), (1, 4)]$ , determinant $8$ , the product $A B$ has columns $A (2, 0) = (6, 2)$ and $A (1, 4) = (7, 9)$ , so $\det (A B) = (6) (9) - (7) (2) = 54 - 14 = 40 = 5 \times 8$ .

Two consequences follow at once. If $A$ has an inverse then $\det (A) \det (A^{- 1}) = \det (I) = 1$ , so $\det (A^{- 1}) = 1 / \det (A)$ , and in particular an invertible matrix cannot have determinant zero. And if either factor has determinant zero then so does the product, which is the algebraic shadow of the fact that once something has been flattened, nothing later can unflatten it.

Zero means collapse

Everything now points at one criterion. The determinant is zero exactly when the image of the unit square has no area, which happens exactly when the two columns lie on a single line, which is exactly when the columns are linearly dependent. Chaining that to the previous lessons gives a list of statements about a square matrix $A$ that are all true together or all false together: the determinant is non-zero; the columns are independent; the columns span the whole space; $A x = b$ has exactly one solution for every $b$ ; $A x = 0$ only for $x = 0$ ; the transformation is reversible; the matrix has an inverse.

A matrix satisfying them is called invertible or non-singular, and one failing them singular. The word chosen by the third lesson, dependent, and the word chosen by this one, singular, are the same condition seen from two directions.

The failure is worth picturing rather than filing. When $\det A = 0$ the map squashes the plane onto a line or onto the origin, so many different inputs share an output, and no rule can send that output back to the right input, because the information identifying it is gone. Non-invertibility is not a defect of technique.

Three dimensions and beyond

In $R^{3}$ the same definition applies to volume: the unit cube becomes the parallelepiped spanned by the three columns, and the determinant is its signed volume. The diagonal matrix $[(1, 0, 0), (0, 2, 0), (0, 0, 3)]$ stretches by $1$ , $2$ and $3$ along the axes, giving a box of volume $6$ , which is its determinant.

For a general three by three the value is computed by cofactor expansion, which peels off one row at a time. Writing the rows as $(a_{11}, a_{12}, a_{13})$ and so on,

\det A = a_{11} (a_{22} a_{33} - a_{23} a_{32}) - a_{12} (a_{21} a_{33} - a_{23} a_{31}) + a_{13} (a_{21} a_{32} - a_{22} a_{31})

Each bracket is the two by two determinant left after deleting the row and column of the leading entry, and the signs alternate. The pattern continues in higher dimensions, an $n$ by $n$ determinant being a combination of $n$ determinants of size $n - 1$ .

Take the coefficient matrix of the very first lesson's system, whose rows are $(2, 1, - 1)$ , $(- 3, - 1, 2)$ and $(- 2, 1, 2)$ . Expanding along the first row: $2 [(- 1) (2) - (2) (1)] - 1 [(- 3) (2) - (2) (- 2)] + (- 1) [(- 3) (1) - (- 1) (- 2)]$ , which is $2 (- 4) - 1 (- 2) - 1 (- 5) = - 8 + 2 + 5 = - 1$ . Non-zero, so that system had exactly one solution, which is what elimination found.

The recursive method is a trap at scale: it costs about $n!$ operations, which the first lesson already priced at seventy-seven years for $n = 20$ . The practical route is elimination, since adding a multiple of one row to another leaves the determinant unchanged, swapping two rows flips its sign, and the determinant of a triangular matrix is the product of its diagonal entries. Reducing to triangular form therefore delivers the determinant for the usual $n^{3} / 3$ operations.

Example. Compute the determinant of the matrix with rows $(1, 2, 3)$ , $(0, 1, 4)$ and $(5, 6, 0)$ .

Expanding along the first row: $1 [(1) (0) - (4) (6)] - 2 [(0) (0) - (4) (5)] + 3 [(0) (6) - (1) (5)]$ , which is $1 (- 24) - 2 (- 20) + 3 (- 5) = - 24 + 40 - 15 = 1$ . The matrix is invertible, and it scales volume by a factor of one while, as it happens, distorting shape considerably.

Now you. Compute the determinant of the matrix with rows $(2, 0, 1)$ , $(1, 3, 2)$ and $(0, 1, 1)$ .

Answer

$2 [(3) (1) - (2) (1)] - 0 [(1) (1) - (2) (0)] + 1 [(1) (1) - (3) (0)] = 2 (1) - 0 + 1 (1) = 3$ . Non-zero, so the matrix is invertible and it triples volume.

The inverse

The inverse of a square matrix $A$ is the matrix $A^{- 1}$ with $A A^{- 1} = A^{- 1} A = I$ , and it represents the transformation that undoes $A$ . For the two by two case it can be derived rather than quoted.

Its first column is the vector $x$ with $A x = e_{1}$ , since the inverse must send $e_{1}$ back where it came from. With $A = [(a, c), (b, d)]$ that is the system $a x_{1} + b x_{2} = 1$ and $c x_{1} + d x_{2} = 0$ . Eliminating gives $x_{1} = d / (a d - b c)$ and $x_{2} = - c / (a d - b c)$ . The second column solves $A y = e_{2}$ , giving $y_{1} = - b / (a d - b c)$ and $y_{2} = a / (a d - b c)$ . So

A^{- 1} = \frac{1}{a d - b c} [(d, - c), (- b, a)]

The determinant appears in the denominator, which is the algebra refusing to divide by zero exactly when the geometry says the map cannot be undone.

For $A = [(3, 1), (1, 2)]$ , with $a = 3$ , $c = 1$ , $b = 1$ , $d = 2$ and determinant $5$ , the inverse is $\frac{1}{5} [(2, - 1), (- 1, 3)]$ . Checking: the first column of $A A^{- 1}$ is $A (0.4, - 0.2) = 0.4 (3, 1) - 0.2 (1, 2) = (1.2 - 0.2, 0.4 - 0.4) = (1, 0)$ , as required.

Beyond two by two there is no formula worth using, and the method is Gauss-Jordan elimination: write the matrix beside the identity, and apply row operations until the left side is the identity, at which point the right side is the inverse. Each operation is applied to both halves, so whatever sequence of operations turns $A$ into $I$ turns $I$ into $A^{- 1}$ . For the matrix with rows $(1, 2, 3)$ , $(0, 1, 4)$ and $(5, 6, 0)$ the result is the matrix with rows $(- 24, 18, 5)$ , $(20, - 15, - 4)$ and $(- 5, 4, 1)$ , and multiplying the two gives the identity exactly, the determinant being $1$ so that no fractions appear.

Inverses reverse under products, $(A B)^{- 1} = B^{- 1} A^{- 1}$ , which is only common sense: to undo putting on socks and then shoes, take off the shoes first. The proof is one line, $(A B) (B^{- 1} A^{- 1}) = A (B B^{- 1}) A^{- 1} = A A^{- 1} = I$ , using associativity.

Example. Invert $A = [(3, 1), (1, 2)]$ and use it to solve $A x = (5, 5)$ .

The inverse is $\frac{1}{5} [(2, - 1), (- 1, 3)]$ from above. Then $x = A^{- 1} (5, 5) = \frac{1}{5} [5 (2, - 1) + 5 (- 1, 3)] = \frac{1}{5} (10 - 5, - 5 + 15) = \frac{1}{5} (5, 10) = (1, 2)$ . Check against the original: $A (1, 2) = 1 (3, 1) + 2 (1, 2) = (5, 5)$ .

Now you. Invert $B = [(2, 1), (5, 3)]$ .

Answer

Here $a = 2$ , $c = 1$ , $b = 5$ , $d = 3$ , so $\det B = 6 - 5 = 1$ and $B^{- 1} = [(3, - 1), (- 5, 2)]$ . Check the first column of $B B^{- 1}$ : $B (3, - 1) = 3 (2, 1) - 1 (5, 3) = (6 - 5, 3 - 3) = (1, 0)$ .

Honest limits

Three warnings, all of which are routinely ignored to the cost of whoever ignores them.

Do not solve a system by computing an inverse. Gauss-Jordan costs roughly three times what plain elimination costs, and multiplying the inverse by $b$ afterwards adds more error than solving directly. The expression $x = A^{- 1} b$ is a statement about what the solution is, not an instruction for finding it, and numerical libraries that appear to be inverting are usually factorising instead.

Do not use the determinant as a test for singularity in floating point. Scale a perfectly well-behaved $100$ by $100$ matrix by $0.1$ and its determinant is multiplied by ${0.1}^{100} = 10^{- 100}$ , which underflows to zero on a computer while the matrix remains as invertible as it ever was. Nearness to singularity is measured by the condition number instead, which compares the largest stretching factor with the smallest, and which the lesson on orthogonality returns to.

Do not expect Cramer's rule to be useful. It gives each unknown as a ratio of determinants, which is elegant and, computed naively, costs $n!$ per unknown. It is a proof device, not an algorithm.

Two lessons ago a projection destroyed information, and this lesson gave the destruction a number: the determinant is zero. What it did not say is what exactly gets destroyed, or how much. A matrix can collapse three dimensions to two, or to one, or to a point, and all four cases have determinant zero. Measuring the survivors is the business of the next lesson.