Tangent planes and linear approximation

Both partial derivatives of a function can exist at a point where the function is not even continuous, so having partial derivatives cannot be what it means for a function of two variables to be differentiable.

The example is the function whose limit depends on the direction of approach, and it shows that partial derivatives, the one direction slopes of the previous lesson, are too weak on their own. This lesson finds the right notion by asking what a derivative did in one variable and demanding the same of a plane, then turns the resulting tangent plane into approximations and into an estimate of how errors in measured inputs spread into a computed result. It assumes partial derivatives, written $f_{x}$ and $f_{y}$ , and the equation of a plane.

Two slopes are not enough

Take $f (x, y) = \frac{x y}{x^{2} + y^{2}}$ with $f (0, 0) = 0$ . On the $x$ axis the function is identically $0$ , so its slope in the $x$ direction at the origin is $f_{x} (0, 0) = 0$ , and the same argument on the $y$ axis gives $f_{y} (0, 0) = 0$ . Both partial derivatives exist. Yet along the line $y = x$ the function equals $\frac{1}{2}$ at every point, however close to the origin, so it is not continuous there.

The partials describe a surface along two lines only, the axes through the point. Here the surface is flat along both, at height $0$ , which suggests that near the origin it should look like the plane $z = 0$ . It does not: along the diagonal it stays half a unit above that plane all the way in. Two slopes say nothing about the infinitely many other directions, the same weakness the limit had when only some paths were checked.

Treat the candidate plane as an approximation and measure its error. At $(h, h)$ the function is $\frac{1}{2}$ and the plane is $0$ , so the error is $\frac{1}{2}$ however small $h$ is. An approximation whose error does not even go to zero is no approximation at all, and that is the property to repair.

What a derivative really promises

In one variable, $f^{'} (a)$ is defined as a limit of slopes, but the same fact can be stated as a promise about approximation. Write the change in $f$ as the tangent line's prediction plus an error:

f (a + h) = f (a) + f^{'} (a) h + E (h)

Dividing by $h$ gives $E (h) / h = (f (a + h) - f (a)) / h - f^{'} (a)$ , and the definition of the derivative says exactly that this tends to $0$ . So $f$ is differentiable at $a$ precisely when some number $m$ makes the error of $f (a) + m h$ shrink faster than $h$ itself, $E (h) / h \to 0$ . The error of a mere continuous function only tends to zero; the error of a differentiable one is small even compared with the step.

This version generalises without any trouble, because it never divides by a vector. A function $f$ of two variables is differentiable at $(a, b)$ if there are numbers $A$ and $B$ such that

f (a + h, b + k) = f (a, b) + A h + B k + E (h, k)

with $E (h, k) / \sqrt{h^{2} + k^{2}} \to 0$ as $(h, k) \to (0, 0)$ . The denominator is the distance from $(a, b)$ to the new point, so the requirement reads: the error of the linear prediction shrinks faster than the distance, in every direction at once.

The numbers $A$ and $B$ are forced. Set $k = 0$ and the condition becomes $(f (a + h, b) - f (a, b)) / h - A \to 0$ , which says $A = f_{x} (a, b)$ . Setting $h = 0$ gives $B = f_{y} (a, b)$ . So a differentiable function has both partials, and they are the only possible coefficients. Differentiability also forces continuity: as $(h, k) \to (0, 0)$ the terms $A h$ , $B k$ and $E$ all tend to $0$ , so $f (a + h, b + k) \to f (a, b)$ .

The pathological function now fails cleanly. Its only candidates are $A = B = 0$ , and at $(h, h)$ the error is $\frac{1}{2}$ while the distance is $h \sqrt{2}$ , so the ratio is $1 / (2 \sqrt{2} h)$ , which grows without bound instead of shrinking. The function is not differentiable at the origin, as its discontinuity already guaranteed.

A well behaved function passes just as cleanly. For $f (x, y) = x y$ at any $(a, b)$ , expanding gives $(a + h) (b + k) = a b + b h + a k + h k$ . The partials are $f_{x} = b$ and $f_{y} = a$ , so the error is exactly $h k$ . Since $2 | h k | \leq h^{2} + k^{2}$ , the error is at most half the square of the distance $d$ , and $| E | / d \leq d / 2 \to 0$ . The error is quadratic in the distance, which is the typical pattern.

The tangent plane

When $f$ is differentiable at $(a, b)$ , the linear part of the promise, written in terms of $x = a + h$ and $y = b + k$ , is the tangent plane to the graph $z = f (x, y)$ :

z = f (a, b) + f_{x} (a, b) (x - a) + f_{y} (a, b) (y - b)

It passes through the point $(a, b, f (a, b))$ on the surface, and it is the only plane that hugs the surface in the strong sense of the definition: any other plane through that point has an error that shrinks only as fast as the distance, or not at all.

The geometry agrees with the algebra. Slicing the surface with the plane $y = b$ gives a curve whose slope at the point is $f_{x} (a, b)$ , so its tangent line runs in the direction $⟨ 1, 0, f_{x} ⟩$ : one step in $x$ , none in $y$ , a rise of $f_{x}$ . The slice $x = a$ gives the direction $⟨ 0, 1, f_{y} ⟩$ . A tangent plane must contain both lines, and the cross product of the two directions is a normal to it:

⟨ 1, 0, f_{x} ⟩ \times ⟨ 0, 1, f_{y} ⟩ = ⟨ - f_{x}, - f_{y}, 1 ⟩

The plane through $(a, b, f (a, b))$ with this normal is $- f_{x} (x - a) - f_{y} (y - b) + (z - f (a, b)) = 0$ , which rearranges to the formula above.

Example. Find the tangent plane to $z = 9 - x^{2} - y^{2}$ at the point above $(1, 2)$ , and compare it with the surface at $(1.1, 2.1)$ .

The value is $f (1, 2) = 9 - 1 - 4 = 4$ . The partials are $f_{x} = - 2 x$ and $f_{y} = - 2 y$ , so $f_{x} (1, 2) = - 2$ and $f_{y} (1, 2) = - 4$ . The plane is $z = 4 - 2 (x - 1) - 4 (y - 2)$ , which simplifies to $z = 14 - 2 x - 4 y$ , with normal $⟨ 2, 4, 1 ⟩$ . At $(1.1, 2.1)$ the plane gives $14 - 2.2 - 8.4 = 3.4$ and the surface gives $9 - 1.21 - 4.41 = 3.38$ . The error is $- 0.02$ , and expanding $9 - (1 + h)^{2} - (2 + k)^{2}$ shows that it is exactly $- (h^{2} + k^{2})$ , minus the square of the distance, whatever the step. The plane lies above the surface, as it must for a dome.

Now you. Find the tangent plane to $z = x e^{y}$ at the point above $(2, 0)$ , and use it to estimate the height of the surface at $(2.1, 0.1)$ .

Answer

Here $f (2, 0) = 2$ , $f_{x} = e^{y} = 1$ and $f_{y} = x e^{y} = 2$ at the point, so the plane is $z = 2 + (x - 2) + 2 y$ , which simplifies to $z = x + 2 y$ . At $(2.1, 0.1)$ it gives $2.3$ , against the true value $2.1 e^{0.1} = 2.3209$ .

Continuous partials are enough

Checking the definition directly is tedious for anything larger than $x y$ , so in practice differentiability is recognised by a theorem.

Theorem. If $f_{x}$ and $f_{y}$ exist near $(a, b)$ and are continuous at $(a, b)$ , then $f$ is differentiable at $(a, b)$ .

The idea of the proof is to travel from $(a, b)$ to $(a + h, b + k)$ along two legs parallel to the axes, first to $(a + h, b)$ and then up to $(a + h, b + k)$ , so that on each leg only one variable moves and Calculus I applies. By the mean value theorem, the change along the first leg is $f_{x} (p) h$ for some point $p$ on that leg, and the change along the second is $f_{y} (q) k$ for some point $q$ on it. Subtracting the linear prediction leaves

E = [f_{x} (p) - f_{x} (a, b)] h + [f_{y} (q) - f_{y} (a, b)] k

Since $| h |$ and $| k |$ are each at most the distance $d$ , dividing by $d$ gives $| E | / d \leq | f_{x} (p) - f_{x} (a, b) | + | f_{y} (q) - f_{y} (a, b) |$ . As the point closes in, $p$ and $q$ are squeezed towards $(a, b)$ , and continuity of the partials sends both brackets to $0$ . That is the definition.

The theorem does its work through the inventory of continuous functions from the lesson on limits and continuity. The partials of a polynomial are polynomials, the partials of $e^{x y}$ or $\sin (x^{2} + y)$ are built from continuous pieces, and so every function assembled from the usual functions is differentiable wherever its formula and its partials make sense. The pathological function fails the hypothesis, as it must: away from the origin $f_{x} = y (y^{2} - x^{2}) / (x^{2} + y^{2})^{2}$ , which on the $y$ axis is $1 / y$ and blows up as the origin approaches. The converse of the theorem is false (there are differentiable functions whose partials are not continuous), but they are rarities built to make the point, and the theorem covers everything met in practice.

Linear approximation

Near a point of differentiability, the tangent plane is a cheap substitute for the function. The linear approximation of $f$ at $(a, b)$ is

f (x, y) \approx f (a, b) + f_{x} (a, b) (x - a) + f_{y} (a, b) (y - b)

and it needs only three numbers computed once at a convenient point: the value and the two partials. Every estimate nearby is then a matter of multiplying the small changes by the partials and adding.

Example. Estimate $\sqrt{{3.02}^{2} + {3.97}^{2}}$ from the linear approximation of $f (x, y) = \sqrt{x^{2} + y^{2}}$ at $(3, 4)$ , and check it against the true value.

At $(3, 4)$ the value is $5$ . The partials are $f_{x} = x / \sqrt{x^{2} + y^{2}}$ and $f_{y} = y / \sqrt{x^{2} + y^{2}}$ , which there are $0.6$ and $0.8$ . The steps are $h = 0.02$ and $k = - 0.03$ , so the estimate is $5 + 0.6 \times 0.02 + 0.8 \times (- 0.03) = 5 + 0.012 - 0.024 = 4.988$ . The true value is $\sqrt{24.8813} = 4.988116$ , so the error is about $0.000116$ , while the distance moved is $\sqrt{{0.02}^{2} + {0.03}^{2}} = 0.036$ . The error is a third of a percent of the distance.

Halving both steps, to $(3.01, 3.985)$ , shows the definition at work. The estimate is $4.994$ , the error drops to about $0.000029$ , a quarter of what it was, and the distance halves to $0.018$ . The ratio of error to distance has halved too, from $0.0032$ to $0.0016$ , and it keeps halving with each halving of the step. That is what shrinking faster than the distance looks like in numbers.

Now you. Use the linear approximation of $f (x, y) = \sqrt{x} \sqrt[3]{y}$ at $(9, 8)$ to estimate $\sqrt{9.2} \sqrt[3]{7.9}$ , and compare with the true value.

Answer

At $(9, 8)$ the value is $3 \times 2 = 6$ , with $f_{x} = \frac{\sqrt[3]{y}}{2 \sqrt{x}} = \frac{1}{3}$ and $f_{y} = \frac{\sqrt{x}}{3 y^{2 / 3}} = \frac{1}{4}$ . The estimate is $6 + \frac{1}{3} (0.2) + \frac{1}{4} (- 0.1) = 6.0417$ , against the true value $6.0409$ .

The approximation is only local. Using the same plane to estimate $f (4, 3)$ , a distance $1.4$ away, gives $5 + 0.6 - 0.8 = 4.8$ , while the true value is again exactly $5$ . The surface is a cone whose slope turns as the point moves round it, and how far a step can go before the error matters depends on how fast the partials themselves change, which is a question of second derivatives.

The total differential

The linear approximation has a compact notation that is worth adopting because it carries over to any number of variables. For $z = f (x, y)$ the total differential is

d z = f_{x} d x + f_{y} d y

where $d x$ and $d y$ are independent small changes in the inputs and $d z$ is the change they produce along the tangent plane. The actual change in $z$ , written $Δ z$ , differs from $d z$ by the error term, which is small compared with the size of the step. For three variables, $w = f (x, y, z)$ , the definition and the theorem go through word for word with three terms and the distance in space, and $d w = f_{x} d x + f_{y} d y + f_{z} d z$ .

Each term answers a separate question: how much does the output move if only this input moves? Near the point, the answers simply add. That additivity is the whole content of differentiability, and it fails for the pathological function, where moving $x$ alone or $y$ alone changes nothing but moving both together changes the value by $\frac{1}{2}$ .

For formulas built from products and powers, dividing by the function gives a relative form that is easier to use. The volume of a cylinder is $V = π r^{2} h$ , so

d V = 2 π r h d r + π r^{2} d h

and dividing through by $V = π r^{2} h$ gives

\frac{d V}{V} = 2 \frac{d r}{r} + \frac{d h}{h}

The relative change in the volume is twice the relative change in the radius plus the relative change in the height. The exponents become weights: a quantity raised to the power $n$ passes on $n$ times its relative change, and a quantity in the denominator passes on its relative change with a minus sign.

Propagating measurement error

Every measured input carries an uncertainty, and a quantity computed from measured inputs inherits one. The total differential says how much. If $x$ is known to within $\pm Δ x$ and $y$ to within $\pm Δ y$ , and these are small, then the computed $z = f (x, y)$ is off by at most about

| f_{x} | Δ x + | f_{y} | Δ y

The absolute values are there because a measurement error can go either way, and the worst case is the one where both terms push in the same direction. This is the standard worst case bound quoted in engineering and in the first year laboratory.

Example. A cylindrical can has a measured radius of $5.0 \pm 0.1$ cm and height of $12.0 \pm 0.1$ cm. Estimate the maximum error in its computed volume, absolute and relative.

The volume is $V = π r^{2} h = 300 π$ , about $942.5$ cm³. The differential gives $d V = 2 π r h d r + π r^{2} d h = 2 π (60) (0.1) + π (25) (0.1) = 14.5 π$ , so the volume is uncertain by about $45.6$ cm³. The relative form gives the same answer more quickly: $2 \times \frac{0.1}{5} + \frac{0.1}{12} = 0.04 + 0.0083 = 0.048$ , a relative error of about $4.8 %$ . As a check, the largest possible can, $π (5.1)^{2} (12.1)$ , exceeds $300 π$ by $46.2$ cm³, and the smallest, $π (4.9)^{2} (11.9)$ , falls short by $44.9$ cm³. The differential sits between the two, as a linear estimate of a slightly curved change should.

The breakdown is the useful part. Of the $4.8 %$ , the radius contributes $4 %$ and the height less than $1 %$ , even though both were measured to the same $0.1$ cm, because the radius is smaller and is squared. A better volume needs a better calliper for the diameter, not a better ruler for the height.

Now you. One mole of gas at a measured temperature of $300 \pm 1$ K occupies a measured volume of $0.0250 \pm 0.0001$ m³. The ideal gas law gives the pressure as $P = n R T / V$ with $R = 8.314$ J/(mol K). Find $P$ and the maximum relative error in it.

Answer

The pressure is $P = 8.314 \times 300 / 0.025 = 99 768$ Pa, about $99.8$ kPa. The differential is $d P = \frac{n R}{V} d T - \frac{n R T}{V^{2}} d V$ , and dividing by $P$ gives $\frac{d P}{P} = \frac{d T}{T} - \frac{d V}{V}$ . The worst case adds the sizes: $\frac{1}{300} + \frac{0.0001}{0.025} = 0.0033 + 0.0040 = 0.0073$ , a relative error of about $0.73 %$ , or roughly $730$ Pa.

The worst case bound is deliberately pessimistic. Independent random errors rarely conspire, and statistics replaces the sum of the terms by the square root of the sum of their squares: for the can, $\sqrt{{0.04}^{2} + {0.0083}^{2}} = 0.041$ , or $4.1 %$ instead of $4.8 %$ . Either way the differential supplies the terms, and either way the errors must be small enough for the tangent plane to be trusted.

Compositions

Differentiability, a single plane whose error vanishes faster than the distance in every direction at once, is the notion the previous lesson was missing. It implies continuity, it holds wherever the partials are continuous, and its practical form, the total differential, turns small changes in the inputs into a change in the output.

In practice the inputs are rarely free to vary on their own. The temperature recorded by a weather balloon is $T (x, y, z)$ evaluated along the balloon's path, where $x$ , $y$ and $z$ are themselves functions of time. The volume of a can being heated has a radius and height that both depend on the temperature. A formula given in $x$ and $y$ may be needed in terms of the polar $r$ and $θ$ . Each is a composition: a function of several variables whose variables depend on something else.

The total differential already hints at what happens. If $z = f (x, y)$ and both $x$ and $y$ change because time does, then $d z = f_{x} d x + f_{y} d y$ should turn into a rate by dividing by $d t$ . Whether that is legitimate, what it becomes when the inner functions have several variables of their own, and how it recovers implicit differentiation in one line are the business of the chain rule, the subject of the next lesson.