Maxima and minima

Setting the gradient to zero finds every point where a smooth function of two variables could have a maximum or a minimum, but it does not say which of those points are summits, which are hollows and which are neither.

The previous lesson showed that at a local maximum or minimum the gradient $\nabla f = ⟨ f_{x}, f_{y} ⟩$ , the vector of partials that points steepest uphill, must be $𝟎$ . In Calculus I the sign of $f^{''}$ sorted the candidates. This lesson builds the two variable version from the second order Taylor polynomial, uses it to fit a line to data, and then finds the largest and smallest values over a whole region. It assumes partial derivatives, the equality of mixed partials, and the chain rule along a path.

Critical points

A function $f$ has a local maximum at $(a, b)$ if $f (a, b) \geq f (x, y)$ for every $(x, y)$ in some disc around $(a, b)$ , and a local minimum if $f (a, b) \leq f (x, y)$ there. The extreme is absolute if the inequality holds over the whole domain in question. At an interior local extreme where $f$ is differentiable both partials vanish, which Calculus I also shows: on the line $y = b$ the function $f (x, b)$ has a local extreme at $x = a$ , so $f_{x} (a, b) = 0$ , and likewise $f_{y} (a, b) = 0$ .

A critical point is an interior point of the domain where $\nabla f = 𝟎$ or where a partial derivative fails to exist. The second clause matters. The cone $f (x, y) = \sqrt{x^{2} + y^{2}}$ is lowest at the origin, but on the $x$ axis it equals $| x |$ , which has no derivative at $0$ , so $f_{x} (0, 0)$ does not exist and no equation $\nabla f = 𝟎$ will find the tip.

Finding critical points means solving two equations in two unknowns, rarely linear, so factoring and substitution are the tools. For $f (x, y) = x^{2} + y^{2} - 2 x + 6 y + 14$ , setting $f_{x} = 2 x - 2$ and $f_{y} = 2 y + 6$ to zero gives the single point $(1, - 3)$ , where $f = 4$ . Completing the square shows what it is: $f = (x - 1)^{2} + (y + 3)^{2} + 4$ is never below $4$ , so the point is an absolute minimum. Completing a square is the whole idea of the test below.

Not every critical point is an extreme. For $f (x, y) = x^{2} - y^{2}$ the gradient $⟨ 2 x, - 2 y ⟩$ vanishes at the origin, yet the surface rises along the $x$ axis and falls along the $y$ axis. A critical point with higher and lower values in every disc around it is a saddle point, after the shape of a horse's saddle, or of a mountain pass, the lowest point of the ridge and the highest point of the road across it.

The second order Taylor polynomial

Calculus I approximated a function of one variable by its Taylor polynomial of degree two, $g (t) \approx g (0) + g^{'} (0) t + \frac{1}{2} g^{''} (0) t^{2}$ , with an error that shrinks faster than $t^{2}$ . The two variable version comes from applying it along a line. Fix a step $(h, k)$ and set $g (t) = f (a + t h, b + t k)$ , the function seen by a point leaving $(a, b)$ in the direction $(h, k)$ . The chain rule gives

g^{'} (t) = f_{x} h + f_{y} k

with the partials evaluated at the moving point. Differentiating again, $f_{x}$ is itself a function along the path, with derivative $f_{x x} h + f_{x y} k$ , and $f_{y}$ has derivative $f_{y x} h + f_{y y} k$ . When the second partials are continuous the mixed ones agree, and

g^{''} (t) = f_{x x} h^{2} + 2 f_{x y} h k + f_{y y} k^{2}

Putting $t = 1$ , with every partial evaluated at $(a, b)$ , gives the second order Taylor polynomial:

f (a + h, b + k) \approx f (a, b) + f_{x} h + f_{y} k + \frac{1}{2} (f_{x x} h^{2} + 2 f_{x y} h k + f_{y y} k^{2})

The first three terms are the tangent plane and the bracket corrects for curvature. With continuous second partials the error is small compared with $h^{2} + k^{2}$ , the square of the distance moved. For $f = e^{x} \cos y$ at the origin, $f = f_{x} = f_{x x} = 1$ , $f_{y} = f_{x y} = 0$ and $f_{y y} = - 1$ , so the polynomial is $1 + x + \frac{1}{2} (x^{2} - y^{2})$ . At $(0.1, 0.2)$ it gives $1.085$ against the true $1.08314$ , while the tangent plane alone gives $1.1$ , nine times further off.

At a critical point the linear terms are zero, and what is left says how the function leaves the point:

f (a + h, b + k) - f (a, b) \approx \frac{1}{2} (A h^{2} + 2 B h k + C k^{2})

with $A = f_{x x}$ , $B = f_{x y}$ and $C = f_{y y}$ at $(a, b)$ . The bracket is a quadratic form in $h$ and $k$ , and whether the point is a maximum, a minimum or a saddle is a question about its sign in every direction.

The second derivative test

Complete the square, as for the paraboloid above. If $A \neq 0$ , then

A h^{2} + 2 B h k + C k^{2} = A [{(h + \frac{B}{A} k)}^{2} + \frac{A C - B^{2}}{A^{2}} k^{2}]

as expanding the square confirms. Write $D = A C - B^{2}$ . If $D > 0$ , the bracket is a square plus a positive multiple of a square, zero only when $k = 0$ and then $h = 0$ , so in every direction the form has the sign of $A$ . If $D < 0$ , the bracket is positive for $k = 0$ and negative for $h = - B k / A$ with $k \neq 0$ , so the function rises in some directions and falls in others. (If $A = 0$ and $D < 0$ , then $B \neq 0$ and the form $k (2 B h + C k)$ also takes both signs.)

That is the second derivative test. At a critical point where the second partials are continuous, let

D = f_{x x} f_{y y} - f_{x y}^{2}

If $D > 0$ and $f_{x x} > 0$ the point is a local minimum; if $D > 0$ and $f_{x x} < 0$ it is a local maximum; if $D < 0$ it is a saddle; and if $D = 0$ the test says nothing. When $D > 0$ , $f_{x x}$ and $f_{y y}$ must share a sign, so either may be checked. The four second partials arranged in a square array are called the Hessian, after the German mathematician Otto Hesse, and $D$ is its determinant.

Discarding the Taylor error is safe because of sizes. When $D > 0$ the form is at least a fixed multiple of $h^{2} + k^{2}$ , while the error is eventually smaller than any such multiple, so near the point it cannot overturn the sign. When $D < 0$ the rising and falling directions are fixed lines, and along each the one variable test applies.

Example. Find and classify the critical points of $f (x, y) = x^{3} + y^{3} - 3 x y$ .

The partials are $f_{x} = 3 x^{2} - 3 y$ and $f_{y} = 3 y^{2} - 3 x$ . The first vanishes when $y = x^{2}$ , and substituting into the second gives $x^{4} = x$ , so $x (x^{3} - 1) = 0$ : the critical points are $(0, 0)$ and $(1, 1)$ . The second partials are $f_{x x} = 6 x$ , $f_{y y} = 6 y$ and $f_{x y} = - 3$ . At the origin $D = 0 \times 0 - 9 = - 9 < 0$ , a saddle: along $y = - x$ the function is $3 x^{2}$ , rising, and along $y = x$ it is $2 x^{3} - 3 x^{2}$ , falling. At $(1, 1)$ , $D = 36 - 9 = 27 > 0$ with $f_{x x} = 6 > 0$ , a local minimum, $f (1, 1) = - 1$ . It is not absolute, since $f (x, 0) = x^{3}$ goes to $- \infty$ .

Now you. Find and classify the critical points of $f (x, y) = 2 x^{3} + 6 x y + 3 y^{2}$ .

Answer

$f_{x} = 6 x^{2} + 6 y$ and $f_{y} = 6 x + 6 y$ . The second gives $y = - x$ , and then the first gives $6 x^{2} - 6 x = 0$ , so the points are $(0, 0)$ and $(1, - 1)$ . With $f_{x x} = 12 x$ , $f_{y y} = 6$ and $f_{x y} = 6$ , the origin has $D = 0 - 36 = - 36$ , a saddle, and $(1, - 1)$ has $D = 72 - 36 = 36 > 0$ with $f_{x x} = 12 > 0$ , a local minimum with value $- 1$ .

Saddles, and the case the test cannot decide

For $x^{2} - y^{2}$ at the origin, $f_{x x} = 2$ , $f_{y y} = - 2$ and $f_{x y} = 0$ , so $D = - 4$ and the test confirms the saddle; the level curve through the pass is the pair of lines $y = \pm x$ , crossing in an X. A saddle need not announce itself on the axes. The function $x^{2} + 4 x y + y^{2}$ has $f_{x x} = f_{y y} = 2$ and curves upward along both axes, yet $D = 4 - 16 = - 12$ , and along $y = - x$ it equals $- 2 x^{2}$ . Checking the axes alone is exactly the mistake the test prevents. The monkey saddle $x^{3} - 3 x y^{2}$ is a pass with three ways down instead of two, room for two legs and a tail, and at its centre every second partial is zero.

So the monkey saddle has $D = 0$ , and $D = 0$ is a genuine gap. Compare $x^{4} + y^{4}$ with $x^{4} - y^{4}$ . Both have gradient $𝟎$ and every second partial $0$ at the origin, so both give $D = 0$ . Yet the first is never negative, a minimum, while the second is positive along the $x$ axis and negative along the $y$ axis, a saddle; $- x^{4} - y^{4}$ adds a maximum with the same $D$ . When $D = 0$ the quadratic form vanishes along at least one line, and the higher terms that the Taylor polynomial discarded decide. The remedy is to study the function itself, by factoring it or by following it along lines and curves through the point.

The least squares line

Charles David Keeling began measuring carbon dioxide on Mauna Loa, Hawaii, in 1958, and the record continues. Here are five annual means, rounded to the nearest part per million, with $x$ counting years from 2019:

Year	$x$	CO₂ (ppm)
2019	0	412
2020	1	414
2021	2	416
2022	3	419
2023	4	421

No line passes through all five points, so the question is which line $y = m x + c$ fits best. The miss at each point is the residual $y_{i} - (m x_{i} + c)$ . Summing residuals lets misses of opposite sign cancel, and summing their sizes gives a function with corners, so the standard choice, published by Adrien-Marie Legendre in 1805, is to minimise the sum of their squares:

S (m, c) = \sum_{i = 1}^{n} {(y_{i} - m x_{i} - c)}^{2}

This is a function of the two unknowns $m$ and $c$ , and its minimum is a critical point. The partials are

\frac{\partial S}{\partial c} = - 2 \sum (y_{i} - m x_{i} - c), \frac{\partial S}{\partial m} = - 2 \sum x_{i} (y_{i} - m x_{i} - c)

Setting the first to zero gives $\overline{y} = m \overline{x} + c$ : the best line passes through the point of means. Setting the second to zero and substituting $c = \overline{y} - m \overline{x}$ gives $\sum x_{i} (y_{i} - \overline{y}) = m \sum x_{i} (x_{i} - \overline{x})$ , and since the deviations $y_{i} - \overline{y}$ and $x_{i} - \overline{x}$ each sum to zero, $x_{i}$ may be replaced by $x_{i} - \overline{x}$ on both sides:

m = \frac{\sum (x_{i} - \overline{x}) (y_{i} - \overline{y})}{\sum (x_{i} - \overline{x})^{2}}

The test confirms a minimum. The second partials are $S_{m m} = 2 \sum x_{i}^{2}$ , $S_{c c} = 2 n$ and $S_{m c} = 2 \sum x_{i}$ , so $D = 4 (n \sum x_{i}^{2} - (\sum x_{i})^{2}) = 4 n \sum (x_{i} - \overline{x})^{2}$ , positive unless every $x_{i}$ is equal, and $S_{c c} > 0$ . Since $S$ is a quadratic polynomial, its Taylor polynomial is exact and the minimum is absolute. Statistics derives the same line from a model of random error.

Example. Fit the least squares line to the Mauna Loa data above.

The means are $\overline{x} = 2$ and $\overline{y} = 2082 / 5 = 416.4$ . The deviations of $y$ are $- 4.4$ , $- 2.4$ , $- 0.4$ , $2.6$ and $4.6$ , so the numerator is $8.8 + 2.4 + 0 + 2.6 + 9.2 = 23$ and the denominator is $4 + 1 + 0 + 1 + 4 = 10$ . The slope is $m = 2.3$ and the intercept $c = 416.4 - 2.3 \times 2 = 411.8$ . The residuals are $0.2$ , $- 0.1$ , $- 0.4$ , $0.3$ and $0$ , summing to zero as the first equation demands, with minimum sum of squares $0.30$ . Carbon dioxide rose by about $2.3$ ppm a year. The line predicts $423.3$ ppm for 2024 and $425.6$ for 2025, and the measured means were $424.61$ and $427.35$ : both above the line, because the rise is itself speeding up, which no straight line can capture.

Now you. The Mauna Loa means for 1960, 1970, 1980 and 1990 were $317$ , $326$ , $339$ and $354$ ppm. With $x$ in years since 1960, fit the least squares line, and compare its slope with the recent one.

Answer

$\overline{x} = 15$ and $\overline{y} = 1336 / 4 = 334$ . The numerator is $(- 15) (- 17) + (- 5) (- 8) + (5) (5) + (15) (20) = 620$ and the denominator is $225 + 25 + 25 + 225 = 500$ , so $m = 1.24$ and $c = 334 - 1.24 \times 15 = 315.4$ . The line is $y = 1.24 x + 315.4$ , a rise of $1.24$ ppm a year, little more than half the recent $2.3$ .

Absolute extrema on a closed and bounded region

A largest value over a region need not exist: $f (x, y) = x$ on the open disc $x^{2} + y^{2} < 1$ gets as close to $1$ as it likes without reaching it, and on the whole plane it is unbounded. The extreme value theorem in two variables rules out both failures. A continuous function on a set that is closed, containing all its boundary points, and bounded, fitting inside some disc, attains an absolute maximum and an absolute minimum. As in one variable, the proof belongs to analysis.

If the largest value occurs inside the region, it is a local maximum there, so it is at a critical point. Otherwise it is on the boundary. That gives a finite procedure: evaluate $f$ at the critical points inside the region; on each piece of the boundary, write $f$ as a function of one variable and find its extremes on a closed interval, at critical points and endpoints (the corners); then take the largest and smallest values found. No second derivative test is needed, only comparison.

Example. A triangular plate has corners $(0, 0)$ , $(4, 0)$ and $(0, 4)$ , and its temperature is $T (x, y) = x^{2} + y^{2} - 2 x - 4 y + 20$ degrees Celsius. Find the hottest and coldest points.

Inside, $T_{x} = 2 x - 2$ and $T_{y} = 2 y - 4$ vanish at $(1, 2)$ , which lies in the triangle since $1 + 2 \leq 4$ , and $T (1, 2) = 15$ . On the bottom edge, $T = x^{2} - 2 x + 20$ for $0 \leq x \leq 4$ , giving $19$ at $x = 1$ and $20$ and $28$ at the ends. On the left edge, $T = y^{2} - 4 y + 20$ gives $16$ at $y = 2$ and $20$ at both ends. On the slanted edge $y = 4 - x$ , substitution gives $T = 2 x^{2} - 6 x + 20$ , which is $15.5$ at $x = 1.5$ , the point $(1.5, 2.5)$ , and $20$ and $28$ at the ends. The coldest point is $(1, 2)$ at $15$ degrees and the hottest is the corner $(4, 0)$ at $28$ degrees. As a check, $T = (x - 1)^{2} + (y - 2)^{2} + 15$ is $15$ plus the squared distance from $(1, 2)$ , and $(4, 0)$ is the point of the triangle farthest from it.

Now you. Find the absolute maximum and minimum of $f (x, y) = x y - x - 2 y$ on the rectangle $0 \leq x \leq 4$ , $0 \leq y \leq 3$ .

Answer

Inside, $f_{x} = y - 1$ and $f_{y} = x - 2$ vanish at $(2, 1)$ , where $f = - 2$ (a saddle, with $D = - 1$ ). Each edge is linear, so its extremes are at the corners, where $f (0, 0) = 0$ , $f (4, 0) = - 4$ , $f (0, 3) = - 6$ and $f (4, 3) = 2$ . The maximum is $2$ at $(4, 3)$ and the minimum is $- 6$ at $(0, 3)$ . The interior critical point is neither.

When the inputs are constrained

Setting $\nabla f = 𝟎$ finds the candidates, and $D = f_{x x} f_{y y} - f_{x y}^{2}$ sorts them by the sign of a quadratic form, except in the degenerate case $D = 0$ . The same machinery fits a line to data, and comparing interior critical points with the boundary finds the absolute extremes on a closed and bounded region.

Look again at the hottest point of the plate. At $(4, 0)$ the gradient is $⟨ 6, - 4 ⟩$ , not $𝟎$ : the temperature would keep rising if the point could leave the triangle, and it cannot. On a boundary the best point need not be critical, and most real problems live entirely on such a boundary. A box must use a fixed area of cardboard, a factory must spend a fixed budget. The inputs are restricted to a curve or surface, and the question is the best point on it.

Take the simplest case: maximise $f = x y$ on the line $x + y = 10$ . Substituting $y = 10 - x$ gives $x (10 - x)$ , largest at $x = 5$ , so the best point is $(5, 5)$ with $f = 25$ . There the gradient $⟨ y, x ⟩ = ⟨ 5, 5 ⟩$ is not zero, but it is parallel to $⟨ 1, 1 ⟩$ , which is perpendicular to the line. Substitution worked only because the constraint was easy to solve; on an ellipse or a sphere it is clumsy or impossible. Turning those parallel gradients into a method that never solves the constraint is the work of the next lesson, on Lagrange multipliers.