Steepest ascent and descent

At a point $(a,b)$, in which direction does $f$ rise fastest?

Every unit direction $\vec{u}$ has its own rate $D_{\vec{u}}f(a,b)$, and only the direction varies from one rate to the next. There are infinitely many directions, so we cannot compare the rates by evaluating $D_{\vec{u}}f(a,b)$ one direction at a time. We need a formula for the rate in which the direction appears only through one angle.

Every one of those rates is a dot product, and the dot product also has a geometric form: the two lengths multiplied by the cosine of the angle between the vectors. Writing $\theta$ for the angle between $\vec{u}$ and $\nabla f(a,b)$,

A direction $\vec{u}$ is a unit vector, so $\|\vec{u}\| = 1$ and the second length drops out:

At a fixed point $\|\nabla f(a,b)\|$ is a fixed number, so turning the direction changes only $\cos\theta$, which takes every value from $-1$ to $1$.

At a point $(a,b)$ where $\|\nabla f(a,b)\| = 10$, find the rate of change of $f$ at $(a,b)$ in a unit direction $\vec u$ at $\theta = 60^\circ$ to $\nabla f(a,b)$, and in a unit direction at $\theta = 90^\circ$ to it.

Solution

Both directions are unit vectors, so each rate is the length of the gradient multiplied by the cosine of the angle:

The length $10$ is the same for both directions, because the point is fixed. Only $\theta$ changes.

At $\theta = 60^\circ$ we have $\cos 60^\circ = \dfrac{1}{2}$, so

and $f$ rises at $5$ per unit of distance in that direction.

At $\theta = 90^\circ$ we have $\cos 90^\circ = 0$, so

and $f$ neither rises nor falls in that direction.

We can now find the rate of change of $f$ in any unit direction, and we can compute $\nabla f(a,b)$ at any point. Together they answer the question of which direction makes $f$ rise fastest, and the answer is the gradient itself.

At each point $(a,b)$ where $\nabla f(a,b) \neq \vec 0$, the direction of $\nabla f(a,b)$ is the direction of steepest ascent, and its length $\|\nabla f(a,b)\|$ is the rate of steepest ascent. This holds at one point at a time: $\nabla f(a,b)$ changes from point to point, and the direction of steepest ascent changes with it.

The reason is the formula $D_{\vec{u}}f(a,b) = \|\nabla f(a,b)\| \cos\theta$. At a fixed point only $\theta$ changes, so the largest and smallest rates come from the largest and smallest values of $\cos\theta$:

At a point where $\nabla f(a,b) \neq \vec 0$, both directions and their rates come from the gradient alone:

The board below draws the gradient at a point and then the two unit directions it gives.

For $f(x,y) = 3x^2 + y^2$, find the unit directions of steepest ascent and steepest descent at $(1,4)$, and the rate of change of $f$ in each.

Solution

First we compute the gradient at the point. The power rule gives the first partials, and substituting $x = 1$ and $y = 4$ turns them into numbers:

Next we compute the length of the gradient:

This length is the largest rate of change at $(1,4)$, so in the direction of steepest ascent $f$ rises at $10$ per unit of distance. Dividing the gradient by its length gives that direction as a unit vector:

Steepest descent is the opposite unit direction, so we negate both components:

In this direction the rate of change is $-10$, so $f$ falls at $10$ per unit of distance.

A contour plot usually comes with no formula for $f$, but it still shows which way $\nabla f$ points at any point.

At a point $P$ on a contour, $f$ has the same value all along that contour, so the rate of change along it is zero. In a unit direction at angle $\theta$ to the gradient the rate is $\|\nabla f\|\cos\theta$, which, when the gradient is not zero, is zero only at $\theta = 90^\circ$. So the gradient crosses the contour at a right angle. Of the two directions at a right angle, it is the one pointing towards the higher levels, because the gradient is the direction of steepest ascent.