Mathematics · live now

Optimisation Foundations

Extend calculus to functions of several variables and use it to optimise them. Learn the multivariable chain rule, the gradient, directional derivatives and tangent planes. Then use the Hessian and convexity to find and classify critical points.

See pricing Preview a lesson

No account needed to read a lesson · 30-day money-back guarantee

See the full learning path

lessons

practice questions

102

prerequisite lessons

See it taught

2 of the 7 lessons here are free to preview - the whole explanation and worked example, with no account and no email.

Free preview

Steepest ascent and descent

Learn how the rate of change depends on the angle to the gradient, find the directions in which a function rises and falls fastest and how fast, and read the direction of the gradient off a contour plot.

D_{\vec{u}}f(a,b) = \|\nabla f(a,b)\| \cos\theta

Free preview

Tangent planes and linear approximation

Learn how to write the equation of the tangent plane to a surface at a point from the height and the two partial derivatives there, and how to use it to estimate values of a function of two variables near that point.

z = 3 + 2(x-1) + 4(y-1)

Live, not a screenshot

Lesson 06 · Steepest ascent and descent

0,0

(a, b)

(a, b)

\nabla f (a, b)

\vec{u}

θ

Move the slider to turn the unit direction $\vec{u}$ .

θ = 0^{\circ}

θ = 15^{\circ}

θ = 30^{\circ}

θ = 45^{\circ}

θ = 60^{\circ}

θ = 75^{\circ}

θ = 90^{\circ}

θ = 105^{\circ}

θ = 120^{\circ}

θ = 135^{\circ}

θ = 150^{\circ}

θ = 165^{\circ}

θ = 180^{\circ}

θ = 165^{\circ}

θ = 150^{\circ}

θ = 135^{\circ}

θ = 120^{\circ}

θ = 105^{\circ}

θ = 90^{\circ}

θ = 75^{\circ}

θ = 60^{\circ}

θ = 45^{\circ}

θ = 30^{\circ}

θ = 15^{\circ}

θ = 0^{\circ}

\cos θ = 1.00

D_{\vec{u}} f (a, b) = ‖ \nabla f (a, b) ‖ \cos θ = 2.24

\cos θ = 0.97

D_{\vec{u}} f (a, b) = ‖ \nabla f (a, b) ‖ \cos θ = 2.16

\cos θ = 0.87

D_{\vec{u}} f (a, b) = ‖ \nabla f (a, b) ‖ \cos θ = 1.94

\cos θ = 0.71

D_{\vec{u}} f (a, b) = ‖ \nabla f (a, b) ‖ \cos θ = 1.58

\cos θ = 0.50

D_{\vec{u}} f (a, b) = ‖ \nabla f (a, b) ‖ \cos θ = 1.12

\cos θ = 0.26

D_{\vec{u}} f (a, b) = ‖ \nabla f (a, b) ‖ \cos θ = 0.58

\cos θ = 0.00

D_{\vec{u}} f (a, b) = ‖ \nabla f (a, b) ‖ \cos θ = 0.00

\cos θ = - 0.26

D_{\vec{u}} f (a, b) = ‖ \nabla f (a, b) ‖ \cos θ = - 0.58

\cos θ = - 0.50

D_{\vec{u}} f (a, b) = ‖ \nabla f (a, b) ‖ \cos θ = - 1.12

\cos θ = - 0.71

D_{\vec{u}} f (a, b) = ‖ \nabla f (a, b) ‖ \cos θ = - 1.58

\cos θ = - 0.87

D_{\vec{u}} f (a, b) = ‖ \nabla f (a, b) ‖ \cos θ = - 1.94

\cos θ = - 0.97

D_{\vec{u}} f (a, b) = ‖ \nabla f (a, b) ‖ \cos θ = - 2.16

\cos θ = - 1.00

D_{\vec{u}} f (a, b) = ‖ \nabla f (a, b) ‖ \cos θ = - 2.24

\cos θ = - 0.97

D_{\vec{u}} f (a, b) = ‖ \nabla f (a, b) ‖ \cos θ = - 2.16

\cos θ = - 0.87

D_{\vec{u}} f (a, b) = ‖ \nabla f (a, b) ‖ \cos θ = - 1.94

\cos θ = - 0.71

D_{\vec{u}} f (a, b) = ‖ \nabla f (a, b) ‖ \cos θ = - 1.58

\cos θ = - 0.50

D_{\vec{u}} f (a, b) = ‖ \nabla f (a, b) ‖ \cos θ = - 1.12

\cos θ = - 0.26

D_{\vec{u}} f (a, b) = ‖ \nabla f (a, b) ‖ \cos θ = - 0.58

\cos θ = 0.00

D_{\vec{u}} f (a, b) = ‖ \nabla f (a, b) ‖ \cos θ = 0.00

\cos θ = 0.26

D_{\vec{u}} f (a, b) = ‖ \nabla f (a, b) ‖ \cos θ = 0.58

\cos θ = 0.50

D_{\vec{u}} f (a, b) = ‖ \nabla f (a, b) ‖ \cos θ = 1.12

\cos θ = 0.71

D_{\vec{u}} f (a, b) = ‖ \nabla f (a, b) ‖ \cos θ = 1.58

\cos θ = 0.87

D_{\vec{u}} f (a, b) = ‖ \nabla f (a, b) ‖ \cos θ = 1.94

\cos θ = 0.97

D_{\vec{u}} f (a, b) = ‖ \nabla f (a, b) ‖ \cos θ = 2.16

\cos θ = 1.00

D_{\vec{u}} f (a, b) = ‖ \nabla f (a, b) ‖ \cos θ = 2.24

Here $\nabla f (a, b) = [\begin{matrix} 2 \\ 1 \end{matrix}]$ , so $‖ \nabla f (a, b) ‖ = \sqrt{5} \approx 2.24$ . The angle $θ$ is always the smaller angle between $\vec{u}$ and $\nabla f (a, b)$ , so it rises to $180^{\circ}$ and then falls back to $0^{\circ}$ .

This is the diagram from the lesson itself, running here. Drag it - the lesson is built round the thing it shows, not round a picture of it.

The 7 lessons

The 7 lessons in this unit. Expand any one to see what it covers. Each unlocks when its own prerequisites are passed.

The multivariable chain rule

12 questions

Learn how to draw a dependency diagram for a function whose inputs each depend on one further variable, read the multivariable chain rule from it, and write the chain rule equation for functions given by formulas.

Covers

Dependency diagrams

The multivariable chain rule

Writing the chain rule for given functions

Computing with the multivariable chain rule

12 questions

Learn how to compute with the multivariable chain rule when the functions are given as formulas, first with one independent variable and then with two, where every derivative is partial.

Covers

One independent variable

Two independent variables

Computing both partial derivatives

The gradient vector

6 questions

Learn how to find the gradient of a function of two or more variables by collecting its first partial derivatives into a single column vector, and how to evaluate it at a point.

Covers

Computing the gradient

Rates of change along a line

14 questions

Learn how to find the point reached by stepping along a line in a unit direction, and how fast a function of two variables changes as you move along that line.

Covers

The point reached after a step

How direction and position affect the rate

Computing the rate at a point

Directional derivatives

13 questions

Learn how to find the rate of change of a function in any direction as the dot product of its gradient with a unit vector, normalise a direction that is not a unit vector, and interpret what the value tells you about the function.

Covers

The directional derivative

Computing a directional derivative

Interpreting a directional derivative

Steepest ascent and descent

Preview

13 questions

Covers

The rate as $\|\nabla f\|\cos\theta$

Steepest ascent is along the gradient

The gradient on a contour plot

Preview this lesson - no account needed

Tangent planes and linear approximation

Preview

8 questions

Covers

The tangent plane to a surface

Linear approximation in two variables

Preview this lesson - no account needed

Where this unit sits in the curriculum

This unit is part of our Essential Calculus for ML learning path, which contains 189 lessons. Every one of them is drawn below.

The diagnostic test lets you skip any lesson you already know, including the ones in this unit.

Hover any dot to name its lesson.

7 lessons in this unit

102 prerequisite lessons across the unit, counting every step back to the start

80 other lessons in the learning path - after this unit, or alongside it

Every dot is a lesson, every line a prerequisite. You can only start a lesson once you have mastered all of its prerequisites, so you are always building on solid foundations.

Builds on: Derivatives, Partial derivatives, Vectors in Euclidean space

Start Optimisation Foundations

One subscription covers every learning path, and you can test out of anything you already know.

See pricing Preview a lesson

One subscription covers every learning path · 30-day money-back guarantee

More in Calculus: Limits, Derivatives, Integrals, Partial derivatives, Double integrals