Free preview

149 lessons

Essential Calculus for ML

Free preview

Essential Calculus for ML · 149 lessons

Master the fundamentals

No surprise gaps

Actually remember it

Skip what you know

One subscription. All learning paths included.

Our content is best on a larger screen

Second-order partial derivatives

The four second-order partials

Explanation

For a function of one variable, differentiating ff' again gives the second derivative ff''. A function of two variables, f(x,y)f(x, y), has two first partials, fxf_x and fyf_y. Each is itself a function of xx and yy, so each can be differentiated by xx or by yy. That gives four second-order partial derivatives, each defined only where both of its steps exist.

Second-order partial derivatives

Definition

A second-order partial derivative of ff is a partial derivative of one of the first partials fxf_x and fyf_y. In subscript notation,

fxy=(fx)y,f_{xy} = \left(f_x\right)_y,

so ff is differentiated by xx, and the result is then differentiated by yy. The two that use both variables, fxyf_{xy} and fyxf_{yx}, are the mixed partial derivatives.

Each of the four has a subscript form and a \partial form:

Subscript form\partial formOrder of differentiation
fxx=(fx)xf_{xx} = \left(f_x\right)_x2fx2\dfrac{\partial^2 f}{\partial x^2}by xx, then by xx
fyy=(fy)yf_{yy} = \left(f_y\right)_y2fy2\dfrac{\partial^2 f}{\partial y^2}by yy, then by yy
fxy=(fx)yf_{xy} = \left(f_x\right)_y2fyx\dfrac{\partial^2 f}{\partial y\,\partial x}by xx, then by yy
fyx=(fy)xf_{yx} = \left(f_y\right)_x2fxy\dfrac{\partial^2 f}{\partial x\,\partial y}by yy, then by xx

Key Point

Subscripts are read left to right, so in fyxf_{yx} we differentiate by yy first. The denominator of the \partial form is read right to left, so in 2fxy\dfrac{\partial^2 f}{\partial x\,\partial y} we also differentiate by yy first.

The notation extends to more variables. A function of three variables has first partials fxf_x, fyf_y and fzf_z, and fxz=(fx)zf_{xz} = \left(f_x\right)_z is the partial of fxf_x with respect to zz.

Practice questions

4 questions

For a function f(x,y)f(x,y), which of these describes

2fyx?\dfrac{\partial^2 f}{\partial y\,\partial x}?

Select the correct answer:

+ 3 more questions

What fxxf_{xx} and fyyf_{yy} measure

Explanation

For a function f(x,y)f(x,y) of two variables, the graph z=f(x,y)z=f(x,y) is a surface. The two second-order partials that repeat a variable, fxxf_{xx} and fyyf_{yy}, each describe one cross-section of that surface.

Key Point

Holding y=by=b fixed cuts the surface along the cross-section z=f(x,b)z=f(x,b), which is a function of xx alone. fxx(a,b)f_{xx}(a,b) is the ordinary second derivative of that cross-section at x=ax=a.

Similarly, fyy(a,b)f_{yy}(a,b) is the ordinary second derivative of the cross-section z=f(a,y)z=f(a,y) at y=by=b.

Each is the second derivative of a curve, so its sign gives the concavity of that cross-section.

Toggle the two views below to see the two cross-sections of one surface through the same point, first with yy held fixed and then with xx held fixed.

x
y
z
Holding y=0 fixed picks out the grey plane. Where it cuts the surface z=x2y2 is the red cross-section z=x2, a curve in x alone. Its second derivative at the origin is 2, so fxx(0,0)=2 and the curve is concave up there. Drag anywhere in the scene to rotate it.

Nothing here is new. Each of fxxf_{xx} and fyyf_{yy} is the second derivative we already know, applied to a cross-section of the surface.

Example

For f(x,y)=x3+xyy4f(x,y) = x^3 + xy - y^4, a calculation at the point (1,2)(1,2) gives

fxx(1,2)=6,fyy(1,2)=48.f_{xx}(1,2) = 6, \qquad f_{yy}(1,2) = -48.

Using these values, and without differentiating:

  1. Name the cross-section that fxx(1,2)f_{xx}(1,2) describes, and state its concavity there.
  2. Name the cross-section that fyy(1,2)f_{yy}(1,2) describes, and state its concavity there.

Solution

  1. Both subscripts of fxxf_{xx} are xx, so we hold y=2y = 2 fixed at both steps. That leaves the cross-section z=f(x,2)z = f(x,2), a function of xx alone, and fxx(1,2)f_{xx}(1,2) is its ordinary second derivative at x=1x = 1. That value is positive, so this cross-section is concave up at x=1x = 1.

  2. For fyyf_{yy} we hold x=1x = 1 fixed instead, which leaves the cross-section z=f(1,y)z = f(1,y), a function of yy alone, and fyy(1,2)f_{yy}(1,2) is its ordinary second derivative at y=2y = 2. That value is negative, so this cross-section is concave down at y=2y = 2.

Practice questions

4 questions

A calculation for a function f(x,y)f(x,y) gives fxx(3,2)=5f_{xx}(3,-2) = -5. Which statement follows?

Select the correct answer:

+ 3 more questions

What fxyf_{xy} and fyxf_{yx} measure

Explanation

The two remaining second-order partials are the mixed partials, fxyf_{xy} and fyxf_{yx}. Neither is the second derivative of a single cross-section, because the two steps use different variables. Instead, each compares one cross-section with its neighbours.

Intuition

Tip

Imagine the surface z=f(x,y)z=f(x,y) as a mountainside. You are standing on it at the point (a,b)(a,b), facing north, the +y+y direction. The ground in front of you has some steepness, and that steepness is fy(a,b)f_y(a,b), the slope of the cross-section z=f(a,y)z=f(a,y) at y=by=b.

Now take one step to your right, east, the +x+x direction, and face north again. The ground in front of you now belongs to a different cross-section, the one at a slightly larger value of xx, and its steepness can be different.

fyx(a,b)f_{yx}(a,b) measures that change. Since fyx=(fy)xf_{yx} = \left(f_y\right)_x, it differentiates the slope in the yy direction, fyf_y, with respect to xx.

Key Point

fyx(a,b)f_{yx}(a,b) is the rate at which fyf_y, the slope in the yy direction, changes as xx increases. In the same way, fxy(a,b)f_{xy}(a,b) is the rate at which fxf_x, the slope in the xx direction, changes as yy increases.

Slide the cross-section below to watch the slope in the yy direction change with xx.

x
y
z

Move the slider to slide the red cross-section of the surface z=xy along the x axis.

x0.5
Holding x=0.5 fixed, the red cross-section is the line z=0.5y.
Its slope in the y direction is fy(0.5,y)=0.5.

Increase x with the slider and the slope in the y direction increases with it. Decrease x and it decreases.

The rate of that change is the mixed partial fyx. Here fy=x, so fyx=1.

Drag anywhere in the scene to rotate it.

As with any derivative, the sign tells us the direction of the change. A positive fyx(a,b)f_{yx}(a,b) means the slope in the yy direction is increasing as xx increases, and a negative one means it is decreasing. A value of 00 means that, at (a,b)(a,b), moving in the xx direction is not changing the slope in the yy direction.

Example

For a function f(x,y)f(x,y), a calculation at the point (3,1)(3,1) gives

fy(3,1)=2,fyx(3,1)=5.f_y(3,1) = 2, \qquad f_{yx}(3,1) = -5.

Using these values, and without differentiating:

  1. State which first partial fyx(3,1)f_{yx}(3,1) tracks, and in which direction it tracks the change.
  2. Describe what the two values say about the surface z=f(x,y)z = f(x,y) at (3,1)(3,1).

Solution

  1. Subscripts are read left to right, so fyx=(fy)xf_{yx} = \left(f_y\right)_x: the first partial is fyf_y, and it is differentiated by xx. So fyx(3,1)f_{yx}(3,1) is the rate at which fyf_y changes as xx increases, with yy held fixed at 11.

  2. At (3,1)(3,1) the slope in the yy direction is fy(3,1)=2f_y(3,1) = 2, so the cross-section z=f(3,y)z = f(3,y) is rising at y=1y = 1. The value fyx(3,1)=5f_{yx}(3,1) = -5 is negative, so that slope is decreasing as xx increases. A small increase in xx brings us to a cross-section that is less steep in the yy direction at y=1y = 1.

Practice questions

4 questions

For a function f(x,y)f(x,y), a calculation gives fxy(4,1)=3f_{xy}(4,-1) = 3. Which statement follows?

Select the correct answer:

+ 3 more questions