The Jacobian and Hessian

Many outputs, many inputs

A function $F$ can map several inputs to several outputs, for example risk factors to the values of several positions. Its Jacobian $J$ is the matrix of all first partial derivatives, with entry $J_{ij}=\frac{\partial F_i}{\partial x_j}$. It is the best linear approximation: for a small move $\Delta$, $F(\mathbf{x}+\Delta)\approx F(\mathbf{x})+J\Delta$.

For one output $f$, the second derivatives form the Hessian $H$, with $H_{ij}=\frac{\partial^2 f}{\partial x_i\partial x_j}$. For smooth functions it is symmetric. It describes curvature: $f(\mathbf{x}+\Delta)\approx f+\nabla f\cdot\Delta+\tfrac{1}{2}\Delta^{T}H\Delta$. Where $\nabla f=0$, a positive definite $H$ marks a local minimum, a negative definite $H$ a local maximum, and a mixed one a saddle point. Newton's method for optimisation uses $H$ to take better steps than plain gradient descent.

• Jacobian: the matrix of first partials of a vector function; the best linear model of it.
• Hessian: the symmetric matrix of second partials of a scalar function; its curvature.
• Positive definite Hessian at a critical point: a local minimum.

Lab · what's graded

• The Jacobian at a point20 pts
• The Hessian at a point20 pts
• The determinant of the Jacobian15 pts
• Reading a Hessian15 pts
• The linear approximation15 pts
• What each matrix describes15 pts