Expected shortfall

What VaR does not say

A 99% VaR of 3% says that on 99 days out of 100 you lose less than 3%. It says nothing about the 100th day: it could be 3.1% or 30%. Expected shortfall (ES, CVaR) closes the gap. It is the average loss on the days when the VaR is exceeded, so it describes the shape of the tail. Because it averages, it also changes smoothly, and unlike VaR it rewards diversification in every case, which is why Basel's market-risk rules moved from VaR to ES.

import numpy as np

rng = np.random.default_rng(5)
returns = rng.standard_t(4, size=1000) * 0.01

cutoff = np.quantile(returns, 0.05)                   # the 95% VaR is minus this
tail = np.sort(returns[returns <= cutoff])            # the worst 5% of the days, lowest first
print(len(tail), round(float(-cutoff), 5))            # 50 days; the VaR
print(np.round(tail[:4], 4))                          # the four worst days

Averaging the tail

Historically, ES is straightforward: select the returns at or below the VaR cutoff, average them, and flip the sign. The result is never smaller than the VaR, because every tail return is at least as bad as the cutoff. The ratio ES / VaR tells how heavy the tail is. For a normal distribution it is about 1.2 at 95%, and fat-tailed data usually show a larger ratio.

import numpy as np

def tail_returns(returns, confidence):
    cutoff = np.quantile(returns, 1 - confidence)
    return np.sort(returns[returns <= cutoff])

def historical_es(returns, confidence):
    return float(-tail_returns(returns, confidence).mean())

rng = np.random.default_rng(5)
returns = rng.standard_t(4, size=1000) * 0.01
for confidence in (0.95, 0.99):
    var = -np.quantile(returns, 1 - confidence)
    es = historical_es(returns, confidence)
    print(confidence, round(float(var), 5), round(es, 5), round(es / float(var), 3))

A formula for the normal case

For a normal return the average of the tail has a closed form: ES = sigma * pdf(z) / (1 - confidence) - mu, where z is the quantile used for the VaR and pdf the standard normal density at z. It turns the VaR calculation into a two-line function, and it is exact only if the data are normal, so comparing it with the historical ES is a quick test of how far the data depart from normality.

from statistics import NormalDist

def normal_es(mu, sigma, confidence):
    z = NormalDist().inv_cdf(confidence)
    return sigma * NormalDist().pdf(z) / (1 - confidence) - mu

for confidence in (0.95, 0.975, 0.99):
    var = NormalDist().inv_cdf(confidence)                        # in standard deviations, mu = 0
    es = normal_es(0.0, 1.0, confidence)
    print(confidence, round(var, 3), round(es, 3), round(es / var, 3))       # ES/VaR: 1.254, 1.193, 1.146

# The formula against a simulated tail
import numpy as np
from statistics import NormalDist

rng = np.random.default_rng(0)
sample = rng.normal(0.0, 0.01, size=1_000_000)
tail = sample[sample <= np.quantile(sample, 0.01)]
simulated = -tail.mean()
z = NormalDist().inv_cdf(0.99)
formula = 0.01 * NormalDist().pdf(z) / 0.01
print(round(float(simulated), 5), round(formula, 5))         # both close to 0.02665

• ES is the average loss in the tail beyond the VaR: minus the mean of the returns at or below the (1 - confidence) quantile.
• It is never below the VaR; ES / VaR measures the tail's weight, and the normal formula is sigma * pdf(z) / (1 - confidence) - mu.
• Report VaR as the threshold and ES as the severity; ES is noisier because it rests on few tail observations.

Next up: stress testing, asking what a portfolio would lose in a specific, extreme scenario.

Lab · what's graded

• Isolate the worst days with tail_returns25 pts
• Average the tail with historical_es25 pts
• Get it from a formula with normal_es25 pts
• Compare tail and threshold with es_var_ratio25 pts