Testing with pytest

Why test, and what a test looks like

In trading, a bug is not an exception on a screen; it is a wrong number that looks plausible: a sign flipped in a P&L, a fee applied twice, a signal that quietly uses tomorrow's price. Tests are small programs that check your code against answers you know, so such a bug is caught on your machine instead of in the market. pytest finds functions named test_*, runs each one and reports the ones where an assert fails. A test needs no framework to read: arrange some input, call the function, assert what you expect.

# A test is a function with an assert; pytest runs every test_* function it finds
def net_pnl(entry, exit_price, quantity, fee):
    return (exit_price - entry) * quantity - fee

def test_net_pnl_long_winner():
    assert net_pnl(100.0, 102.0, 50, 1.0) == 99.0

def test_net_pnl_short_loser():
    assert net_pnl(100.0, 102.0, -50, 1.0) == -101.0

for test in (test_net_pnl_long_winner, test_net_pnl_short_loser):
    test()                                           # pytest does this for you and reports failures
    print(test.__name__, "passed")

Floats, errors and many cases

Floats are approximate, so 0.1 + 0.2 == 0.3 is false. Compare numbers with a tolerance: pytest.approx(expected) accepts a small relative difference (default one part in a million, plus a tiny absolute floor of 1e-12), and you can add an absolute tolerance for values near zero, where a relative one is useless. To check that bad input is rejected, pytest.raises(ValueError) passes only if that exception happens, and fails if none (or another one) does. To run one test over many cases, @pytest.mark.parametrize feeds a table of inputs and expected values, reports each row separately, and makes adding a case a one-line change.

# Floats need a tolerance: approx accepts a tiny relative difference, == does not
import pytest

print(0.1 + 0.2 == 0.3)                              # False
print(0.1 + 0.2 == pytest.approx(0.3))               # True
print(1e-10 == pytest.approx(0.0))                   # False  near zero a relative tolerance cannot help
print(1e-10 == pytest.approx(0.0, abs=1e-9))         # True   a larger absolute tolerance does

# raises: the test passes only if the expected exception happens
import pytest

def leverage(gross, equity):
    if equity <= 0:
        raise ValueError("equity must be positive")
    return gross / equity

with pytest.raises(ValueError) as info:
    leverage(1000.0, 0.0)
print("raised:", info.value)                         # raised: equity must be positive

try:
    with pytest.raises(ValueError):
        leverage(1000.0, 500.0)                      # no exception: pytest.raises itself fails
except BaseException as outcome:
    print(type(outcome).__name__)                    # Failed

# parametrize in miniature: one function, a table of cases, each row judged on its own
def fee(notional, rate=0.001, minimum=1.0):
    return max(notional * rate, minimum)

cases = [
    ((100_000.0,), 100.0),
    ((500.0,), 1.0),                                 # the minimum fee applies
    ((2_000.0, 0.002), 4.0),
    ((0.0,), 1.0),
]
for index, (args, expected) in enumerate(cases):
    got = fee(*args)
    print(index, "ok" if abs(got - expected) <= 1e-9 * abs(expected) else f"FAIL got {got}")

Fixtures, seeds and what to test

A fixture is shared setup that tests ask for by name: a small price series, a seeded random generator, a temporary file. With random data, fix the seed (or print it) so a failure can be reproduced: a test that fails once in a hundred runs and cannot be replayed is worse than none. Test the behaviour you care about rather than the internals: known cases by hand, edge cases (empty input, zero, one element), invariants that must always hold (a probability is between 0 and 1, buying then selling at the same price loses exactly the fees), and a regression test for every bug you fix. When you can state a rule that must hold for every input, searching many random inputs for a counterexample is a cheap and powerful test.

# A seed makes a random test replayable; a property is searched for a counterexample
import random

def find_counterexample(prop, inputs):
    for value in inputs:
        if not prop(value):
            return value
    return None

rng = random.Random(42)                              # fixed seed: the same inputs on every run
returns = [round(rng.uniform(-0.05, 0.05), 4) for _ in range(1000)]

def compound_is_positive_growth(r):
    return (1 + r) > 0                               # a daily return above -100% keeps the value positive

print(find_counterexample(compound_is_positive_growth, returns))      # None  the rule holds for all 1000
print(find_counterexample(lambda r: r > -0.049, returns))             # a return that breaks a wrong rule

• A test arranges input, calls the function and asserts the result; pytest runs every test_* function and reports failures.
• Compare floats with pytest.approx (add an absolute tolerance near zero), expect errors with pytest.raises, and use parametrize for tables of cases.
• Fix random seeds, test edge cases, invariants and fixed bugs, search for counterexamples, and make each test fail once before trusting it.

Next up: profiling Python code, measuring where the time goes before trying to make it faster.

Lab · what's graded

• Compare floats with a tolerance in is_close25 pts
• Run a table of cases with run_cases25 pts
• Expect an error with raises25 pts
• Hunt for a failure with find_counterexample25 pts