CI/CD for quant code

Checks that run themselves

Tests only protect you if they are run, every time, by something that is not your memory. Continuous integration (CI) is a server that, on every push and pull request, checks out the code in a clean environment and runs the checks you defined: lint, type check, tests, a small backtest. If anything fails, the change is blocked. Continuous delivery/deployment (CD) goes further: when the checks pass on the main branch, the new version is built and released automatically. In quant work the stakes are concrete: a refactor that silently shifts a pricing result by a basis point should be caught by a machine before it reaches a portfolio.

# A test matrix: every operating system with every Python version, minus the combinations you skip
from itertools import product

os_list = ["linux", "macos", "windows"]
py_versions = ["3.11", "3.12"]
exclude = {("windows", "3.11")}

jobs = [(os_name, version) for os_name, version in product(os_list, py_versions) if (os_name, version) not in exclude]
print(len(jobs))                                     # 5
print(jobs[0], jobs[-1])                             # ('linux', '3.11') ('windows', '3.12')

What a quant pipeline checks

A typical pipeline has stages that fail fast: lint (style and obvious errors) and type checking run in seconds, then the unit tests, then slower checks such as a small regression backtest compared with a stored result. Make the checks deterministic: fixed random seeds, no network, no dependence on today's date, and data from a small committed snapshot rather than a live feed. Run them on a matrix of Python versions and operating systems you claim to support. To keep feedback quick, run only the tests that a change can affect (test selection), with a safe fallback to the whole suite when you cannot tell.

# Test selection: run only the tests a change can affect; an unknown file means "run everything"
def select_tests(changed_files, mapping):
    chosen = set()
    for path in changed_files:
        matches = [tests for prefix, tests in mapping.items() if path.startswith(prefix)]
        if not matches:
            return sorted({test for tests in mapping.values() for test in tests})
        for tests in matches:
            chosen.update(tests)
    return sorted(chosen)

mapping = {"src/pricing/": ["tests/test_bs.py", "tests/test_greeks.py"], "src/risk/": ["tests/test_var.py"]}
print(select_tests(["src/risk/var.py"], mapping))                    # ['tests/test_var.py']
print(select_tests(["src/risk/var.py", "setup.cfg"], mapping))       # all three tests: setup.cfg could affect anything

Gates, secrets and the workflow file

The pipeline is described in a file kept in the repository, for example .github/workflows/ci.yml for GitHub Actions: when it runs (on:), the jobs, the matrix and the steps, each step either uses: a ready-made action or run:s a command. Treat the file as code, reviewed like any other. Add a deploy gate: only the main branch deploys, and only when every required job succeeded (a skipped or missing job is not a success). Protect the branch so it cannot be merged without passing checks. Keep API keys and passwords in the CI system's secrets, never in the repository, and give each job only the permissions it needs. Release to a paper-trading or staging environment first.

# The deploy gate: main branch only, and every required job must have succeeded
def can_deploy(results, required, branch):
    return branch == "main" and all(results.get(job) == "success" for job in required)

print(can_deploy({"lint": "success", "tests": "success"}, ["lint", "tests"], "main"))     # True
print(can_deploy({"lint": "success", "tests": "skipped"}, ["lint", "tests"], "main"))     # False  skipped is not success
print(can_deploy({"lint": "success"}, ["lint", "tests"], "main"))                         # False  a missing job is not success
print(can_deploy({"lint": "success", "tests": "success"}, ["lint", "tests"], "dev"))      # False  only main deploys

# The workflow file is plain text, so it can be generated and checked like any other output
def workflow_yaml(name, python_versions, command):
    versions = ", ".join(f'"{version}"' for version in python_versions)
    lines = [f"name: {name}", "on: [push, pull_request]", "jobs:", "  test:", "    runs-on: ubuntu-latest",
             "    strategy:", "      matrix:", f"        python-version: [{versions}]", "    steps:",
             "      - uses: actions/checkout@v4", "      - uses: actions/setup-python@v5", "        with:",
             "          python-version: ${{ matrix.python-version }}", "      - run: pip install -e .[test]",
             f"      - run: {command}"]
    return "\n".join(lines) + "\n"

text = workflow_yaml("ci", ["3.11", "3.12"], "pytest -q")
print(text.splitlines()[7])                          #         python-version: ["3.11", "3.12"]
print(text.splitlines()[-1])                         #       - run: pytest -q

• CI runs lint, type checks and tests in a clean environment on every push; CD releases automatically only when they pass on main.
• Make checks deterministic (seeds, snapshots, no network), run them on a matrix of Python versions and systems, and select tests by what changed.
• Describe the pipeline in a reviewed workflow file, gate deploys on main and required jobs, keep secrets in the CI system, and fix flaky tests.

Next up: scheduled pipelines, running data and trading jobs at the right time, in the right order.

Lab · what's graded

• Expand the test matrix with matrix_jobs25 pts
• Run only what matters with select_tests25 pts
• Guard the release with can_deploy25 pts
• Generate the workflow with workflow_yaml25 pts