Packaging a Python library

From a folder of scripts to an installable library

Code that lives in a folder of scripts works on one machine, for one person, until the day someone else needs it or you need it in a different environment. Packaging turns your code into a library with a name, a version and declared dependencies that anyone can install with pip install. It is how a team shares a pricing library without copying files around, how a research notebook and a production job run exactly the same code, and how a bug fix reaches everyone through a new version number.

# Versions are numbers, not text: compare them as tuples
def parse_version(text):
    parts = [int(part) for part in text.split(".")]
    return tuple(parts + [0] * (3 - len(parts)))

print("1.10" > "1.9")                                # False  as text, "1" < "9" decides it: wrong
print(parse_version("1.10") > parse_version("1.9"))  # True   as numbers (1, 10, 0) is newer than (1, 9, 0)
print(parse_version("2"))                            # (2, 0, 0)

Layout, pyproject.toml and building

A common layout puts the code in src/yourlib/ (with an __init__.py), the tests in tests/, and a pyproject.toml at the root. That one file names the build tool (the [build-system] table) and describes the project: its name, version, a description, the Python versions it supports and its dependencies. pip install -e . installs the folder in editable mode while you develop, so edits take effect at once; python -m build produces a wheel (.whl, ready to install) and an sdist (source archive) in dist/. Put the version in one place only, and expose it as yourlib.__version__ if users need it.

# A minimal pyproject.toml, generated and then read back with tomllib to prove it is valid
import tomllib

def pyproject_toml(name, version, dependencies):
    lines = ["[build-system]", 'requires = ["setuptools>=68"]', 'build-backend = "setuptools.build_meta"', "",
             "[project]", f'name = "{name}"', f'version = "{version}"']
    if dependencies:
        lines += ["dependencies = ["] + [f'    "{d}",' for d in dependencies] + ["]"]
    else:
        lines.append("dependencies = []")
    return "\n".join(lines) + "\n"

text = pyproject_toml("qcpricing", "0.3.0", ["numpy>=1.26", "pandas>=2"])
data = tomllib.loads(text)
print(data["project"]["name"], data["project"]["version"])      # qcpricing 0.3.0
print(data["project"]["dependencies"])                          # ['numpy>=1.26', 'pandas>=2']

Versions and version specifiers

Most libraries follow semantic versioning: MAJOR.MINOR.PATCH. A bug fix bumps the patch (1.4.2 to 1.4.3), a backwards-compatible feature bumps the minor and resets the patch (1.5.0), and a breaking change bumps the major (2.0.0). Dependencies are declared with specifiers: numpy>=1.26,<3 accepts any NumPy from 1.26 up to but not including 3. Be as loose as you can honestly be: a lower bound for the oldest version you tested, an upper bound only for a major version you know breaks you. Pinning exact versions (==) belongs in the lock file of an application, not in a library other people install.

# Bumping a version and checking a requirement, with versions compared as tuples
import operator

def parse_version(text):
    parts = [int(part) for part in text.split(".")]
    return tuple(parts + [0] * (3 - len(parts)))

def bump_version(version, part):
    major, minor, patch = parse_version(version)
    return {"major": f"{major + 1}.0.0", "minor": f"{major}.{minor + 1}.0", "patch": f"{major}.{minor}.{patch + 1}"}[part]

OPS = {">=": operator.ge, "<": operator.lt, "==": operator.eq}

def satisfies(version, specifier):
    for clause in specifier.split(","):
        symbol = next(op for op in (">=", "==", "<") if clause.startswith(op))
        if not OPS[symbol](parse_version(version), parse_version(clause[len(symbol):])):
            return False
    return True

print(bump_version("1.4.2", "minor"))                # 1.5.0
print(satisfies("1.26.4", ">=1.26,<3"), satisfies("3.0.0", ">=1.26,<3"))      # True False

# One place for the version: read it from the source instead of repeating it in several files
import re

init_py = """__version__ = "0.3.0"
from .pricing import price
"""
match = re.search(r'^__version__ = "([^"]+)"', init_py, re.MULTILINE)
version = match.group(1)
print(version)                                       # 0.3.0
print(tuple(int(part) for part in version.split(".")))      # (0, 3, 0)

• Packaging gives code a name, a version and declared dependencies, installable with pip install; develop with pip install -e ., build with python -m build.
• pyproject.toml holds the build system and the [project] metadata; compare versions as number tuples, never as text.
• Use semantic versioning (MAJOR.MINOR.PATCH) and loose specifiers (numpy>=1.26,<3); never change a published version, and keep secrets out of the package.

Next up: CI/CD for quant code, running the checks automatically on every change.

Lab · what's graded

• Compare versions as numbers with parse_version25 pts
• Release the next version with bump_version25 pts
• Check a requirement with satisfies25 pts
• Describe the package with pyproject_toml25 pts