Packages and project layout

From a file to a folder of modules

A single file stops being enough once a project grows. A package is a folder of modules that Python can import as one unit. Mark the folder with a file named __init__.py (it can be empty) and every .py file inside becomes a module of the package, addressed with dots: quant/pricing.py is quant.pricing, and quant/data/loaders.py is quant.data.loaders. Folders inside a package are subpackages and need their own __init__.py.

# A small project on disk
#
# quant-tools/
#   pyproject.toml
#   README.md
#   quant/
#     __init__.py
#     pricing.py
#     data/
#       __init__.py
#       loaders.py
#   tests/
#     test_pricing.py

# quant/pricing.py
def mid(bid, ask):
    return (bid + ask) / 2

# anywhere else in the project
from quant.pricing import mid
import quant.data.loaders

print(mid(184.0, 185.0))  # 184.5

A layout that scales

Keep the importable code in one package folder named after the project, and keep everything else beside it, not inside: tests in tests/, a README.md that explains the project, a pyproject.toml that names it and lists its dependencies, and data or notebooks in their own folders. Test files are named test_*.py so tools like pytest find them. Small, focused modules with clear names (pricing.py, loaders.py) are easier to test and reuse than one long utils.py.

# pyproject.toml (names the project and its dependencies)
#
# [project]
# name = "quant-tools"
# version = "0.1.0"
# requires-python = ">=3.11"
# dependencies = ["numpy"]

# tests/test_pricing.py: found automatically because of the name
from quant.pricing import mid

def test_mid():
    assert mid(184.0, 185.0) == 184.5

Importing inside a package

Inside the package you can import by full path from the top (from quant.pricing import mid, an absolute import) or relative to the current module (from .pricing import mid, one dot for the same package, two for its parent, a relative import). Absolute imports are usually clearer. __init__.py runs when the package is first imported, so it can re-export the names users need (from .pricing import mid) and then import quant; quant.mid(...) works. Run package code with python -m quant.data.loaders, not python quant/data/loaders.py, so the dotted names resolve.

# quant/__init__.py: runs on the first "import quant"
from .pricing import mid          # relative: same package

__all__ = ["mid"]                 # what "from quant import *" would give

# quant/data/loaders.py
from quant.pricing import mid     # absolute: always starts at the top
from .. import pricing            # relative: two dots = the parent package

# In a terminal, from the project folder:
#   python -m quant.data.loaders   (runs the module as part of the package)
#   pytest -q                      (finds tests/test_*.py)

# A file path and its module name are two spellings of one thing
path = "quant/data/loaders.py"
name = path[:-3].replace("/", ".")
print(name)                          # quant.data.loaders
print(name.split(".")[0])            # quant  (the top-level package)

# __init__.py stands for its folder
print("quant/__init__.py"[:-3].replace("/", "."))  # quant.__init__ -> the package is "quant"

• A package is a folder with __init__.py; quant/data/loaders.py is the module quant.data.loaders.
• Keep code in the package; tests, README, pyproject.toml and data beside it.
• Prefer absolute imports; run package code with python -m package.module.

Next up: virtual environments and pip, so each project gets exactly the libraries it needs.

Lab · what's graded

• Turn a file path into a module name with module_name25 pts
• List a project's importable modules with find_modules25 pts
• Spot folders that lack an __init__.py with missing_init25 pts
• Write the import statement for a module with import_line25 pts