Cython

Python with C types, compiled

Cython is a compiler for a language that is Python plus optional C type declarations. You write a .pyx file, Cython translates it into C, a C compiler turns that into an extension module, and you import it like any other module. Plain Python code compiled this way already runs a little faster, but the real gain comes when you declare types: cdef double total tells Cython that total is a machine number, not a Python object, so the loop works on raw numbers with no interpreter overhead. Heavy numeric loops can become fifty to a hundred times faster, like Numba, but you get an ordinary compiled module you can ship.

# What a Cython file looks like: Python with cdef type declarations (shown here as text)
pyx = """\
cpdef double dot(double[:] a, double[:] b):
    cdef double total = 0.0
    cdef long i
    for i in range(a.shape[0]):
        total += a[i] * b[i]
    return total
"""
print(pyx.splitlines()[0])                           # cpdef double dot(double[:] a, double[:] b):
print(sum("cdef" in line for line in pyx.splitlines()))     # 2   two typed declarations inside the function

Declaring types: cdef, cpdef and memoryviews

cdef declares a C variable (cdef double total, cdef long i) or a function that only C code can call. cpdef declares a function callable from both Python and C, which is what you want for the entry points of a module. def stays an ordinary Python function. Arrays are passed as typed memoryviews, such as double[:] for a one-dimensional array of doubles: they accept NumPy arrays without copying and index at C speed. A signature like cpdef double mean(double[:] x, long n) therefore replaces the dynamic types of Python with exact ones, and a type that is left out, or declared object, stays a slow Python object.

# Building the declarations as text: one table from Python type names to C types
C_TYPES = {"int": "long", "float": "double", "bool": "bint", "array[float]": "double[:]"}

def c_type(py_type):
    return C_TYPES.get(py_type, "object")

def cdef_signature(name, params, returns):
    result = "void" if returns is None else c_type(returns)
    arguments = ", ".join(f"{c_type(t)} {p}" for p, t in params)
    return f"cpdef {result} {name}({arguments})"

print(cdef_signature("mean", [("x", "array[float]"), ("n", "int")], "float"))    # cpdef double mean(double[:] x, long n)
print(cdef_signature("log_event", [("message", "dict")], None))                  # cpdef void log_event(object message)

Building the module and finding what to type

A setup.py with cythonize(["pricing.pyx"]) tells setuptools to translate and compile the file; python setup.py build_ext --inplace produces a shared library next to your code, and import pricing loads it. For quick experiments pyximport compiles on import. To see what is still slow, run cython -a pricing.pyx: it writes an HTML page where each line is shaded yellow by how much Python machinery it uses, and the white lines are pure C. Type the variables in the yellow lines inside your hot loops, then turn off checks you have proven unnecessary with @cython.boundscheck(False) and @cython.wraparound(False) so indexing is as cheap as in C.

# The build script is a few lines of text; this generates it for a list of modules
def build_setup(modules):
    files = ", ".join(f'"{module}.pyx"' for module in modules)
    return "from setuptools import setup\nfrom Cython.Build import cythonize\n\n" + f"setup(ext_modules=cythonize([{files}]))\n"

print(build_setup(["pricing", "risk"]), end="")
# from setuptools import setup
# from Cython.Build import cythonize
#
# setup(ext_modules=cythonize(["pricing.pyx", "risk.pyx"]))

# The loop a .pyx file would hold, as plain Python: same result, compiled once types are added
def dot(a, b):
    total = 0.0
    for i in range(len(a)):
        total += a[i] * b[i]
    return total

prices = [184.25, 410.5, 99.0]
quantities = [100.0, 50.0, 200.0]
print(dot(prices, quantities))                       # 58750.0  the compiled version returns exactly this number
print(dot(prices, quantities) == sum(p * q for p, q in zip(prices, quantities)))     # True

• Cython compiles a .pyx file (Python plus optional C types) to C and then to an extension module you import normally.
• cdef declares C variables and C-only functions, cpdef a function callable from Python and C; arrays are typed memoryviews such as double[:].
• Build with cythonize in setup.py, use cython -a to find the yellow (slow) lines in hot loops, and verify against the plain Python result.

Next up: calling C++ with pybind11, exposing existing C++ code to Python.

Lab · what's graded

• Map Python types to C with c_type25 pts
• Write a function header with cdef_signature25 pts
• Type the locals with declare_locals25 pts
• Write the build script with build_setup25 pts