Threads and the GIL

Threads: concurrency inside one process

A thread is an independent line of execution inside one program, sharing its memory with the other threads. A trading program that must fetch quotes from ten venues, write a log and keep a UI responsive uses threads so that waiting for one does not stop the rest. Python's threading module and, more conveniently, concurrent.futures.ThreadPoolExecutor start and manage them. Because the threads share variables, they are fast to communicate and dangerous to get wrong.

# Ten slow "network calls" in a thread pool wait at the same time, so the total is about one call, not ten
import time
from concurrent.futures import ThreadPoolExecutor

def fetch_quote(symbol):
    time.sleep(0.1)                                  # stands in for waiting on a venue
    return symbol, len(symbol) * 10.0

symbols = ["AAPL", "MSFT", "NVDA", "TSLA", "AMZN", "META", "GOOG", "NFLX", "ORCL", "INTC"]

start = time.perf_counter()
with ThreadPoolExecutor(max_workers=10) as pool:
    quotes = list(pool.map(fetch_quote, symbols))
elapsed = time.perf_counter() - start

print(quotes[0], len(quotes))                        # ('AAPL', 40.0) 10
print(elapsed < 0.5)                                 # True  about 0.1 s in total, not 10 x 0.1 s

The GIL: threads that wait, and threads that compute

CPython has a Global Interpreter Lock (GIL): only one thread executes Python bytecode at any moment. A thread gives it up while it waits for the network, a file or time.sleep, so threads overlap waiting very well: ten threads fetching quotes finish in about the time of one. But for CPU-bound Python code (a pure-Python loop doing arithmetic) the threads take turns, so four threads are no faster than one, and often a little slower. Code that runs outside Python, such as NumPy or a C extension that releases the GIL, can run in parallel in threads. For heavy pure-Python computation, use separate processes (next topic), or the free-threaded build of Python 3.13+ that removes the GIL, where available.

# Splitting a list into chunks for workers: sizes differ by at most one, the first chunks get the extra items
def split_chunks(items, workers):
    size, extra = divmod(len(items), workers)
    chunks, start = [], 0
    for index in range(workers):
        stop = start + size + (1 if index < extra else 0)
        chunks.append(list(items[start:stop]))
        start = stop
    return chunks

chunks = split_chunks(list(range(7)), 3)
print(chunks)                                        # [[0, 1, 2], [3, 4], [5, 6]]
print([sum(chunk) for chunk in chunks])              # [3, 7, 11]   one partial sum per worker
print(sum(sum(chunk) for chunk in chunks))           # 21

Shared data, race conditions and locks

Threads that update the same variable can corrupt it, because total += amount is three steps (read, add, write) and a thread switch may happen between them: two threads both read 100, both add 5, both write 105, and one update is lost. This is a race condition, and it appears rarely and unpredictably, which makes it the worst kind of bug. The fix is a lock: with lock: lets one thread at a time run the block, and the read and the write must both be inside it. Better still, avoid sharing: give each worker its own chunk and its own result, and combine the results at the end, or pass work and results through a queue.Queue, which is already thread-safe.

# A lock makes the read-modify-write a single step; the total comes out exact
import threading

totals = {"AAPL": 0}
lock = threading.Lock()

def safe_add(shared, key, amount, lock):
    with lock:
        shared[key] = shared.get(key, 0) + amount
        return shared[key]

def worker():
    for _ in range(10_000):
        safe_add(totals, "AAPL", 1, lock)

threads = [threading.Thread(target=worker) for _ in range(4)]
for thread in threads:
    thread.start()
for thread in threads:
    thread.join()

print(totals["AAPL"])                                # 40000  nothing lost: four threads x 10,000 additions

# A back-of-envelope estimate before writing any threaded code
import math

def estimate_time(n_tasks, seconds, workers, kind):
    if kind in ("io", "processes"):
        return math.ceil(n_tasks / workers) * seconds
    return n_tasks * seconds                         # "cpu_threads": the GIL makes the threads take turns

print(estimate_time(40, 0.5, 8, "io"))               # 2.5    forty slow requests, eight at a time
print(estimate_time(40, 0.5, 8, "cpu_threads"))      # 20.0   forty CPU-bound tasks in threads: no gain
print(estimate_time(40, 0.5, 8, "processes"))        # 2.5    separate processes use separate cores

• Threads share memory and overlap waiting well (network, files); one ThreadPoolExecutor can fetch from many venues at once.
• The GIL lets one thread run Python bytecode at a time: no speed-up for CPU-bound pure Python, but NumPy and C extensions can release it.
• Protect shared updates with with lock: around the read and the write, prefer not sharing (chunks, queues), and watch for deadlocks.

Next up: multiprocessing, using several processes to put every CPU core to work.

Lab · what's graded

• Share the work out with split_chunks25 pts
• Map a function over the chunks with run_in_chunks25 pts
• Protect shared data with safe_add25 pts
• Know what threads buy you with estimate_time25 pts