NumPy arrays and dtypes

Why arrays, and what an array is

A Python list can hold anything, which makes it flexible but slow for numbers: every element is a separate object. NumPy adds the ndarray, a block of memory holding numbers of one single type laid out one after another. Operating on the whole block in compiled code is typically tens of times faster than a Python loop, and it uses far less memory. Almost every quantitative library (pandas, SciPy, scikit-learn) is built on it. Import it by convention as import numpy as np.

import numpy as np

# Create arrays
prices = np.array([184.25, 185.0, 183.2])
print(prices)                       # [184.25 185.   183.2 ]
print(np.zeros(3))                  # [0. 0. 0.]
print(np.arange(0, 10, 3))          # [0 3 6 9]
print(np.linspace(0, 1, 5))         # [0.   0.25 0.5  0.75 1.  ]

# An array knows its shape
table = np.array([[1, 2, 3], [4, 5, 6]])
print(table.shape, table.ndim, table.size)   # (2, 3) 2 6

Creating arrays

np.array(list) converts a list (or a list of lists for two dimensions). np.zeros(n) and np.ones(n) make filled arrays, and np.zeros((rows, cols)) takes a shape tuple. np.arange(start, stop, step) works like range but allows float steps and excludes stop; because of rounding, prefer np.linspace(start, stop, num), which gives exactly num evenly spaced points and includes both ends. Every array has .shape (the size in each dimension), .ndim (number of dimensions) and .size (total number of elements).

import numpy as np

# arange with floats can surprise; linspace gives exactly the points you ask for
print(len(np.arange(0, 1, 0.1)))     # 10  (1.0 is excluded)
print(len(np.linspace(0, 1, 11)))    # 11  (both ends included)

# Shapes and sizes
grid = np.zeros((3, 4))
print(grid.shape, grid.ndim, grid.size)   # (3, 4) 2 12
print(np.ones(2) * 5)                      # [5. 5.]

dtypes: one type for the whole array

Every array has a dtype: int64 (whole numbers), float64 (decimals), bool, and smaller or bigger variants such as float32 or int8. NumPy picks the dtype from the data: integers give int64, a single decimal turns everything into float64, and np.nan (not-a-number, the marker for a missing value) is a float, so it forces float64. Choose a dtype yourself with dtype=..., or convert with .astype(...), which returns a new array. A smaller dtype saves memory (float32 takes half of float64), at the price of precision.

import numpy as np

print(np.array([1, 2, 3]).dtype)            # int64
print(np.array([1, 2.5, 3]).dtype)          # float64  (one decimal upgrades everything)
print(np.array([True, False]).dtype)        # bool
print(np.array([1, 2, np.nan]))             # [ 1.  2. nan]  (nan is a float)

# astype returns a new array; smaller dtypes use less memory
prices = np.zeros(1000)
print(prices.nbytes)                        # 8000
print(prices.astype(np.float32).nbytes)     # 4000
print(np.array(["1.5", "2"], dtype=float))  # [1.5 2. ]  (text converted to numbers)

Casting traps: truncation and overflow

Converting decimals to integers with .astype(int) truncates toward zero, and floats cannot represent most decimals exactly, so 19.99 * 100 is 1998.9999999999998 and truncates to 1998, one cent short. To convert prices to whole cents, round first: np.round(prices * 100).astype(np.int64). Integer dtypes also have a fixed range and wrap around silently: an int8 holds -128 to 127, so 100 * 2 becomes -56. Use int64 for money and counts, and remember that dividing integers with / always gives floats while // floors.

import numpy as np

# Truncation: astype(int) cuts the decimals off
print(np.array([1.9, -1.9]).astype(int))              # [ 1 -1]
print(19.99 * 100, int(19.99 * 100))                  # 1998.9999999999998 1998
print(np.round(np.array([19.99]) * 100).astype(np.int64))   # [1999]

# Overflow: an int8 wraps around silently
small = np.array([100, 50], dtype=np.int8)
print(small * 2)                                      # [-56 100]

# / always gives floats, // floors
counts = np.array([1, 2, 3])
print(counts / 2, counts // 2)                        # [0.5 1.  1.5] [0 1 1]

• np.array, zeros, ones, arange, linspace create arrays; .shape, .ndim, .size describe them.
• One dtype per array: int64, float64, bool; astype converts into a new array.
• astype(int) truncates (round first); small integer dtypes wrap around silently; nan forces floats.

Next up: indexing and slicing arrays, picking out rows, columns, windows and values that meet a condition.

Lab · what's graded

• Lay out evenly spaced points with make_grid25 pts
• Turn mixed raw values into floats with to_float_array25 pts
• Report shape, size and dtype with describe_array25 pts
• Convert prices to whole cents safely with to_cents25 pts