Indexing and slicing arrays

Positions and slices, now in several dimensions

Arrays are indexed like lists: a[0] is the first element, a[-1] the last and a[start:stop:step] a slice that stops before stop. The difference is that an array can have several dimensions, and you give one index or slice per dimension, separated by commas: m[row, column]. A bare : means "all of this dimension", so m[:, 1] is the whole second column and m[1] (or m[1, :]) the whole second row. Negative numbers count from the end, and a step of -1 reverses.

import numpy as np

a = np.array([10., 11., 12., 13., 14., 15.])
print(a[0], a[-1])           # 10.0 15.0
print(a[1:4])                # [11. 12. 13.]
print(a[-2:])                # [14. 15.]
print(a[::2])                # [10. 12. 14.]
print(a[::-1])               # [15. 14. 13. 12. 11. 10.]

# Two dimensions: one index or slice per dimension
m = np.arange(12).reshape(3, 4)
print(m[1, 2])               # 6
print(m[:, 1])               # [1 5 9]      the whole second column
print(m[-1, :])              # [ 8  9 10 11] the last row
print(m[0:2, 1:3])           # [[1 2]
                             #  [5 6]]

Slices are views, not copies

Unlike a list slice, a NumPy basic slice does not copy the data: it is a view onto the same memory, so writing through it changes the original array. This makes slicing instant even for huge arrays, but it surprises people: part = a[1:4]; part[0] = 99 also changes a[1]. When you need an independent array, call .copy(). You can check with np.shares_memory(a, b). The same view behaviour holds for rows, columns, steps and reversed slices.

import numpy as np

a = np.array([10., 11., 12., 13., 14., 15.])

part = a[1:4]           # a view, no data copied
part[0] = 99
print(a)                # [10. 99. 12. 13. 14. 15.]   a changed too

safe = a[1:4].copy()    # an independent array
safe[0] = -1
print(a)                # [10. 99. 12. 13. 14. 15.]   unchanged

print(np.shares_memory(a, a[::2]))   # True
print(np.shares_memory(a, safe))     # False

Boolean masks and index lists

Comparing an array gives an array of booleans the same size: r > 0 is True where the return is positive. Using that mask as the index, r[r > 0], picks only those elements. Combine conditions with & (and), | (or) and ~ (not), each condition in parentheses, because Python's and/or do not work on arrays. A list or array of integers works too, a[[0, 2, 5]], picking exactly those positions in that order. Selection by mask or by index list copies the data, and you can also assign through a mask: a[a > 3] = 0.

import numpy as np

r = np.array([0.01, -0.02, 0.03, -0.01, 0.0, 0.02])
print(r > 0)                        # [ True False  True False False  True]
print(r[r > 0])                     # [0.01 0.03 0.02]
print(r[(r > 0) & (r < 0.025)])     # [0.01 0.02]    parentheses are required
print(np.where(r > 0)[0])           # [0 2 5]        positions where it holds

# Pick by position, in any order
w = np.array([3.0, 9.0, 1.0, 7.0])
order = np.argsort(-w)              # positions from the largest value down
print(order[:2], w[order[:2]])      # [1 3] [9. 7.]

# Assign through a mask
counts = np.array([1, 2, 3, 4, 5, 6])
counts[counts > 3] = 0
print(counts)                       # [1 2 3 0 0 0]

import numpy as np

# Rows are days, columns are assets (AAPL, MSFT, GOOG)
R = np.array([[ 0.02, -0.01,  0.03],
              [ 0.01,  0.00,  0.02],
              [-0.03,  0.02, -0.01],
              [ 0.00,  0.01,  0.04]])

print(R[:, [0, 2]])             # the AAPL and GOOG columns, as a copy
print(R[R[:, 0] > 0])           # the days on which AAPL rose (whole rows)
print(R[R[:, 0] > 0].shape)     # (2, 3)
print(R[-2:, 1].mean())         # 0.015  last two days of MSFT

• One index or slice per dimension: m[row, col], m[:, 1] (a column), m[-1] (the last row), a[::-1].
• Basic slices are views (writes show through; .copy() to detach); masks and index lists make copies.
• Masks: a[a > 0], combine with &, |, ~ in parentheses; np.argsort gives positions in sorted order.

Next up: vectorisation, replacing Python loops with whole-array arithmetic.

Lab · what's graded

• Take the most recent rows with last_rows25 pts
• Keep only the gains with positive_values25 pts
• Sample every k-th value with every_kth25 pts
• Find the positions of the k largest values with top_k_indices25 pts