Selecting data with loc and iloc

Two ways to point at data: by label and by position

A DataFrame can be addressed in two ways, and mixing them up is the commonest pandas bug. .loc selects by label: row labels from the index, column labels from the column names. .iloc selects by integer position, counted from 0, as in a list or NumPy array. Both take [rows, columns]: df.loc["AAPL", "price"] is one value, df.iloc[0, 0] is the value in the first row and first column. Leave out the second part to take whole rows: df.loc["AAPL"] is the AAPL row as a Series.

import pandas as pd

df = pd.DataFrame(
    {"price": [184.25, 410.2, 171.0, 251.0],
     "volume": [200, 50, 300, 120],
     "sector": ["tech", "tech", "tech", "auto"]},
    index=["AAPL", "MSFT", "GOOG", "TSLA"],
)

print(df.loc["AAPL", "price"], df.iloc[0, 0])     # 184.25 184.25   same cell, two addresses
print(df.loc["TSLA"].to_dict())                    # {'price': 251.0, 'volume': 120, 'sector': 'auto'}
print(df.iloc[-1].to_dict())                       # the same row: position -1 is the last
print(df.loc[["TSLA", "AAPL"], "price"])           # a list of labels picks in that order
# TSLA    251.00
# AAPL    184.25
# Name: price, dtype: float64

try:
    df.loc["XXX", "price"]                         # a label that does not exist
except KeyError as error:
    print("KeyError", error)                       # KeyError 'XXX'

Slices: label slices include the end

Slices behave differently in the two systems. A position slice df.iloc[1:3] follows the Python rule and excludes the end: positions 1 and 2. A label slice df.loc["MSFT":"TSLA"] includes both end labels, because there is no natural "one past" for a label. Label slices follow the current order of the index, which need not be sorted. For columns, pass a list (["price", "volume"]) to pick them in the order you give. Forgetting which system you are in gives off-by-one results without any error.

import pandas as pd

df = pd.DataFrame(
    {"price": [184.25, 410.2, 171.0, 251.0], "volume": [200, 50, 300, 120]},
    index=["AAPL", "MSFT", "GOOG", "TSLA"],
)

print(df.iloc[1:3])                              # positions 1 and 2: the end is excluded
#        price  volume
# MSFT   410.2      50
# GOOG   171.0     300

print(df.loc["MSFT":"TSLA", ["volume", "price"]])    # labels MSFT..TSLA: the end is included
#       volume  price
# MSFT      50  410.2
# GOOG     300  171.0
# TSLA     120  251.0

print(df.iloc[:, 0].tolist())                    # the first column, by position
# [184.25, 410.2, 171.0, 251.0]

Conditions and assignment with loc

.loc also accepts a Boolean mask for the rows: df.loc[df["volume"] > 100, "price"] is the price column of the rows that pass the test, in one step. And .loc is how you assign to a chosen part: df.loc["MSFT", "price"] = 1.0 changes one cell and df.loc[df["volume"] < 100, "volume"] = 0 changes all rows that match. Avoid chained forms like df[df["volume"] < 100]["volume"] = 0: they work on a temporary copy. In pandas 3 a derived frame never changes the original, so to modify a table inside a function, first df.copy(), assign with .loc, and return the copy.

import pandas as pd

df = pd.DataFrame(
    {"price": [184.25, 410.2, 171.0, 251.0], "volume": [200, 50, 300, 120]},
    index=["AAPL", "MSFT", "GOOG", "TSLA"],
)

print(df.loc[df["volume"] > 100, "price"])       # a Boolean mask for rows, a label for the column
# AAPL    184.25
# GOOG    171.00
# TSLA    251.00
# Name: price, dtype: float64

def with_price(frame, symbol, price):
    out = frame.copy()                           # never edit the caller's table
    out.loc[symbol, "price"] = price
    return out

changed = with_price(df, "MSFT", 1.0)
print(changed.loc["MSFT", "price"], df.loc["MSFT", "price"])   # 1.0 410.2

fixed = df.copy()
fixed.loc[fixed["volume"] < 100, "volume"] = 0   # assign to all matching rows at once
print(fixed["volume"].tolist())                  # [200, 0, 300, 120]

import pandas as pd

# Integer labels are labels, not positions: a classic trap
d = pd.DataFrame({"x": [1.0, 2.0, 3.0]}, index=[10, 20, 30])

print(d.loc[20, "x"], d.iloc[0, 0])        # 2.0 1.0    label 20 is the second row
print(d.loc[10:20]["x"].tolist())          # [1.0, 2.0] labels 10 through 20, end included
print(d.iloc[0:2]["x"].tolist())           # [1.0, 2.0] positions 0 and 1, end excluded

try:
    d.loc[0, "x"]                          # there is no label 0, even though position 0 exists
except KeyError as error:
    print("KeyError", error)               # KeyError 0

• df.loc[row_label, column_label] by label; df.iloc[row_pos, col_pos] by position from 0.
• iloc slices exclude the end; loc label slices include it; a missing label raises KeyError.
• df.loc[mask, "col"] filters and picks; assign through .loc on a .copy() and return it.

Next up: groupby and aggregation, splitting a table into groups and summarising each one.

Lab · what's graded

• Look up one price by symbol with price_of25 pts
• Take the rows between two symbols with label_range25 pts
• Take rows by position with position_range25 pts
• Correct one price in a copy with set_price25 pts