Chapter 33 — Type Hints: Labelling Data, Checking Before Running

Series I4 — Advanced Python Language

NoteIn this chapter, you will:

• Write type annotations (type hints) on variables and functions
• Use types for collections, choices (X | None), functions, and generics
• Meet TypedDict, Literal, and Protocol
• Run the real mypy and read its output
• Understand a Python 3.14 change: annotations are evaluated later

All scripts for this chapter are in the kode/ch33/ folder. On the web version, you can also download them all at once (zip).

Imagine a feed store with unlabelled sacks. You must open each sack to know what is inside, and sometimes you grab the wrong one. Now imagine every sack labelled “fish pellets, 25 kg”. You know at once, and an inspector can flag a misplaced sack before the feed is used.

A type hint is that label on code: a note about the kind of data a variable or function expects. Python itself does not enforce it while the program runs, but people, editors, and checking tools can use it to find mistakes before the program runs.

%%{init: {"flowchart": {"diagramPadding": 40}}}%%
flowchart LR
  A["Code with<br/>type hints"] --> B["Type checker<br/>(mypy / pyright)"]
  B --> C{"Anything<br/>mismatched?"}
  C -- "yes" --> D["List of mistakes,<br/>program not run yet"]
  C -- "no" --> E["Go ahead and<br/>run the program"]

Type hints are used by checking tools (mypy, pyright) to find mistakes before the program runs.

Basic annotations

01_basic_annotations.py

# Chapter 33 - Type annotations: notes for humans and tools, not enforcement
def biomass_kg(count: int, weight_g: float) -> float:
    return count * weight_g / 1000


age_days: int = 120
pond_name: str = "A"

print(biomass_kg(800, 540.5))
print(biomass_kg.__annotations__)


def double(count: int) -> int:
    return count * 2


# Python does NOT check annotations while the program runs
print(double(21))
print(double("ab"))             # not an int, yet it still runs

432.4
{'count': <class 'int'>, 'weight_g': <class 'float'>, 'return': <class 'float'>}
42
abab

• count: int means “count is expected to be a whole number”. -> float at the end means “the function returns a decimal number”.
• Variables can be labelled too: age_days: int = 120.
• Annotations are stored in __annotations__ as notes.
• Python does not check them while running. double("ab") is not an int, yet it runs and gives abab (text times two repeats the text). No error. The checking is done by a separate tool, not by Python.

NoteREMEMBER

A type hint is a note, not a doorkeeper. Without a checking tool, a wrong type is only discovered while the program runs, or not discovered at all like the abab above. For a doorkeeper at run time, see Pydantic in Chapter 34.

Types for collections, choices, and functions

02_collection_types.py

# Chapter 33 - Types for collections, choices, and functions
from collections.abc import Callable, Iterator


def average(numbers: list[float]) -> float:
    return sum(numbers) / len(numbers)


def find_pond(data: dict[str, float], name: str) -> float | None:
    """Returns the pond weight, or None if it does not exist."""
    return data.get(name)


def point(x: float, y: float) -> tuple[float, float]:
    return (x, y)


def apply(function: Callable[[float], float], values: list[float]) -> list[float]:
    return [function(v) for v in values]


def countdown(n: int) -> Iterator[int]:
    while n > 0:
        yield n
        n -= 1


weights = {"A": 540.5, "B": 498.0}
print(average([540.5, 498.0, 610.0]))
print(find_pond(weights, "A"), find_pond(weights, "Z"))
print(point(1.5, 2.0))
print(apply(lambda g: g / 1000, [540.5, 498.0]))
print(list(countdown(3)))

549.5
540.5 None
(1.5, 2.0)
[0.5405, 0.498]
[3, 2, 1]

Written as	Meaning
list[float]	a list whose items are decimals
dict[str, float]	a dict with text keys and decimal values
tuple[float, float]	a tuple of two decimals
float \| None	a decimal or None (meaning “may be empty”)
Callable[[float], float]	a function taking one decimal and returning a decimal
Iterator[int]	something you can next() that gives whole numbers

find_pond(weights, "Z") returns None because pond Z does not exist; that is why its type is float | None. Writing float | None uses the | sign (Python 3.10 and up). Callable and Iterator come from collections.abc. Note: countdown is a generator (Chapter 30), and the fitting type for a simple generator is Iterator[int].

Generics: types that follow the content

The function first(items) can be used on a list of decimals or a list of text. The result type must follow: decimal if given a list of decimals, text if given a list of text. That is the job of generics, with a type variable T.

03_generics.py

# Chapter 33 - Generics: types that follow the content
from collections.abc import Sequence


def first[T](items: Sequence[T]) -> T:
    """Returns the first element; its type matches the list content."""
    return items[0]


print(first([540.5, 498.0]))        # T = float
print(first(["A", "B"]))            # T = str


class Warehouse[T]:
    """A place to store items of the same type."""

    def __init__(self) -> None:
        self.content: list[T] = []

    def store(self, item: T) -> None:
        self.content.append(item)

    def take(self) -> T:
        return self.content.pop()


weight_store = Warehouse[float]()
weight_store.store(540.5)
print(weight_store.take())

540.5
A
540.5

The syntax def first[T](...) and class Warehouse[T] is the way of writing introduced in Python 3.12 (before that, TypeVar and Generic were used). Warehouse[float]() makes a decimals-only warehouse. A checking tool will then complain if you store text in it.

TypedDict, Literal, and Protocol

Field data often takes the form of a dict. TypedDict states which keys exist and their types. Literal limits the choices to certain values. Protocol states “anything that has this” without requiring inheritance (called duck typing: “if it walks like a duck and quacks like a duck, it is a duck”).

04_field_data_types.py

# Chapter 33 - TypedDict, Literal, and Protocol
from dataclasses import dataclass
from typing import Literal, Protocol, TypedDict

Sector = Literal["forest", "orchard", "ricefield", "pond", "barn"]


class Record(TypedDict):
    sector: Sector
    unit: str
    value: float


@dataclass
class Plot:
    name: str
    area_ha: float


class HasName(Protocol):
    """A 'duck' type: anything that has a name attribute."""
    name: str


def greet(thing: HasName) -> str:
    return "Hello, " + thing.name


r: Record = {"sector": "pond", "unit": "A", "value": 540.5}
print(r["sector"], r["value"])
print(greet(Plot("Block A", 1.5)))

pond 540.5
Hello, Block A

Record is a dict with three keys, and sector may only be one of five values. greet() accepts anything with a name attribute: here Plot, a dataclass (Chapter 16), inheriting from nothing. While running, everything is an ordinary dict and an ordinary object; the labels are only for the checker.

Running the real mypy

mypy is a type-checking tool. The script below writes a small file that is deliberately wrong, runs mypy on it, and shows the real output.

05_run_mypy.py

# Chapter 33 - Running the real mypy on a small file
import subprocess
import sys
from pathlib import Path

folder = Path(__file__).parent / "_tmp"
folder.mkdir(exist_ok=True)
example = folder / "wrong_example.py"
example.write_text('''def biomass_kg(count: int, weight_g: float) -> float:
    return count * weight_g / 1000


total: int = biomass_kg(800, 540.5)
print(biomass_kg("800", 540.5))
print(biomass_kg(800))
name: str = None
''', encoding="utf-8")

result = subprocess.run(
    [sys.executable, "-m", "mypy", "--no-color-output", "--no-error-summary",
     "--cache-dir", str(folder / ".mypy_cache"), "wrong_example.py"],
    cwd=folder, capture_output=True, text=True,
)
print(result.stdout.strip())
print("mypy exit code:", result.returncode)

wrong_example.py:5: error: Incompatible types in assignment (expression has type "float", variable has type "int")  [assignment]
wrong_example.py:6: error: Argument 1 to "biomass_kg" has incompatible type "str"; expected "int"  [arg-type]
wrong_example.py:7: error: Missing positional argument "weight_g" in call to "biomass_kg"  [call-arg]
wrong_example.py:8: error: Incompatible types in assignment (expression has type "None", variable has type "str")  [assignment]
mypy exit code: 1

Mypy finds four mistakes without running the file:

• Line 5: biomass_kg returns a float, but it is stored in a variable promised to be int.
• Line 6: the first argument is the text "800", while an int is expected.
• Line 7: the argument weight_g is missing.
• Line 8: None is given to a variable of type str.

The code in square brackets at the end ([assignment], [arg-type], [call-arg]) is the name of the error category. The mypy exit code 1 means “there are problems”, which is useful when mypy is run automatically (for example in a pre-commit check, Chapters 19 and 20).

TipTIP

Start small. Add type hints to the most used functions, run mypy, fix, then expand. mypy’s --strict option is stricter (it demands types on almost every function); it suits a project that is already tidy.

Pyright, another checking tool

Besides mypy, there is pyright, a type checker made by Microsoft. On its project page, pyright is described as a static type checker for Python offered as a command-line tool and a Visual Studio Code extension. Pyright is not run in this book; how to install it and how its behaviour differs from mypy have not been verified on the author’s computer, so please consult its own documentation if you want to try it. The concepts you learn here (annotations, generics, TypedDict) apply equally to both.

Python 3.14: annotations are evaluated later

In the past, an annotation such as x: SomeClass was evaluated immediately when def ran. If SomeClass was not yet defined, an error occurred, so people wrote "SomeClass" (in quotes) or from __future__ import annotations. Since Python 3.14, annotations are stored and evaluated only when needed (source: the “What’s New in Python 3.14” documentation, section on deferred annotations). As a result, a reference to a name that does not exist yet no longer needs quotes. The new module annotationlib reads annotations in three formats.

06_deferred_annotations.py

# Chapter 33 - Python 3.14: annotations are evaluated later
from annotationlib import Format, get_annotations


def compute(x: NotCreatedYet) -> None:     # this name does NOT exist yet
    pass


print("function was created without any error")
print(get_annotations(compute, format=Format.STRING))
reference = get_annotations(compute, format=Format.FORWARDREF)["x"]
print(type(reference).__name__, reference.__forward_arg__)
try:
    get_annotations(compute, format=Format.VALUE)
except NameError as e:
    print("NameError:", e)


class NotCreatedYet:
    pass


print(get_annotations(compute, format=Format.VALUE))   # now the name exists

function was created without any error
{'x': 'NotCreatedYet', 'return': 'None'}
ForwardRef NotCreatedYet
NameError: name 'NotCreatedYet' is not defined
{'x': <class '__main__.NotCreatedYet'>, 'return': None}

• compute(x: NotCreatedYet) is created successfully even though NotCreatedYet does not exist. On older Pythons this is an immediate NameError.
• The STRING format shows the annotation as text, FORWARDREF as a reference (ForwardRef) for a name that does not exist yet, and VALUE really evaluates the value.
• The VALUE format before the class exists gives a NameError. After NotCreatedYet is defined, VALUE succeeds and returns the class.

CautionTECHNICAL STUFF (feel free to skip)

This script runs only on Python 3.14 or later, because annotationlib is new there and the behaviour “the function is created even though the name is missing” is also specific to 3.14. Readers still on an older Python should keep writing forward references in quotes. The code from __future__ import annotations is still recognised and keeps the old behaviour.

Running Project I4: typing the project code

We add type hints to the record-reading functions from Chapter 30, then check with mypy in strict mode. The script writes a small module field.py into _tmp/, runs mypy, inserts one deliberate mistake, runs mypy again, then runs the module.

07_project_typed_code.py

# Chapter 33 - Running project I4: typing the project code, checked by mypy
import subprocess
import sys
from pathlib import Path

folder = Path(__file__).parent / "_tmp"
folder.mkdir(exist_ok=True)
module = folder / "field.py"
module.write_text('''from collections.abc import Iterable, Iterator
from typing import Literal, TypedDict, cast

Sector = Literal["forest", "orchard", "ricefield", "pond", "barn"]


class Record(TypedDict):
    sector: Sector
    unit: str
    value: float


def read(lines: Iterable[str]) -> Iterator[Record]:
    for line in lines:
        sector, unit, value = line.split(",")
        yield {"sector": cast(Sector, sector), "unit": unit, "value": float(value)}


def average(records: Iterable[Record]) -> float | None:
    values = [r["value"] for r in records]
    return sum(values) / len(values) if values else None


print(average(read(["pond,A,540.5", "pond,B,498.0"])))
print(average(read([])))
''', encoding="utf-8")


def mypy(*options: str) -> None:
    result = subprocess.run(
        [sys.executable, "-m", "mypy", "--no-color-output", "--cache-dir", str(folder / ".mypy_cache"),
         *options, "field.py"],
        cwd=folder, capture_output=True, text=True,
    )
    print(result.stdout.strip())


print("== mypy ==")
mypy("--strict")
print("== one mistake added on purpose ==")
module.write_text(module.read_text(encoding="utf-8") + "print(average([1.5, 2.5]))" + chr(10), encoding="utf-8")
mypy("--strict")
print("== running the module ==")
run = subprocess.run([sys.executable, "field.py"], cwd=folder, capture_output=True, text=True)
print(run.stdout.strip())
print("error while running:", run.stderr.strip().splitlines()[-1])

== mypy ==
Success: no issues found in 1 source file
== one mistake added on purpose ==
field.py:26: error: List item 0 has incompatible type "float"; expected "Record"  [list-item]
field.py:26: error: List item 1 has incompatible type "float"; expected "Record"  [list-item]
Found 2 errors in 1 file (checked 1 source file)
== running the module ==
519.25
None
error while running: TypeError: 'float' object is not subscriptable

Follow along:

• Mypy --strict declares the module clean: Success: no issues found in 1 source file. cast(Sector, sector) tells mypy that we are sure this text is one of the Sector values (mypy cannot know that from free text). That is our promise, not a check; the real check comes in Chapter 34.
• After the line print(average([1.5, 2.5])) is inserted (it should be given Records, not numbers), mypy reports two errors, one for each list element. The program has not been run at that point.
• If the module is run anyway, 519.25 and None from the first two lines are printed, then the program crashes at the inserted line with TypeError: 'float' object is not subscriptable. Mypy caught the same mistake without waiting for the program to crash.

Exercises

1. Add type hints to a function average_weight(weights) that takes a list[float] and returns a float.
2. Write a function find_tree(items: list[str], name: str) -> int | None that returns the position of name in the list, or None if absent.
3. Change 05_run_mypy.py: fix the four mistakes in the example file, run again, and see the Success message.
4. Challenge: make a TypedDict called Livestock (keys barn: str, kind: Literal["cattle", "goat", "chicken"], weight_kg: float), then a function total_weight(data: list[Livestock]) -> float. Check with mypy, including one record whose kind has a typo.

Chapter Summary

• A type hint is a type label on variables and functions; Python does not check it while running.
• Tools such as mypy (and pyright) check types before the program runs and report mismatches.
• Common types: list[float], dict[str, float], float | None, Callable, Iterator; generics ([T]) make types follow the content.
• TypedDict, Literal, and Protocol state the shape of a dict, limited choices, and “anything that has this”.
• Python 3.14 defers the evaluation of annotations; annotationlib reads them in the VALUE, FORWARDREF, and STRING formats.
• Project I4: the record-reading code is typed and passes mypy --strict; the inserted mistake is visible before running.

Next, in Chapter 34: data validation and configuration with Pydantic, a doorkeeper that works while the program runs.