Pythonic Python

53 steps in 14 sets of Python.

The difference between Python that works and Python that reads like Python. Comprehensions first, then decorators and the closures underneath them, generators and the itertools that pair with them, and the dunder protocols that let your own objects behave like built-in ones.

After that it gets less polite. Descriptors, __init_subclass__ and metaclass-adjacent machinery, functools and operator, typing beyond the basics, and async twice over, once for the syntax and once for the patterns that actually come up. Not all of this is code you should write. All of it is code you will read.

Start this tour

Comprehensions

List comprehensionsBuilding a list from a loop in a single expression.
Dict & set comprehensionsThe comprehension syntax, aimed at dicts and sets.
Nested comprehensionsComprehensions over two dimensions: grids and matrices.
Generator expressionsLazy comprehensions that compute one item at a time.

Decorators & closures

Closures and nonlocalInner functions that remember and mutate outer state.
Writing a decoratorWrapping a function to add behavior around it.
Decorators with argumentsA decorator factory: one more layer to take parameters.
Memoize decoratorMemoization trades memory for speed - standard for expensive pure functions.
Retry with backoffExponential backoff is the polite way to retry flaky calls.

Generators & itertools

Generator pipelinesChaining generators so data flows through stages lazily.
yield fromDelegating iteration to a sub-generator, for recursion.
The itertools toolkitStandard-library building blocks for iterators.
itertools.groupbyCollapsing consecutive equal items into groups.
reduce and partialTwo functools staples: folding and pre-filling arguments.

Protocols & context managers

contextlib managersMaking context managers without writing a class.
Timing context managerContext managers guarantee cleanup; this one measures elapsed time.
Container dunder methodsTeaching a class to behave like a built-in container.
Operator overloadingGiving a class meaning for +, *, and repr.
ProtocolsStructural typing: duck typing the type checker understands.

Typing & dataclasses

Type annotationsAnnotating parameters and returns with modern types.
Dataclass featuresBeyond the basics: ordering, defaults, and frozen data.
EnumsNaming a fixed set of related constants with Enum.
Dataclass with distanceDataclasses cut the boilerplate out of small value classes.

Async

async and awaitDefining coroutines and awaiting them in an event loop.
asyncio.gatherRunning many coroutines concurrently and collecting results.
async timeoutsBounding how long an await is allowed to take.
asyncio queuesA producer and consumer sharing an async queue.

Threads & processes

Thread poolsRunning IO-bound work across a pool of threads.
LocksGuarding shared state so threads don't corrupt it.
Process poolsSpreading CPU-bound work across real cores.
Futures as they completeReacting to results in finish order, not submit order.

Attributes & descriptors

__slots__Declaring the attributes up front drops the per-instance dict.
Properties with validationA property turns attribute access into a method call, so invariants hold.
DescriptorsThe protocol properties are built on: __get__ and __set__ on a class attribute.

Class machinery

Abstract base classesABCs state the interface and refuse to instantiate until it is met.
__init_subclass__A hook that runs when a subclass is defined — a registry without a metaclass.
A metaclassThe class of a class: it runs at definition time and can rewrite the class.

functools & operator

functoolspartial, reduce and cached_property: three tools worth reaching for.
singledispatchOne function name, an implementation chosen by the first argument's type.
The operator moduleNamed functions for the operators, which beats a lambda in a sort key.

Typing in depth

TypeVar and GenericA type parameter lets one class or function keep its caller's type.
ProtocolsStructural typing: anything with the right methods satisfies the protocol.
TypedDict and NewTypeTyping the shape of a dict, and giving a primitive a distinct name.
overloadSeveral signatures for one function, so the checker knows what comes back.

Data & enums

Dataclasses, furtherfrozen, slots, field factories and __post_init__.
Enums, furtherauto, StrEnum and Flag cover most of what enums are for.
Structural pattern matching, furtherClass patterns, guards, and capturing the rest of a mapping.

Async in depth

TaskGroupA nursery for tasks: it waits for all of them and cancels the rest on error.
Async iterators and context managersThe async twins of the two protocols you already know.
Limiting concurrencyA semaphore caps how many coroutines are in the expensive part at once.
Exception groupsSeveral failures at once, and the except* syntax that splits them.

Encore

async_crawler.pyFetch many URLs concurrently and report sizes and timing.
parallel_hash.pyHash every file under a directory using a process pool.