PythonMastery

Chunk with batched, compare neighbours with pairwise

itertools.batched splits a sequence into fixed-size chunks and pairwise walks neighbouring pairs, replacing fiddly index arithmetic with one clear call.

Two jobs come up constantly and are both easy to get off by one: splitting a list into chunks of N, and comparing each item with the one before it. itertools has a named function for each, batched (Python 3.12+) and pairwise (3.10+).

Before

python
ids = list(range(1, 11))
for i in range(0, len(ids), 4):
    print(ids[i:i + 4])

readings = [12.0, 15.5, 15.0, 19.25]
for i in range(1, len(readings)):
    print(readings[i] - readings[i - 1])
output
[1, 2, 3, 4]
[5, 6, 7, 8]
[9, 10]
3.5
-0.5
4.25

It works, but every index is a place for len - 1 or i + 1 to go wrong, and it only works on things you can index.

After

python
from itertools import batched, pairwise

ids = range(1, 11)
for chunk in batched(ids, 4):
    print(chunk)

readings = [12.0, 15.5, 15.0, 19.25]
for before, after in pairwise(readings):
    print(after - before)
output
(1, 2, 3, 4)
(5, 6, 7, 8)
(9, 10)
3.5
-0.5
4.25

The chunks come out as tuples, and the last one is simply shorter. Both work on any iterable, including a file or a generator, with no indexing at all.

And one more: flatten one level

chain.from_iterable joins a list of lists without a nested loop or sum(lists, []).

python
from itertools import chain

weekly = [["Mon", "Tue"], ["Wed"], ["Thu", "Fri"]]
print(list(chain.from_iterable(weekly)))
output
['Mon', 'Tue', 'Wed', 'Thu', 'Fri']

Why it works

Each is a small iterator that remembers its place. batched pulls N items at a time; pairwise keeps the previous item and hands you (previous, current). Neither copies your data first, so they work the same on ten items or ten million.

When not to use it

batched is Python 3.12 or newer, so older servers need the slicing loop (the browser here runs 3.13). If you need the chunks padded to equal length, that's a different recipe (zip_longest), and for rolling windows wider than two, a collections.deque(maxlen=N) reads more clearly.

Learn it properly: Itertools: Composable Iterator Building Blocks, Python itertools Cheat Sheet