reference
5 min read
·
lesson 14 of 16 in Reference
Python itertools Cheat Sheet
1 · The lesson
readQuick reference for itertools — lazy iterator building blocks. For the full tutorial see Itertools.
python
import itertools as it
1. Infinite Iterators
| Tool | Yields | Example |
|---|---|---|
count(start=0, step=1) | start, start+step, start+2*step, ... | count(10, 2) -> 10, 12, 14, ... |
cycle(iterable) | Loops the iterable forever | cycle("AB") -> A, B, A, B, ... |
repeat(obj, times=None) | obj times times (or forever) | repeat(0, 3) -> 0, 0, 0 |
Always pair with islice or break early — they don't stop on their own.
2. Terminating Iterators
| Tool | One-liner description | Example |
|---|---|---|
accumulate(iter, func=add, initial=None) | Running totals (or any binary op) | accumulate([1,2,3]) -> 1, 3, 6 |
chain(*iters) | Concatenate iterables | chain([1,2], [3,4]) -> 1, 2, 3, 4 |
chain.from_iterable(iter_of_iters) | Flatten one level | chain.from_iterable([[1,2],[3]]) -> 1, 2, 3 |
compress(data, selectors) | Keep items where selector is truthy | compress("ABCD", [1,0,1,0]) -> A, C |
dropwhile(pred, iter) | Drop while pred true, then yield rest | dropwhile(lambda x:x<3, [1,2,3,4,1]) -> 3, 4, 1 |
takewhile(pred, iter) | Yield while pred true, then stop | takewhile(lambda x:x<3, [1,2,3,4,1]) -> 1, 2 |
filterfalse(pred, iter) | Yield items where pred is false | filterfalse(None, [0,1,0,2]) -> 0, 0 |
groupby(iter, key=None) | (key, sub-iterator) for adjacent equal-key runs | see below — sort first for global groups |
islice(iter, stop) / islice(iter, start, stop, step) | Slice an iterator | islice(count(), 5) -> 0, 1, 2, 3, 4 |
starmap(func, iter_of_tuples) | func(*args) for each tuple | starmap(pow, [(2,3),(3,2)]) -> 8, 9 |
tee(iter, n=2) | Split one iterator into n independent ones | tee(range(3), 2) -> two iterators |
zip_longest(*iters, fillvalue=None) | Zip but pad shorter ones | zip_longest("AB","XYZ", fillvalue="-") -> ("A","X"),("B","Y"),("-","Z") |
pairwise(iter) (3.10+) | Consecutive overlapping pairs | pairwise("ABCD") -> ("A","B"),("B","C"),("C","D") |
batched(iter, n) (3.12+) | Fixed-size tuples, last one may be shorter | batched("ABCDE", 2) -> ("A","B"),("C","D"),("E",) |
3. Combinatoric Iterators
| Tool | Yields | Example |
|---|---|---|
product(*iters, repeat=1) | Cartesian product (nested for-loops) | product("AB", "12") -> ("A","1"),("A","2"),("B","1"),("B","2") |
permutations(iter, r=None) | r-length orderings, no repeats | permutations("ABC", 2) -> 6 tuples |
combinations(iter, r) | r-length selections, sorted, no repeats | combinations("ABCD", 2) -> 6 tuples |
combinations_with_replacement(iter, r) | r-length selections, sorted, with repeats | combinations_with_replacement("ABC", 2) -> 6 tuples |
4. groupby Gotcha
groupby groups consecutive equal-key runs. Sort first for global groups:
python
data = [("a", 1), ("b", 2), ("a", 3)] # WRONG — yields 3 groups because 'a' isn't adjacent for k, g in it.groupby(data, key=lambda x: x[0]): print(k, list(g)) # RIGHT — sort by key first data.sort(key=lambda x: x[0]) for k, g in it.groupby(data, key=lambda x: x[0]): print(k, list(g)) # a [('a',1),('a',3)] then b [('b',2)]
setup added so this can run · defines it
# Lightweight mock for objects whose attributes/methods aren't critical class _AutoMock: def __init__(self, name='mock'): self._name = name def __getattr__(self, k): return _AutoMock(self._name + '.' + k) def __call__(self, *a, **kw): print('-> ' + self._name + '() called') return _AutoMock(self._name + '()') def __repr__(self): return '<mock ' + self._name + '>' def __str__(self): return '<mock ' + self._name + '>' def __bool__(self): return True def __iter__(self): return iter([]) def __len__(self): return 0 def __getitem__(self, k): return _AutoMock(self._name + '[...]') def __setitem__(self, k, v): pass def __enter__(self): return self def __exit__(self, *a): return False async def __aenter__(self): return self async def __aexit__(self, *a): return False def __add__(self, o): return self def __radd__(self, o): return self def __sub__(self, o): return self def __mul__(self, o): return self def __rmul__(self, o): return self def __truediv__(self, o): return self def __eq__(self, o): return isinstance(o, _AutoMock) def __hash__(self): return hash(self._name) def __lt__(self, o): return True def __le__(self, o): return True def __gt__(self, o): return False def __ge__(self, o): return False def __mro_entries__(self, bases): return (object,) it = _AutoMock('it')
5. Recipes (from docs / commonly needed)
python
import itertools as it from collections import deque # flatten one level (chain.from_iterable does this directly) def flatten(seq_of_seqs): return it.chain.from_iterable(seq_of_seqs) # pairwise — 3.10+ built-in; for older Pythons: def pairwise(iterable): a, b = it.tee(iterable) next(b, None) return zip(a, b) # sliding window of size n def windowed(iterable, n): it_ = iter(iterable) window = deque(it.islice(it_, n), maxlen=n) if len(window) == n: yield tuple(window) for x in it_: window.append(x) yield tuple(window) # partition by predicate def partition(pred, iterable): t1, t2 = it.tee(iterable) return it.filterfalse(pred, t1), filter(pred, t2) # unique while preserving order def unique_everseen(iterable, key=None): seen = set() for x in iterable: k = x if key is None else key(x) if k not in seen: seen.add(k) yield x # round-robin from multiple iterables def roundrobin(*iterables): iters = [iter(i) for i in iterables] while iters: nxt = [] for i in iters: try: yield next(i) nxt.append(i) except StopIteration: pass iters = nxt # consume / advance n steps (or all) — useful side-effect runner def consume(iterator, n=None): if n is None: deque(iterator, maxlen=0) else: next(it.islice(iterator, n, n), None) # nth element (0-indexed) def nth(iterable, n, default=None): return next(it.islice(iterable, n, None), default)
setup added so this can run · defines iterables
iterables = ["alpha", "beta", "gamma"]
Common Patterns
python
# Chunk a list into groups of 3 (3.12+) chunks = list(it.batched(range(10), 3)) # [(0,1,2),(3,4,5),(6,7,8),(9,)] # Sliding window of size 2 (3.10+) list(it.pairwise([1,2,3,4])) # [(1,2),(2,3),(3,4)] # Dedup preserving order list(unique_everseen("AAABBC")) # ['A','B','C'] # Cartesian (all (i,j) pairs) list(it.product(range(2), repeat=2)) # [(0,0),(0,1),(1,0),(1,1)] # Flatten one level list(it.chain.from_iterable([[1,2],[3,4]])) # [1,2,3,4] # Repeat each value N times (e.g. ground truth labels) list(it.chain.from_iterable(it.repeat(x, 3) for x in "AB")) # ['A','A','A','B','B','B'] # First 5 primes-ish — use islice with infinite count print(list(it.islice(it.count(), 5))) # [0,1,2,3,4]
setup added so this can run · defines unique_everseen, it
# Lightweight mock for objects whose attributes/methods aren't critical class _AutoMock: def __init__(self, name='mock'): self._name = name def __getattr__(self, k): return _AutoMock(self._name + '.' + k) def __call__(self, *a, **kw): print('-> ' + self._name + '() called') return _AutoMock(self._name + '()') def __repr__(self): return '<mock ' + self._name + '>' def __str__(self): return '<mock ' + self._name + '>' def __bool__(self): return True def __iter__(self): return iter([]) def __len__(self): return 0 def __getitem__(self, k): return _AutoMock(self._name + '[...]') def __setitem__(self, k, v): pass def __enter__(self): return self def __exit__(self, *a): return False async def __aenter__(self): return self async def __aexit__(self, *a): return False def __add__(self, o): return self def __radd__(self, o): return self def __sub__(self, o): return self def __mul__(self, o): return self def __rmul__(self, o): return self def __truediv__(self, o): return self def __eq__(self, o): return isinstance(o, _AutoMock) def __hash__(self): return hash(self._name) def __lt__(self, o): return True def __le__(self, o): return True def __gt__(self, o): return False def __ge__(self, o): return False def __mro_entries__(self, bases): return (object,) def unique_everseen(*_a, **_kw): print('-> unique_everseen() called') return _AutoMock('unique_everseen()') it = _AutoMock('it')
Common Errors
| Error / Symptom | Cause | Fix |
|---|---|---|
Prints <itertools.chain object> | Forgot to materialise | list(chain(...)) or iterate |
groupby returns more groups than expected | Input not sorted by key | data.sort(key=fn) first |
| Memory / time blow-up | Iterated count() or cycle() without bound | Wrap in islice, break out, or use zip with finite iter |
| Empty result after iterating once | Iterator already consumed | Use tee to split, or list(...) once if small |
TypeError: 'list_iterator' object is not subscriptable | Treated iterator like a sequence | Materialise with list(...) or use islice |
groupby sub-iterators look empty | Outer loop moved past them | Convert each g to list(g) before advancing |
zip silently truncates | Mismatched lengths | Use zip_longest, or zip(..., strict=True) (3.10+) |
See Also
Practice this
on practicepython.inShort exercises that run in your browser and tell you what your code actually did, not just whether a test passed.