Itertools: Composable Iterator Building Blocks

The itertools module gives you small building blocks for loops, which you can combine like Lego pieces. Each function returns a lazy iterator that reads its input one element at a time and produces one element at a time. You can connect them (the output of one is the input of the next), and they almost never build intermediate lists. That makes them memory-efficient on huge or infinite sequences. They can also replace nested loops and accumulator variables with a single line.

Once you know the dozen functions in this lesson, you will find uses for them often.

1. Why Itertools

Three properties shared by everything in the module:

1. Lazy: nothing is computed until you iterate. itertools.count() represents 0, 1, 2, 3, … forever in constant memory.
2. Composable: chain(a, b) returns an iterator. Pass that to islice(..., 10) and you get the first ten items of the joined stream, still without building a list of either input.
3. Single-use: like all iterators, once you have iterated over them they are exhausted. Use list(iter_obj) if you need to keep the result.

python

import itertools as it

# Lazy: nothing here is computed until you use the values
stream = it.islice(it.count(start=1), 5)        # iterator over [1, 2, 3, 4, 5]
print(list(stream))                             # [1, 2, 3, 4, 5]
print(list(stream))                             # []  (the iterator is already used up)

Almost everyone makes this mistake once. After iterating, the iterator is empty. Call the functions (it.count, it.islice) again to get a new iterator, or save the values in a list.

2. The Infinite Ones: `count`, `cycle`, `repeat`

These three iterators never stop on their own. Always use them together with islice, takewhile, zip, or something else that stops the loop.

python

import itertools as it

# count(start=0, step=1): like range() with no end
for i in it.count(start=10, step=2):
    if i > 20:
        break
    print(i)                                    # 10, 12, 14, 16, 18, 20

# cycle(iter): repeats the items again and again, with no end
colours = it.cycle(["red", "green", "blue"])
for _ in range(7):
    print(next(colours))                        # red, green, blue, red, green, blue, red

# repeat(x, n=infinite): the same value, n times or forever
list(it.repeat("hi", 3))                        # ['hi', 'hi', 'hi']

# A common pattern: number the rows of a CSV
rows = ["a", "b", "c", "d"]
list(zip(it.count(1), rows))                    # [(1,'a'), (2,'b'), (3,'c'), (4,'d')]

zip(it.count(1), rows) gives the same result as enumerate(rows, start=1), which is the usual way to write it. But count also works where enumerate doesn't, for example with takewhile to "keep generating numbers until a condition fails".

3. `chain` and `chain.from_iterable`: Join Iterables

chain(*iterables) goes through each iterable in turn and yields its items, so it removes one level of nesting.

python

import itertools as it

list(it.chain([1, 2], [3, 4], (5, 6)))         # [1, 2, 3, 4, 5, 6]
list(it.chain("ab", "cd", "ef"))               # ['a', 'b', 'c', 'd', 'e', 'f']

chain.from_iterable(iter_of_iters) takes a single iterable of iterables. This is useful when you already have a list of lists:

python

matrix = [[1, 2, 3], [4, 5], [6, 7, 8, 9]]

list(it.chain.from_iterable(matrix))           # [1, 2, 3, 4, 5, 6, 7, 8, 9]

# Equivalent comprehension
[x for row in matrix for x in row]

+ setup added so this can run · re-runs 3 earlier blocks on this page first

# Lightweight mock for objects whose attributes/methods aren't critical
class _AutoMock:
    def __init__(self, name='mock'): self._name = name
    def __getattr__(self, k): return _AutoMock(self._name + '.' + k)
    def __call__(self, *a, **kw):
        print('-> ' + self._name + '() called')
        return _AutoMock(self._name + '()')
    def __repr__(self): return '<mock ' + self._name + '>'
    def __str__(self): return '<mock ' + self._name + '>'
    def __bool__(self): return True
    def __iter__(self): return iter([])
    def __len__(self): return 0
    def __getitem__(self, k): return _AutoMock(self._name + '[...]')
    def __setitem__(self, k, v): pass
    def __enter__(self): return self
    def __exit__(self, *a): return False
    async def __aenter__(self): return self
    async def __aexit__(self, *a): return False
    def __add__(self, o): return self
    def __radd__(self, o): return self
    def __sub__(self, o): return self
    def __mul__(self, o): return self
    def __rmul__(self, o): return self
    def __truediv__(self, o): return self
    def __eq__(self, o): return isinstance(o, _AutoMock)
    def __hash__(self): return hash(self._name)
    def __lt__(self, o): return True
    def __le__(self, o): return True
    def __gt__(self, o): return False
    def __ge__(self, o): return False
    def __mro_entries__(self, bases): return (object,)

it = _AutoMock('it')

# Earlier blocks on this page that this one builds on, re-run quietly first.
try:
    import itertools as it
except Exception:
    pass
_pm_earlier = [
    'import itertools as it\n\n# Lazy: nothing here is computed until you use the values\nstream = it.islice(it.count(start=1), 5)        # iterator over [1, 2, 3, 4, 5]\nprint(list(stream))                             # [1, 2, 3, 4, 5]\nprint(list(stream))                             # []  (the iterator is already used up)',
    'import itertools as it\n\n# count(start=0, step=1): like range() with no end\nfor i in it.count(start=10, step=2):\n    if i > 20:\n        break\n    print(i)                                    # 10, 12, 14, 16, 18, 20\n\n# cycle(iter): repeats the items again and again, with no end\ncolours = it.cycle(["red", "green", "blue"])\nfor _ in range(7):\n    print(next(colours))                        # red, green, blue, red, green, blue, red\n\n# repeat(x, n=infinite): the same value, n times or forever\nlist(it.repeat("hi", 3))                        # [\'hi\', \'hi\', \'hi\']\n\n# A common pattern: number the rows of a CSV\nrows = ["a", "b", "c", "d"]\nlist(zip(it.count(1), rows))                    # [(1,\'a\'), (2,\'b\'), (3,\'c\'), (4,\'d\')]',
    'import itertools as it\n\nlist(it.chain([1, 2], [3, 4], (5, 6)))         # [1, 2, 3, 4, 5, 6]\nlist(it.chain("ab", "cd", "ef"))               # [\'a\', \'b\', \'c\', \'d\', \'e\', \'f\']',
]
def _pm_run_earlier(_srcs, _g):
    import sys as _s, io as _io, warnings as _w
    for _src in _srcs:
        _o, _e = _s.stdout, _s.stderr
        _s.stdout = _s.stderr = _io.StringIO()
        try:
            with _w.catch_warnings():
                _w.simplefilter('ignore')
                exec(compile(_src, '<earlier block>', 'exec'), _g)
        except (Exception, SystemExit):
            pass
        finally:
            _s.stdout, _s.stderr = _o, _e
    if 'matplotlib.pyplot' in _s.modules:
        _s.modules['matplotlib.pyplot'].close('all')   # their plots belong to their own blocks
_pm_run_earlier(_pm_earlier, globals())
del _pm_earlier, _pm_run_earlier

For a simple list of lists, both forms work. When you are joining many streams, from_iterable is especially useful:

python

files = ["a.txt", "b.txt", "c.txt"]
all_lines = it.chain.from_iterable(open(f) for f in files)
for line in all_lines:                         # reads one line at a time, never all the files at once
    process(line)

+ setup added so this can run · defines process, it

# Lightweight mock for objects whose attributes/methods aren't critical
class _AutoMock:
    def __init__(self, name='mock'): self._name = name
    def __getattr__(self, k): return _AutoMock(self._name + '.' + k)
    def __call__(self, *a, **kw):
        print('-> ' + self._name + '() called')
        return _AutoMock(self._name + '()')
    def __repr__(self): return '<mock ' + self._name + '>'
    def __str__(self): return '<mock ' + self._name + '>'
    def __bool__(self): return True
    def __iter__(self): return iter([])
    def __len__(self): return 0
    def __getitem__(self, k): return _AutoMock(self._name + '[...]')
    def __setitem__(self, k, v): pass
    def __enter__(self): return self
    def __exit__(self, *a): return False
    async def __aenter__(self): return self
    async def __aexit__(self, *a): return False
    def __add__(self, o): return self
    def __radd__(self, o): return self
    def __sub__(self, o): return self
    def __mul__(self, o): return self
    def __rmul__(self, o): return self
    def __truediv__(self, o): return self
    def __eq__(self, o): return isinstance(o, _AutoMock)
    def __hash__(self): return hash(self._name)
    def __lt__(self, o): return True
    def __le__(self, o): return True
    def __gt__(self, o): return False
    def __ge__(self, o): return False
    def __mro_entries__(self, bases): return (object,)

def process(*_a, **_kw):
    print('-> process() called')
    return _AutoMock('process()')
it = _AutoMock('it')

4. `takewhile` and `dropwhile`: Split at the First Failure

takewhile(pred, iter) yields items while the predicate is true, then stops, even if later items would also pass.

python

import itertools as it

nums = [1, 3, 5, 4, 6, 7, 9]

list(it.takewhile(lambda n: n % 2 == 1, nums))  # [1, 3, 5]   (stops at 4)
list(it.dropwhile(lambda n: n % 2 == 1, nums))  # [4, 6, 7, 9]  (skips items until 4, then keeps all)

dropwhile does the opposite: it skips items while the predicate is true, then yields everything from the first failure onwards (predicate is not re-checked after the first miss).

Example: remove the comment lines at the start of a file, without reading it twice:

python

lines = ["# header", "# notes", "data1", "data2", "# inline comment"]
data = list(it.dropwhile(lambda l: l.startswith("#"), lines))
# ['data1', 'data2', '# inline comment']   (the last comment is kept!)

+ setup added so this can run · defines it

# Lightweight mock for objects whose attributes/methods aren't critical
class _AutoMock:
    def __init__(self, name='mock'): self._name = name
    def __getattr__(self, k): return _AutoMock(self._name + '.' + k)
    def __call__(self, *a, **kw):
        print('-> ' + self._name + '() called')
        return _AutoMock(self._name + '()')
    def __repr__(self): return '<mock ' + self._name + '>'
    def __str__(self): return '<mock ' + self._name + '>'
    def __bool__(self): return True
    def __iter__(self): return iter([])
    def __len__(self): return 0
    def __getitem__(self, k): return _AutoMock(self._name + '[...]')
    def __setitem__(self, k, v): pass
    def __enter__(self): return self
    def __exit__(self, *a): return False
    async def __aenter__(self): return self
    async def __aexit__(self, *a): return False
    def __add__(self, o): return self
    def __radd__(self, o): return self
    def __sub__(self, o): return self
    def __mul__(self, o): return self
    def __rmul__(self, o): return self
    def __truediv__(self, o): return self
    def __eq__(self, o): return isinstance(o, _AutoMock)
    def __hash__(self): return hash(self._name)
    def __lt__(self, o): return True
    def __le__(self, o): return True
    def __gt__(self, o): return False
    def __ge__(self, o): return False
    def __mro_entries__(self, bases): return (object,)

it = _AutoMock('it')

This is not the same as filter. filter checks every element. takewhile/dropwhile stop checking at the first item that fails.

5. `islice`: Slice Any Iterable

Lists support lst[2:10:2]. Generators don't. islice gives you slicing for any iterator:

python

import itertools as it

# islice(iter, stop)               gives the first N items
list(it.islice(it.count(), 5))                 # [0, 1, 2, 3, 4]

# islice(iter, start, stop)        gives the items in a range
list(it.islice(range(20), 5, 10))              # [5, 6, 7, 8, 9]

# islice(iter, start, stop, step)  takes every step-th item
list(it.islice(range(20), 0, 20, 3))           # [0, 3, 6, 9, 12, 15, 18]

# stop=None means go on to the end
list(it.islice("abcdefgh", 2, None))           # ['c', 'd', 'e', 'f', 'g', 'h']

Two limits versus list slicing:

No negative indices: you can't ask for "the last five" items of an iterator, because it doesn't know its length.
It consumes the underlying iterator. After it.islice(stream, 5), the next item in stream is the sixth, not the first.

6. `zip_longest`: Zip Without Losing Items

Plain zip stops at the shortest iterable. zip_longest runs until the longest is exhausted, filling missing values with fillvalue.

python

from itertools import zip_longest

names  = ["Alice", "Bob", "Charlie"]
scores = [90, 85]

list(zip(names, scores))                        # [('Alice', 90), ('Bob', 85)]   (Charlie is dropped)
list(zip_longest(names, scores, fillvalue=0))   # [('Alice', 90), ('Bob', 85), ('Charlie', 0)]

Use it when missing data matters: survey responses, sparse columns, partial migrations. The fillvalue is a single value applied to all missing slots in all shorter iterables.

7. `groupby`: Group Consecutive Items With the Same Key

groupby(iter, key=fn) walks the iterable and yields (key, group_iterator) pairs whenever the key changes. It only groups items that are next to each other. If you want one group per key, sort first.

python

from itertools import groupby

# Already sorted by first letter, so groupby works as expected
words = ["apple", "ant", "banana", "berry", "cherry"]
for key, group in groupby(words, key=lambda w: w[0]):
    print(key, list(group))
# a ['apple', 'ant']
# b ['banana', 'berry']
# c ['cherry']

# NOT sorted, so groupby splits a group without any warning
words = ["apple", "banana", "ant", "berry", "cherry"]
for key, group in groupby(words, key=lambda w: w[0]):
    print(key, list(group))
# a ['apple']
# b ['banana']
# a ['ant']          (a separate group!)
# b ['berry']
# c ['cherry']

The fix is always the same: sort by the key first.

python

words.sort(key=lambda w: w[0])
groups = {k: list(g) for k, g in groupby(words, key=lambda w: w[0])}
# {'a': ['apple', 'ant'], 'b': ['banana', 'berry'], 'c': ['cherry']}

+ setup added so this can run · re-runs 1 earlier block on this page first

# Lightweight mock for objects whose attributes/methods aren't critical
class _AutoMock:
    def __init__(self, name='mock'): self._name = name
    def __getattr__(self, k): return _AutoMock(self._name + '.' + k)
    def __call__(self, *a, **kw):
        print('-> ' + self._name + '() called')
        return _AutoMock(self._name + '()')
    def __repr__(self): return '<mock ' + self._name + '>'
    def __str__(self): return '<mock ' + self._name + '>'
    def __bool__(self): return True
    def __iter__(self): return iter([])
    def __len__(self): return 0
    def __getitem__(self, k): return _AutoMock(self._name + '[...]')
    def __setitem__(self, k, v): pass
    def __enter__(self): return self
    def __exit__(self, *a): return False
    async def __aenter__(self): return self
    async def __aexit__(self, *a): return False
    def __add__(self, o): return self
    def __radd__(self, o): return self
    def __sub__(self, o): return self
    def __mul__(self, o): return self
    def __rmul__(self, o): return self
    def __truediv__(self, o): return self
    def __eq__(self, o): return isinstance(o, _AutoMock)
    def __hash__(self): return hash(self._name)
    def __lt__(self, o): return True
    def __le__(self, o): return True
    def __gt__(self, o): return False
    def __ge__(self, o): return False
    def __mro_entries__(self, bases): return (object,)

words = _AutoMock('words')
def groupby(*_a, **_kw):
    print('-> groupby() called')
    return _AutoMock('groupby()')

# Earlier blocks on this page that this one builds on, re-run quietly first.
try:
    from itertools import groupby
except Exception:
    pass
_pm_earlier = [
    'from itertools import groupby\n\n# Already sorted by first letter, so groupby works as expected\nwords = ["apple", "ant", "banana", "berry", "cherry"]\nfor key, group in groupby(words, key=lambda w: w[0]):\n    print(key, list(group))\n# a [\'apple\', \'ant\']\n# b [\'banana\', \'berry\']\n# c [\'cherry\']\n\n# NOT sorted, so groupby splits a group without any warning\nwords = ["apple", "banana", "ant", "berry", "cherry"]\nfor key, group in groupby(words, key=lambda w: w[0]):\n    print(key, list(group))\n# a [\'apple\']\n# b [\'banana\']\n# a [\'ant\']          (a separate group!)\n# b [\'berry\']\n# c [\'cherry\']',
]
def _pm_run_earlier(_srcs, _g):
    import sys as _s, io as _io, warnings as _w
    for _src in _srcs:
        _o, _e = _s.stdout, _s.stderr
        _s.stdout = _s.stderr = _io.StringIO()
        try:
            with _w.catch_warnings():
                _w.simplefilter('ignore')
                exec(compile(_src, '<earlier block>', 'exec'), _g)
        except (Exception, SystemExit):
            pass
        finally:
            _s.stdout, _s.stderr = _o, _e
    if 'matplotlib.pyplot' in _s.modules:
        _s.modules['matplotlib.pyplot'].close('all')   # their plots belong to their own blocks
_pm_run_earlier(_pm_earlier, globals())
del _pm_earlier, _pm_run_earlier

For pure group-and-collect with no need for the order of groups, collections.defaultdict(list) (one pass, no sort) is often simpler. Use groupby when the grouping should depend on order: runs of equal pixels, events in time windows, log lines from the same request.

One more thing to watch: the inner group is an iterator that shares state with the outer groupby. Use up each group before moving to the next one (list(group) in the example). Otherwise it will be empty when you try to read it.

8. `accumulate`: Running Totals (and More)

accumulate(iter, func=operator.add) yields the running result.

python

from itertools import accumulate
import operator

list(accumulate([1, 2, 3, 4]))                  # [1, 3, 6, 10]   (running sum, the default)
list(accumulate([1, 2, 3, 4], operator.mul))    # [1, 2, 6, 24]   (running product)
list(accumulate([3, 1, 4, 1, 5, 9, 2, 6], max)) # [3, 3, 4, 4, 5, 9, 9, 9]   (running maximum)

# Bank balance: start with 100, then apply each day's change
changes = [+50, -30, +100, -75]
list(accumulate(changes, initial=100))          # [100, 150, 120, 220, 145]  (needs Python 3.8+)

The initial= keyword (Python 3.8+) adds a starting value at the front. This is useful for bank balances, prefix sums starting at 0, and anything else that needs a start value.

Watch out: the function must take two arguments, (acc, x) -> new_acc, and the order of the arguments matters for non-commutative operations (like subtraction). accumulate([2, 1, 3], lambda a, b: a - b) gives [2, 1, -2], not [2, -1, -4].

9. `product`: Nested Loops as One Call

product(*iterables, repeat=1) is the Cartesian product: every combination of one item from each iterable.

python

from itertools import product

# Pairs of dice rolls
list(product([1, 2, 3], [1, 2, 3]))
# [(1,1), (1,2), (1,3), (2,1), (2,2), (2,3), (3,1), (3,2), (3,3)]

# Equivalent nested-loop form
[(a, b) for a in [1, 2, 3] for b in [1, 2, 3]]

# repeat=3: the Cartesian product of one iterable with itself, 3 times
list(product("ab", repeat=3))
# [('a','a','a'), ('a','a','b'), ('a','b','a'), ..., ('b','b','b')]   (every 3-letter string of a and b)

# Three-level loop over a grid
for x, y, z in product(range(3), range(3), range(3)):
    ...     # 27 iterations, with no nested loops

product(repeat=n) is how you write "all possible passwords of length n from this alphabet" or "every coordinate in this n-dimensional grid" without an n-deep nested loop.

10. `permutations` and `combinations`

When order matters → permutations. When it doesn't → combinations.

python

from itertools import permutations, combinations

# All orderings of 3 letters, length 2
list(permutations("abc", 2))
# [('a','b'), ('a','c'), ('b','a'), ('b','c'), ('c','a'), ('c','b')]   (6 = 3*2)

# All subsets of size 2. Order does not matter, so (a,b) is the same as (b,a)
list(combinations("abc", 2))
# [('a','b'), ('a','c'), ('b','c')]   (3 = C(3,2), "3 choose 2")

# combinations_with_replacement: an element can be used more than once
from itertools import combinations_with_replacement
list(combinations_with_replacement("abc", 2))
# [('a','a'), ('a','b'), ('a','c'), ('b','b'), ('b','c'), ('c','c')]

Use them for combinatorial problems: subset-sum, anagram detection, brute-force searches over small spaces. Note that the size grows very fast: permutations(range(10), 10) is 3.6 million, and for range(12) it's 479 million. Use islice to look at a few results without building the whole list.

11. `pairwise` (Python 3.10+): Sliding Window of 2

This is a very common operation that used to need several lines of code.

python

from itertools import pairwise

list(pairwise([1, 2, 3, 4, 5]))                 # [(1,2), (2,3), (3,4), (4,5)]

# Find the price change from each day to the next
prices = [100, 102, 99, 105, 110]
deltas = [b - a for a, b in pairwise(prices)]   # [2, -3, 6, 5]

For windows wider than 2, there is no built-in function. Write a generator with collections.deque(maxlen=n), or use the itertools recipes from the docs.

Common Mistakes

1. Forgetting itertools returns iterators, not lists

python

import itertools as it

result = it.chain([1, 2], [3, 4])
print(result)                                   # <itertools.chain object at 0x...>
print(len(result))                              # TypeError  (iterators have no len)

# Fix: make a list if you need the length, indexing, or reuse
result = list(it.chain([1, 2], [3, 4]))

Don't call list() on everything by habit, because you lose the memory benefit. But when you need len, indexing, or a second iteration, make a list.

2. groupby without pre-sorting

python

from itertools import groupby

records = [
    ("Alice", "eng"), ("Bob", "sales"), ("Carol", "eng"),
]

# Bug: Alice and Carol go into separate groups, because Bob is between them
{k: [name for name, _ in g] for k, g in groupby(records, key=lambda r: r[1])}
# {'eng': ['Carol'], 'sales': ['Bob']}  (the second 'eng' group replaced the first, so Alice is lost)

# Fix: sort by the same key first
records.sort(key=lambda r: r[1])
{k: [name for name, _ in g] for k, g in groupby(records, key=lambda r: r[1])}
# {'eng': ['Alice', 'Carol'], 'sales': ['Bob']}

Losing data without any error is the worst kind of bug. If you don't want grouping by consecutive items, use defaultdict(list) instead.

3. Iterating cycle() without a stop condition

python

import itertools as it

for c in it.cycle("abc"):
    print(c)                                    # never ends (press Ctrl+C to stop it)

Use every infinite iterator together with islice, takewhile, zip with a finite iterable, or an explicit break. Zipping two infinite iterators (zip(cycle(a), cycle(b))) also runs forever, because the shortest one is still infinite.

4. accumulate with non-commutative ops

python

from itertools import accumulate

list(accumulate([10, 1, 2, 3], lambda a, b: a - b))
# [10, 9, 7, 4]    (10, then 10-1=9, 9-2=7, 7-3=4)

Many people expect [10, 1-10, 2-9, 3-7] or something similar. That is wrong: accumulate passes (running_total, next_item), in that order. For subtraction, division, and string formatting, write a small test before you rely on your expectation.

5. Trying to slice an iterator with [:n]

python

import itertools as it

stream = it.count()
first_five = stream[:5]                         # TypeError  (you cannot slice an iterator)

# Use islice instead
first_five = list(it.islice(stream, 5))

[:n] works on lists, tuples and strings, but not on iterators in general. For iterators, use islice.

🎯 Your Turn: Chunked

Write chunked(iterable, n), a generator that yields successive n-item lists from any iterable. The final chunk may be shorter than n.

python

list(chunked(range(10), 3))         # [[0, 1, 2], [3, 4, 5], [6, 7, 8], [9]]
list(chunked("abcdefgh", 2))        # [['a', 'b'], ['c', 'd'], ['e', 'f'], ['g', 'h']]
list(chunked([], 4))                # []

+ setup added so this can run · defines chunked

# Lightweight mock for objects whose attributes/methods aren't critical
class _AutoMock:
    def __init__(self, name='mock'): self._name = name
    def __getattr__(self, k): return _AutoMock(self._name + '.' + k)
    def __call__(self, *a, **kw):
        print('-> ' + self._name + '() called')
        return _AutoMock(self._name + '()')
    def __repr__(self): return '<mock ' + self._name + '>'
    def __str__(self): return '<mock ' + self._name + '>'
    def __bool__(self): return True
    def __iter__(self): return iter([])
    def __len__(self): return 0
    def __getitem__(self, k): return _AutoMock(self._name + '[...]')
    def __setitem__(self, k, v): pass
    def __enter__(self): return self
    def __exit__(self, *a): return False
    async def __aenter__(self): return self
    async def __aexit__(self, *a): return False
    def __add__(self, o): return self
    def __radd__(self, o): return self
    def __sub__(self, o): return self
    def __mul__(self, o): return self
    def __rmul__(self, o): return self
    def __truediv__(self, o): return self
    def __eq__(self, o): return isinstance(o, _AutoMock)
    def __hash__(self): return hash(self._name)
    def __lt__(self, o): return True
    def __le__(self, o): return True
    def __gt__(self, o): return False
    def __ge__(self, o): return False
    def __mro_entries__(self, bases): return (object,)

def chunked(*_a, **_kw):
    print('-> chunked() called')
    return _AutoMock('chunked()')

Use itertools.islice on one shared iterator inside a loop. That is the standard pattern.

Skeleton:

python

from itertools import islice

def chunked(iterable, n):
    # TODO 1: get an iterator over the input  (iter(iterable))
    # TODO 2: loop forever, pulling the next n items via islice
    # TODO 3: stop when islice returns an empty chunk
    # TODO 4: otherwise yield the chunk as a list
    ...

Hint 1: Sharing one iterator across calls

The key is to call iter(iterable) once to get a single iterator, then call islice(iterator, n) repeatedly on the same iterator. Each islice consumes up to n items from where the last one left off.

Hint 2: Knowing when to stop

list(islice(it, n)) returns an empty list once it is exhausted. Use while True, turn each chunk into a list, and stop (break) when the chunk is empty, before you would yield it.

Show full solution

python

from itertools import islice

def chunked(iterable, n):
    """Yield successive n-sized lists from `iterable`. Final chunk may be shorter."""
    if n < 1:
        raise ValueError("n must be at least 1")
    it = iter(iterable)
    while True:
        chunk = list(islice(it, n))
        if not chunk:
            return
        yield chunk


print(list(chunked(range(10), 3)))
# [[0, 1, 2], [3, 4, 5], [6, 7, 8], [9]]

print(list(chunked("abcdefgh", 2)))
# [['a', 'b'], ['c', 'd'], ['e', 'f'], ['g', 'h']]

print(list(chunked([], 4)))
# []

Why the single it = iter(iterable)? Because islice consumes items from whatever iterator you give it. If you wrote list(islice(iterable, n)) inside the loop, every call would start a new iterator over the same list. You would get the first chunk again and again, in an infinite loop.

A one-liner using iter's two-argument form:

python

from itertools import islice

def chunked(iterable, n):
    it = iter(iterable)
    return iter(lambda: list(islice(it, n)), [])

iter(callable, sentinel) calls callable() repeatedly and stops when it returns sentinel. Here, lambda: list(islice(it, n)) returns the next chunk, and [] (an empty list) signals exhaustion. It behaves the same, but it is harder to understand. Use the loop version in code that other people read.

Bonus: chunked is now in the standard library as itertools.batched(iterable, n), from Python 3.12 onwards. It yields tuples instead of lists, but otherwise it is the same. If you're on 3.12+:

python

from itertools import batched
list(batched("abcdefgh", 2))
# [('a', 'b'), ('c', 'd'), ('e', 'f'), ('g', 'h')]

Writing it yourself is still a better way to learn, and your version also works on Python versions before 3.12.

What You Learned

Itertools functions return lazy iterators: you can combine them, they save memory, and they work only once.
The three infinite ones, count, cycle and repeat, always need a stop condition.
chain concatenates iterables, and chain.from_iterable removes one level of nesting.
takewhile/dropwhile split at the first failure, but filter checks every element.
islice gives you slicing for any iterator (no negative indices, consumes the source).
zip_longest(fillvalue=...) for uneven iterables.
groupby groups consecutive items with the same key. Sort first if you want one group per key.
accumulate for running totals/maxes/products. Watch the argument order for non-commutative operations.
product replaces n nested loops, and permutations/combinations for combinatorial searches.
pairwise (3.10+) for size-2 sliding windows.
Always check whether you need to list(...) the result, because iterators are single-use.

Review: Decorators, from earlier in this path, use the same higher-order function ideas as lambdas to wrap functions with extra behaviour.

1 · The lesson

1. Why Itertools

2. The Infinite Ones: `count`, `cycle`, `repeat`

3. `chain` and `chain.from_iterable`: Join Iterables

4. `takewhile` and `dropwhile`: Split at the First Failure

5. `islice`: Slice Any Iterable

6. `zip_longest`: Zip Without Losing Items

7. `groupby`: Group Consecutive Items With the Same Key

8. `accumulate`: Running Totals (and More)

9. `product`: Nested Loops as One Call

10. `permutations` and `combinations`

11. `pairwise` (Python 3.10+): Sliding Window of 2

Common Mistakes

🎯 Your Turn: Chunked

What You Learned

Practice this

Itertools: Composable Iterator Building Blocks

1. Why Itertools

2. The Infinite Ones: count, cycle, repeat

3. chain and chain.from_iterable: Join Iterables

4. takewhile and dropwhile: Split at the First Failure

5. islice: Slice Any Iterable

6. zip_longest: Zip Without Losing Items

7. groupby: Group Consecutive Items With the Same Key

8. accumulate: Running Totals (and More)

9. product: Nested Loops as One Call

10. permutations and combinations

11. pairwise (Python 3.10+): Sliding Window of 2

Common Mistakes

🎯 Your Turn: Chunked

What You Learned

2. The Infinite Ones: `count`, `cycle`, `repeat`

3. `chain` and `chain.from_iterable`: Join Iterables

4. `takewhile` and `dropwhile`: Split at the First Failure

5. `islice`: Slice Any Iterable

6. `zip_longest`: Zip Without Losing Items

7. `groupby`: Group Consecutive Items With the Same Key

8. `accumulate`: Running Totals (and More)

9. `product`: Nested Loops as One Call

10. `permutations` and `combinations`

11. `pairwise` (Python 3.10+): Sliding Window of 2