collections: The Stdlib's Hidden Power

Plain dict, list, and tuple cover most jobs. But every Python developer sooner or later writes a dict.get(key, 0) + 1 pattern, or a dict.setdefault(key, []).append(...), or pops items from the front of a list, which is O(n) without looking like it. The collections module is the standard library's answer: small, focused tools for patterns that appear again and again.

This lesson covers the six containers you will really use (Counter, defaultdict, deque, namedtuple, ChainMap, OrderedDict) and explains when to choose each one.

1. `Counter`: Counting Things

Counter is a dict subclass made for counting hashable things.

python

from collections import Counter

words = "the quick brown fox jumps over the lazy dog the".split()
c = Counter(words)
print(c)                # Counter({'the': 3, 'quick': 1, 'brown': 1, ...})
print(c["the"])         # 3
print(c["missing"])     # 0  (a missing key gives 0, not a KeyError)

Counter accepts any iterable. To count characters, pass a string. To count votes, pass a list of choices.

python

print(Counter("mississippi"))   # Counter({'i': 4, 's': 4, 'p': 2, 'm': 1})

+ setup added so this can run · re-runs 1 earlier block on this page first

# Lightweight mock for objects whose attributes/methods aren't critical
class _AutoMock:
    def __init__(self, name='mock'): self._name = name
    def __getattr__(self, k): return _AutoMock(self._name + '.' + k)
    def __call__(self, *a, **kw):
        print('-> ' + self._name + '() called')
        return _AutoMock(self._name + '()')
    def __repr__(self): return '<mock ' + self._name + '>'
    def __str__(self): return '<mock ' + self._name + '>'
    def __bool__(self): return True
    def __iter__(self): return iter([])
    def __len__(self): return 0
    def __getitem__(self, k): return _AutoMock(self._name + '[...]')
    def __setitem__(self, k, v): pass
    def __enter__(self): return self
    def __exit__(self, *a): return False
    async def __aenter__(self): return self
    async def __aexit__(self, *a): return False
    def __add__(self, o): return self
    def __radd__(self, o): return self
    def __sub__(self, o): return self
    def __mul__(self, o): return self
    def __rmul__(self, o): return self
    def __truediv__(self, o): return self
    def __eq__(self, o): return isinstance(o, _AutoMock)
    def __hash__(self): return hash(self._name)
    def __lt__(self, o): return True
    def __le__(self, o): return True
    def __gt__(self, o): return False
    def __ge__(self, o): return False
    def __mro_entries__(self, bases): return (object,)

def Counter(*_a, **_kw):
    print('-> Counter() called')
    return _AutoMock('Counter()')

# Earlier blocks on this page that this one builds on, re-run quietly first.
try:
    from collections import Counter
except Exception:
    pass
_pm_earlier = [
    'from collections import Counter\n\nwords = "the quick brown fox jumps over the lazy dog the".split()\nc = Counter(words)\nprint(c)                # Counter({\'the\': 3, \'quick\': 1, \'brown\': 1, ...})\nprint(c["the"])         # 3\nprint(c["missing"])     # 0  (a missing key gives 0, not a KeyError)',
]
def _pm_run_earlier(_srcs, _g):
    import sys as _s, io as _io, warnings as _w
    for _src in _srcs:
        _o, _e = _s.stdout, _s.stderr
        _s.stdout = _s.stderr = _io.StringIO()
        try:
            with _w.catch_warnings():
                _w.simplefilter('ignore')
                exec(compile(_src, '<earlier block>', 'exec'), _g)
        except (Exception, SystemExit):
            pass
        finally:
            _s.stdout, _s.stderr = _o, _e
    if 'matplotlib.pyplot' in _s.modules:
        _s.modules['matplotlib.pyplot'].close('all')   # their plots belong to their own blocks
_pm_run_earlier(_pm_earlier, globals())
del _pm_earlier, _pm_run_earlier

most_common(n) gives you the n most common items in one call. Pass n when you only want the top items, not the whole sorted list:

python

print(c.most_common(2))         # [('the', 3), ('quick', 1)]

+ setup added so this can run · re-runs 1 earlier block on this page first

# Lightweight mock for objects whose attributes/methods aren't critical
class _AutoMock:
    def __init__(self, name='mock'): self._name = name
    def __getattr__(self, k): return _AutoMock(self._name + '.' + k)
    def __call__(self, *a, **kw):
        print('-> ' + self._name + '() called')
        return _AutoMock(self._name + '()')
    def __repr__(self): return '<mock ' + self._name + '>'
    def __str__(self): return '<mock ' + self._name + '>'
    def __bool__(self): return True
    def __iter__(self): return iter([])
    def __len__(self): return 0
    def __getitem__(self, k): return _AutoMock(self._name + '[...]')
    def __setitem__(self, k, v): pass
    def __enter__(self): return self
    def __exit__(self, *a): return False
    async def __aenter__(self): return self
    async def __aexit__(self, *a): return False
    def __add__(self, o): return self
    def __radd__(self, o): return self
    def __sub__(self, o): return self
    def __mul__(self, o): return self
    def __rmul__(self, o): return self
    def __truediv__(self, o): return self
    def __eq__(self, o): return isinstance(o, _AutoMock)
    def __hash__(self): return hash(self._name)
    def __lt__(self, o): return True
    def __le__(self, o): return True
    def __gt__(self, o): return False
    def __ge__(self, o): return False
    def __mro_entries__(self, bases): return (object,)

c = _AutoMock('c')

# Earlier blocks on this page that this one builds on, re-run quietly first.
try:
    from collections import Counter
except Exception:
    pass
_pm_earlier = [
    'from collections import Counter\n\nwords = "the quick brown fox jumps over the lazy dog the".split()\nc = Counter(words)\nprint(c)                # Counter({\'the\': 3, \'quick\': 1, \'brown\': 1, ...})\nprint(c["the"])         # 3\nprint(c["missing"])     # 0  (a missing key gives 0, not a KeyError)',
]
def _pm_run_earlier(_srcs, _g):
    import sys as _s, io as _io, warnings as _w
    for _src in _srcs:
        _o, _e = _s.stdout, _s.stderr
        _s.stdout = _s.stderr = _io.StringIO()
        try:
            with _w.catch_warnings():
                _w.simplefilter('ignore')
                exec(compile(_src, '<earlier block>', 'exec'), _g)
        except (Exception, SystemExit):
            pass
        finally:
            _s.stdout, _s.stderr = _o, _e
    if 'matplotlib.pyplot' in _s.modules:
        _s.modules['matplotlib.pyplot'].close('all')   # their plots belong to their own blocks
_pm_run_earlier(_pm_earlier, globals())
del _pm_earlier, _pm_run_earlier

Counters also support arithmetic:

python

a = Counter("aabbc")            # a:2 b:2 c:1
b = Counter("abbcc")            # a:1 b:2 c:2

print(a + b)                    # adds the counts, giving a:3 b:4 c:3
print(a - b)                    # subtracts, drops counts of 0 or less, giving a:1
print(a & b)                    # smaller count of each (intersection), giving a:1 b:2 c:1
print(a | b)                    # larger count of each (union), giving a:2 b:2 c:2

+ setup added so this can run · re-runs 1 earlier block on this page first

# Lightweight mock for objects whose attributes/methods aren't critical
class _AutoMock:
    def __init__(self, name='mock'): self._name = name
    def __getattr__(self, k): return _AutoMock(self._name + '.' + k)
    def __call__(self, *a, **kw):
        print('-> ' + self._name + '() called')
        return _AutoMock(self._name + '()')
    def __repr__(self): return '<mock ' + self._name + '>'
    def __str__(self): return '<mock ' + self._name + '>'
    def __bool__(self): return True
    def __iter__(self): return iter([])
    def __len__(self): return 0
    def __getitem__(self, k): return _AutoMock(self._name + '[...]')
    def __setitem__(self, k, v): pass
    def __enter__(self): return self
    def __exit__(self, *a): return False
    async def __aenter__(self): return self
    async def __aexit__(self, *a): return False
    def __add__(self, o): return self
    def __radd__(self, o): return self
    def __sub__(self, o): return self
    def __mul__(self, o): return self
    def __rmul__(self, o): return self
    def __truediv__(self, o): return self
    def __eq__(self, o): return isinstance(o, _AutoMock)
    def __hash__(self): return hash(self._name)
    def __lt__(self, o): return True
    def __le__(self, o): return True
    def __gt__(self, o): return False
    def __ge__(self, o): return False
    def __mro_entries__(self, bases): return (object,)

def Counter(*_a, **_kw):
    print('-> Counter() called')
    return _AutoMock('Counter()')

# Earlier blocks on this page that this one builds on, re-run quietly first.
try:
    from collections import Counter
except Exception:
    pass
_pm_earlier = [
    'from collections import Counter\n\nwords = "the quick brown fox jumps over the lazy dog the".split()\nc = Counter(words)\nprint(c)                # Counter({\'the\': 3, \'quick\': 1, \'brown\': 1, ...})\nprint(c["the"])         # 3\nprint(c["missing"])     # 0  (a missing key gives 0, not a KeyError)',
]
def _pm_run_earlier(_srcs, _g):
    import sys as _s, io as _io, warnings as _w
    for _src in _srcs:
        _o, _e = _s.stdout, _s.stderr
        _s.stdout = _s.stderr = _io.StringIO()
        try:
            with _w.catch_warnings():
                _w.simplefilter('ignore')
                exec(compile(_src, '<earlier block>', 'exec'), _g)
        except (Exception, SystemExit):
            pass
        finally:
            _s.stdout, _s.stderr = _o, _e
    if 'matplotlib.pyplot' in _s.modules:
        _s.modules['matplotlib.pyplot'].close('all')   # their plots belong to their own blocks
_pm_run_earlier(_pm_earlier, globals())
del _pm_earlier, _pm_run_earlier

This lets you compare word counts or check stock levels in two lines.

2. `defaultdict`: Auto-Creating Values

The classic "group items by key" loop in plain dict:

python

people = [("eng", "Alice"), ("eng", "Bob"), ("design", "Carol"), ("eng", "Dan")]

by_team = {}
for team, name in people:
    by_team.setdefault(team, []).append(name)

defaultdict(list) makes it cleaner:

python

from collections import defaultdict

by_team = defaultdict(list)
for team, name in people:
    by_team[team].append(name)              # no setdefault needed. The first access creates []

print(dict(by_team))                        # {'eng': ['Alice', 'Bob', 'Dan'], 'design': ['Carol']}

+ setup added so this can run · re-runs 1 earlier block on this page first

people = [("alpha", 1), ("beta", 2), ("gamma", 3)]

# Earlier blocks on this page that this one builds on, re-run quietly first.
_pm_earlier = [
    'people = [("eng", "Alice"), ("eng", "Bob"), ("design", "Carol"), ("eng", "Dan")]\n\nby_team = {}\nfor team, name in people:\n    by_team.setdefault(team, []).append(name)',
]
def _pm_run_earlier(_srcs, _g):
    import sys as _s, io as _io, warnings as _w
    for _src in _srcs:
        _o, _e = _s.stdout, _s.stderr
        _s.stdout = _s.stderr = _io.StringIO()
        try:
            with _w.catch_warnings():
                _w.simplefilter('ignore')
                exec(compile(_src, '<earlier block>', 'exec'), _g)
        except (Exception, SystemExit):
            pass
        finally:
            _s.stdout, _s.stderr = _o, _e
    if 'matplotlib.pyplot' in _s.modules:
        _s.modules['matplotlib.pyplot'].close('all')   # their plots belong to their own blocks
_pm_run_earlier(_pm_earlier, globals())
del _pm_earlier, _pm_run_earlier

The argument is a factory: a callable that returns the default value. Use list for grouping, set for grouping without duplicates, and int for counting:

python

counts = defaultdict(int)
for word in "to be or not to be".split():
    counts[word] += 1                       # int() returns 0, so no key check is needed
print(dict(counts))                         # {'to': 2, 'be': 2, 'or': 1, 'not': 1}

+ setup added so this can run · re-runs 2 earlier blocks on this page first

# Lightweight mock for objects whose attributes/methods aren't critical
class _AutoMock:
    def __init__(self, name='mock'): self._name = name
    def __getattr__(self, k): return _AutoMock(self._name + '.' + k)
    def __call__(self, *a, **kw):
        print('-> ' + self._name + '() called')
        return _AutoMock(self._name + '()')
    def __repr__(self): return '<mock ' + self._name + '>'
    def __str__(self): return '<mock ' + self._name + '>'
    def __bool__(self): return True
    def __iter__(self): return iter([])
    def __len__(self): return 0
    def __getitem__(self, k): return _AutoMock(self._name + '[...]')
    def __setitem__(self, k, v): pass
    def __enter__(self): return self
    def __exit__(self, *a): return False
    async def __aenter__(self): return self
    async def __aexit__(self, *a): return False
    def __add__(self, o): return self
    def __radd__(self, o): return self
    def __sub__(self, o): return self
    def __mul__(self, o): return self
    def __rmul__(self, o): return self
    def __truediv__(self, o): return self
    def __eq__(self, o): return isinstance(o, _AutoMock)
    def __hash__(self): return hash(self._name)
    def __lt__(self, o): return True
    def __le__(self, o): return True
    def __gt__(self, o): return False
    def __ge__(self, o): return False
    def __mro_entries__(self, bases): return (object,)

def defaultdict(*_a, **_kw):
    print('-> defaultdict() called')
    return _AutoMock('defaultdict()')
people = [("alpha", 1), ("beta", 2), ("gamma", 3)]

# Earlier blocks on this page that this one builds on, re-run quietly first.
try:
    from collections import defaultdict
except Exception:
    pass
_pm_earlier = [
    'people = [("eng", "Alice"), ("eng", "Bob"), ("design", "Carol"), ("eng", "Dan")]\n\nby_team = {}\nfor team, name in people:\n    by_team.setdefault(team, []).append(name)',
    "from collections import defaultdict\n\nby_team = defaultdict(list)\nfor team, name in people:\n    by_team[team].append(name)              # no setdefault needed. The first access creates []\n\nprint(dict(by_team))                        # {'eng': ['Alice', 'Bob', 'Dan'], 'design': ['Carol']}",
]
def _pm_run_earlier(_srcs, _g):
    import sys as _s, io as _io, warnings as _w
    for _src in _srcs:
        _o, _e = _s.stdout, _s.stderr
        _s.stdout = _s.stderr = _io.StringIO()
        try:
            with _w.catch_warnings():
                _w.simplefilter('ignore')
                exec(compile(_src, '<earlier block>', 'exec'), _g)
        except (Exception, SystemExit):
            pass
        finally:
            _s.stdout, _s.stderr = _o, _e
    if 'matplotlib.pyplot' in _s.modules:
        _s.modules['matplotlib.pyplot'].close('all')   # their plots belong to their own blocks
_pm_run_earlier(_pm_earlier, globals())
del _pm_earlier, _pm_run_earlier

For simple counting, Counter is still better. defaultdict(int) is more useful when you update the value in other ways than += 1.

3. `deque`: Fast at Both Ends

A list is fast for indexing and appending. But popping from the front (list.pop(0)) is O(n), because Python has to move every remaining element. A deque ("deck") is O(1) at both ends.

python

from collections import deque

queue = deque(["a", "b", "c"])
queue.append("d")               # right side
queue.appendleft("z")           # left side
print(queue)                    # deque(['z', 'a', 'b', 'c', 'd'])

print(queue.popleft())          # 'z'   (append + popleft makes a FIFO queue)
print(queue.pop())              # 'd'   (append + pop makes a LIFO stack)

Use cases:

FIFO queues: producer/consumer patterns.

Sliding windows: pass maxlen and the oldest items are removed automatically:

python

recent = deque(maxlen=3)
for n in [1, 2, 3, 4, 5]:
    recent.append(n)
    print(list(recent))
# [1] → [1,2] → [1,2,3] → [2,3,4] → [3,4,5]

+ setup added so this can run · re-runs 1 earlier block on this page first

# Lightweight mock for objects whose attributes/methods aren't critical
class _AutoMock:
    def __init__(self, name='mock'): self._name = name
    def __getattr__(self, k): return _AutoMock(self._name + '.' + k)
    def __call__(self, *a, **kw):
        print('-> ' + self._name + '() called')
        return _AutoMock(self._name + '()')
    def __repr__(self): return '<mock ' + self._name + '>'
    def __str__(self): return '<mock ' + self._name + '>'
    def __bool__(self): return True
    def __iter__(self): return iter([])
    def __len__(self): return 0
    def __getitem__(self, k): return _AutoMock(self._name + '[...]')
    def __setitem__(self, k, v): pass
    def __enter__(self): return self
    def __exit__(self, *a): return False
    async def __aenter__(self): return self
    async def __aexit__(self, *a): return False
    def __add__(self, o): return self
    def __radd__(self, o): return self
    def __sub__(self, o): return self
    def __mul__(self, o): return self
    def __rmul__(self, o): return self
    def __truediv__(self, o): return self
    def __eq__(self, o): return isinstance(o, _AutoMock)
    def __hash__(self): return hash(self._name)
    def __lt__(self, o): return True
    def __le__(self, o): return True
    def __gt__(self, o): return False
    def __ge__(self, o): return False
    def __mro_entries__(self, bases): return (object,)

def deque(*_a, **_kw):
    print('-> deque() called')
    return _AutoMock('deque()')

# Earlier blocks on this page that this one builds on, re-run quietly first.
try:
    from collections import deque
except Exception:
    pass
_pm_earlier = [
    'from collections import deque\n\nqueue = deque(["a", "b", "c"])\nqueue.append("d")               # right side\nqueue.appendleft("z")           # left side\nprint(queue)                    # deque([\'z\', \'a\', \'b\', \'c\', \'d\'])\n\nprint(queue.popleft())          # \'z\'   (append + popleft makes a FIFO queue)\nprint(queue.pop())              # \'d\'   (append + pop makes a LIFO stack)',
]
def _pm_run_earlier(_srcs, _g):
    import sys as _s, io as _io, warnings as _w
    for _src in _srcs:
        _o, _e = _s.stdout, _s.stderr
        _s.stdout = _s.stderr = _io.StringIO()
        try:
            with _w.catch_warnings():
                _w.simplefilter('ignore')
                exec(compile(_src, '<earlier block>', 'exec'), _g)
        except (Exception, SystemExit):
            pass
        finally:
            _s.stdout, _s.stderr = _o, _e
    if 'matplotlib.pyplot' in _s.modules:
        _s.modules['matplotlib.pyplot'].close('all')   # their plots belong to their own blocks
_pm_run_earlier(_pm_earlier, globals())
del _pm_earlier, _pm_run_earlier

The end of a log: keep the last N lines without holding the whole file in memory.

If you write lst.pop(0) in a loop that runs many times, use a deque instead.

4. `namedtuple`: Tuples With Field Names

A tuple is a good lightweight record, until you write point[0] and point[1] and forget which is which. namedtuple is still an immutable tuple, but it adds field names:

python

from collections import namedtuple

Point = namedtuple("Point", ["x", "y"])
p = Point(3, 4)
print(p.x, p.y)                 # 3 4
print(p[0], p[1])               # 3 4   (you can still use an index)
print(p)                        # Point(x=3, y=4)

It's hashable, immutable, and unpacks normally:

python

x, y = p                        # tuple unpacking works
points = {Point(0, 0): "origin"}   # works as a dict key

+ setup added so this can run · re-runs 1 earlier block on this page first

# Lightweight mock for objects whose attributes/methods aren't critical
class _AutoMock:
    def __init__(self, name='mock'): self._name = name
    def __getattr__(self, k): return _AutoMock(self._name + '.' + k)
    def __call__(self, *a, **kw):
        print('-> ' + self._name + '() called')
        return _AutoMock(self._name + '()')
    def __repr__(self): return '<mock ' + self._name + '>'
    def __str__(self): return '<mock ' + self._name + '>'
    def __bool__(self): return True
    def __iter__(self): return iter([])
    def __len__(self): return 0
    def __getitem__(self, k): return _AutoMock(self._name + '[...]')
    def __setitem__(self, k, v): pass
    def __enter__(self): return self
    def __exit__(self, *a): return False
    async def __aenter__(self): return self
    async def __aexit__(self, *a): return False
    def __add__(self, o): return self
    def __radd__(self, o): return self
    def __sub__(self, o): return self
    def __mul__(self, o): return self
    def __rmul__(self, o): return self
    def __truediv__(self, o): return self
    def __eq__(self, o): return isinstance(o, _AutoMock)
    def __hash__(self): return hash(self._name)
    def __lt__(self, o): return True
    def __le__(self, o): return True
    def __gt__(self, o): return False
    def __ge__(self, o): return False
    def __mro_entries__(self, bases): return (object,)

p = _AutoMock('p')
def Point(*_a, **_kw):
    print('-> Point() called')
    return _AutoMock('Point()')

# Earlier blocks on this page that this one builds on, re-run quietly first.
try:
    from collections import namedtuple
except Exception:
    pass
_pm_earlier = [
    'from collections import namedtuple\n\nPoint = namedtuple("Point", ["x", "y"])\np = Point(3, 4)\nprint(p.x, p.y)                 # 3 4\nprint(p[0], p[1])               # 3 4   (you can still use an index)\nprint(p)                        # Point(x=3, y=4)',
]
def _pm_run_earlier(_srcs, _g):
    import sys as _s, io as _io, warnings as _w
    for _src in _srcs:
        _o, _e = _s.stdout, _s.stderr
        _s.stdout = _s.stderr = _io.StringIO()
        try:
            with _w.catch_warnings():
                _w.simplefilter('ignore')
                exec(compile(_src, '<earlier block>', 'exec'), _g)
        except (Exception, SystemExit):
            pass
        finally:
            _s.stdout, _s.stderr = _o, _e
    if 'matplotlib.pyplot' in _s.modules:
        _s.modules['matplotlib.pyplot'].close('all')   # their plots belong to their own blocks
_pm_run_earlier(_pm_earlier, globals())
del _pm_earlier, _pm_run_earlier

In modern code, use typing.NamedTuple, which gives you a class syntax with type hints. Prefer it in new code:

python

from typing import NamedTuple

class Point(NamedTuple):
    x: float
    y: float
    label: str = ""             # default values are allowed

p = Point(3, 4, "origin")
print(p.label)                  # origin

Rule of thumb: use NamedTuple for very light, immutable value objects (coordinates, RGB colours, records read from a CSV). Use @dataclass when you need more: mutation, __post_init__ validation, custom comparison. If you want a dataclass that can't change, use @dataclass(frozen=True).

5. `ChainMap`: Layered Lookups

ChainMap searches several dicts in order. This is ideal for config that comes in layers (defaults → file → environment → CLI args):

python

from collections import ChainMap

defaults = {"host": "localhost", "port": 8000, "debug": False}
env      = {"port": 9000}
cli      = {"debug": True}

config = ChainMap(cli, env, defaults)       # the leftmost dict wins
print(config["host"])                       # 'localhost'  (from defaults)
print(config["port"])                       # 9000         (env beats defaults)
print(config["debug"])                      # True         (cli beats env)

Writes go only to the first map (cli here). The dicts are not merged, and the other dicts don't change:

python

config["host"] = "remote"
print(cli)                                  # {'debug': True, 'host': 'remote'}
print(defaults)                             # unchanged

+ setup added so this can run · re-runs 1 earlier block on this page first

# Lightweight mock for objects whose attributes/methods aren't critical
class _AutoMock:
    def __init__(self, name='mock'): self._name = name
    def __getattr__(self, k): return _AutoMock(self._name + '.' + k)
    def __call__(self, *a, **kw):
        print('-> ' + self._name + '() called')
        return _AutoMock(self._name + '()')
    def __repr__(self): return '<mock ' + self._name + '>'
    def __str__(self): return '<mock ' + self._name + '>'
    def __bool__(self): return True
    def __iter__(self): return iter([])
    def __len__(self): return 0
    def __getitem__(self, k): return _AutoMock(self._name + '[...]')
    def __setitem__(self, k, v): pass
    def __enter__(self): return self
    def __exit__(self, *a): return False
    async def __aenter__(self): return self
    async def __aexit__(self, *a): return False
    def __add__(self, o): return self
    def __radd__(self, o): return self
    def __sub__(self, o): return self
    def __mul__(self, o): return self
    def __rmul__(self, o): return self
    def __truediv__(self, o): return self
    def __eq__(self, o): return isinstance(o, _AutoMock)
    def __hash__(self): return hash(self._name)
    def __lt__(self, o): return True
    def __le__(self, o): return True
    def __gt__(self, o): return False
    def __ge__(self, o): return False
    def __mro_entries__(self, bases): return (object,)

config = _AutoMock('config')
cli = _AutoMock('cli')
defaults = _AutoMock('defaults')

# Earlier blocks on this page that this one builds on, re-run quietly first.
try:
    from collections import ChainMap
except Exception:
    pass
_pm_earlier = [
    'from collections import ChainMap\n\ndefaults = {"host": "localhost", "port": 8000, "debug": False}\nenv      = {"port": 9000}\ncli      = {"debug": True}\n\nconfig = ChainMap(cli, env, defaults)       # the leftmost dict wins\nprint(config["host"])                       # \'localhost\'  (from defaults)\nprint(config["port"])                       # 9000         (env beats defaults)\nprint(config["debug"])                      # True         (cli beats env)',
]
def _pm_run_earlier(_srcs, _g):
    import sys as _s, io as _io, warnings as _w
    for _src in _srcs:
        _o, _e = _s.stdout, _s.stderr
        _s.stdout = _s.stderr = _io.StringIO()
        try:
            with _w.catch_warnings():
                _w.simplefilter('ignore')
                exec(compile(_src, '<earlier block>', 'exec'), _g)
        except (Exception, SystemExit):
            pass
        finally:
            _s.stdout, _s.stderr = _o, _e
    if 'matplotlib.pyplot' in _s.modules:
        _s.modules['matplotlib.pyplot'].close('all')   # their plots belong to their own blocks
_pm_run_earlier(_pm_earlier, globals())
del _pm_earlier, _pm_run_earlier

It's a lightweight alternative to merging three dicts every time you need a config value.

6. `OrderedDict`: Mostly Replaced by `dict`

Before Python 3.7, regular dict made no order guarantees. OrderedDict preserved insertion order. Since 3.7, the language guarantees that dict preserves insertion order, so OrderedDict is mostly needed only in old code.

Two methods still make it useful:

python

from collections import OrderedDict

od = OrderedDict([("a", 1), ("b", 2), ("c", 3)])

od.move_to_end("a")             # move existing key to the end
print(od)                       # OrderedDict({'b': 2, 'c': 3, 'a': 1})

od.move_to_end("c", last=False) # ...or to the front
print(od)                       # OrderedDict({'c': 3, 'b': 2, 'a': 1})

print(od.popitem(last=False))   # ('c', 3)  (pops from the front)

These are useful for LRU-cache style structures. For everything else, use a plain dict.

7. Which Container, When?

Need	Use
Count occurrences of hashable things	`Counter`
Group items by key into lists/sets	`defaultdict(list / set)`
FIFO queue or sliding window	`deque(maxlen=...)`
Lightweight immutable record	`typing.NamedTuple`
Record with methods, defaults, mutation	`@dataclass`
Layered config lookup	`ChainMap`
LRU-style "move recent to end"	`OrderedDict.move_to_end`
Anything else	plain `dict` / `list`

When in doubt, write it with plain types first. Switch to a specialised container only when the pattern is clear or the performance matters.

Common Mistakes

dict.setdefault(k, []).append(v) everywhere: it works, but defaultdict(list) is shorter and faster. Use setdefault only for one-off cases.
Counter.most_common() without an n: it returns all items sorted, which wastes work if you only want the top 5. Pass n.
deque without maxlen when you wanted a sliding window: it will grow without limit. The sliding-window pattern depends on maxlen.
Treating namedtuple like a mutable class: it's immutable. Trying to assign p.x = 5 raises AttributeError. For state that changes, use a dataclass.
Using OrderedDict when plain dict works: a modern Python dict already preserves insertion order. Only use OrderedDict if you need move_to_end or popitem(last=False).
Forgetting Counter counts can be negative after subtraction: Counter arithmetic with + and - drops zero and negative counts, but c["x"] -= 1 can leave a negative count. Use unary plus to clean them up: +c returns a new Counter without zero or negative counts.

🎯 Your Turn: Word Counter and Rolling Average

Two small functions, both of which use containers you just met.

Part 1: top_words(text, n=10) returns the top-n most common words in text. Lowercase the words, and keep only "word characters" (letters, digits, underscore: use re.findall(r"\w+", ...)). You don't need to filter out stopwords.

Part 2: rolling_average(stream, window=5) is a generator that reads an iterable of numbers and yields the running average over the last window items. For the first window-1 items, yield the partial average (average of what you have so far).

python

import re
from collections import Counter, deque

def top_words(text, n=10):
    """Return list of (word, count) for the n most common words."""
    # TODO 1: tokenize with re.findall(r"\w+", text.lower())
    # TODO 2: count the tokens with Counter
    # TODO 3: return .most_common(n)
    pass

def rolling_average(stream, window=5):
    """Yield the running mean over the last `window` items from `stream`."""
    # TODO 1: deque with maxlen=window
    # TODO 2: for each item, append, then yield sum(buf) / len(buf)
    pass

# Tests
text = "The rain in Spain falls mainly on the plain. The plain is plain."
print(top_words(text, 3))
# Expected: [('the', 3), ('plain', 3), ('rain', 1)]   (ties keep first-seen order)

avgs = list(rolling_average([10, 20, 30, 40, 50], window=3))
print(avgs)
# Expected: [10.0, 15.0, 20.0, 30.0, 40.0]
#            └─ 10/1   (10+20)/2   (10+20+30)/3   (20+30+40)/3   (30+40+50)/3

Hint 1: Splitting text into words

re.findall(r"\w+", text.lower()) returns a list of all word-character runs, lowercased. Punctuation and whitespace are skipped automatically.

Hint 2: Sliding mean

A deque(maxlen=window) automatically drops the oldest item when you append beyond maxlen. So sum(buf) / len(buf) always gives the average of the most recent ≤ window items, even before the window is full.

Show full solution

python

import re
from collections import Counter, deque

def top_words(text, n=10):
    tokens = re.findall(r"\w+", text.lower())
    return Counter(tokens).most_common(n)

def rolling_average(stream, window=5):
    buf = deque(maxlen=window)
    for item in stream:
        buf.append(item)
        yield sum(buf) / len(buf)

# Tests
text = "The rain in Spain falls mainly on the plain. The plain is plain."
print(top_words(text, 3))
# [('the', 3), ('plain', 3), ('rain', 1)]

print(list(rolling_average([10, 20, 30, 40, 50], window=3)))
# [10.0, 15.0, 20.0, 30.0, 40.0]

The two functions together have five lines of real logic. That's the point of collections: the container does the counting and bookkeeping, and your code only states the intent.

A faster version of rolling_average would keep a running sum, adding the new item and subtracting the item that leaves the window (O(1) per yield instead of O(window)). For small windows the difference is tiny, but for windows of thousands of items, use the faster version.

What You Learned

Counter(iterable) counts hashable items. Use most_common(n), and arithmetic (+ - & |) to combine counts.
defaultdict(factory) creates missing values automatically, and list/set/int are the common factories.
deque is O(1) at both ends, and maxlen gives you a sliding window.
namedtuple / typing.NamedTuple are immutable records with field names. Prefer the typed class form. Use @dataclass when you need behaviour.
ChainMap looks up keys across several dicts in order. It is ideal for layered config.
OrderedDict is mostly needed in old code, but move_to_end and popitem(last=False) keep it relevant for LRU patterns.
The decision table above is worth learning. Choosing the right container is a big part of writing clean Python.

Review: itertools, from earlier in this path, works well with these containers (chain, groupby, islice, accumulate, combinations).

1 · The lesson

1. `Counter`: Counting Things

2. `defaultdict`: Auto-Creating Values

3. `deque`: Fast at Both Ends

4. `namedtuple`: Tuples With Field Names

5. `ChainMap`: Layered Lookups

6. `OrderedDict`: Mostly Replaced by `dict`

7. Which Container, When?

Common Mistakes

🎯 Your Turn: Word Counter and Rolling Average

What You Learned

Practice this

collections: The Stdlib's Hidden Power

1. Counter: Counting Things

2. defaultdict: Auto-Creating Values

3. deque: Fast at Both Ends

4. namedtuple: Tuples With Field Names

5. ChainMap: Layered Lookups

6. OrderedDict: Mostly Replaced by dict

7. Which Container, When?

Common Mistakes

🎯 Your Turn: Word Counter and Rolling Average

What You Learned

1. `Counter`: Counting Things

2. `defaultdict`: Auto-Creating Values

3. `deque`: Fast at Both Ends

4. `namedtuple`: Tuples With Field Names

5. `ChainMap`: Layered Lookups

6. `OrderedDict`: Mostly Replaced by `dict`