collections: The Stdlib's Hidden Power

Plain dict, list, and tuple cover most jobs. But every Python developer eventually writes a dict.get(key, 0) + 1 pattern, or a dict.setdefault(key, []).append(...), or a list-pop-from-front that's secretly O(n). The collections module is the standard library's answer: small, sharp tools for patterns that show up over and over.

This lesson walks through the six containers you'll actually use — Counter, defaultdict, deque, namedtuple, ChainMap, OrderedDict — with the decision rules for when to pick each.

1. `Counter` — The Universal Tally

Counter is a dict subclass purpose-built for counting hashable things.

python

from collections import Counter

words = "the quick brown fox jumps over the lazy dog the".split()
c = Counter(words)
print(c)                # Counter({'the': 3, 'quick': 1, 'brown': 1, ...})
print(c["the"])         # 3
print(c["missing"])     # 0  — missing keys return 0, not KeyError

The signature accepts any iterable. Want character frequencies? Pass a string. Vote tallies? Pass a list of choices.

python

print(Counter("mississippi"))   # Counter({'i': 4, 's': 4, 'p': 2, 'm': 1})

+ setup added so this can run · defines Counter

# Lightweight mock for objects whose attributes/methods aren't critical
class _AutoMock:
    def __init__(self, name='mock'): self._name = name
    def __getattr__(self, k): return _AutoMock(self._name + '.' + k)
    def __call__(self, *a, **kw):
        print('-> ' + self._name + '() called')
        return _AutoMock(self._name + '()')
    def __repr__(self): return '<mock ' + self._name + '>'
    def __str__(self): return '<mock ' + self._name + '>'
    def __bool__(self): return True
    def __iter__(self): return iter([])
    def __len__(self): return 0
    def __getitem__(self, k): return _AutoMock(self._name + '[...]')
    def __setitem__(self, k, v): pass
    def __enter__(self): return self
    def __exit__(self, *a): return False
    async def __aenter__(self): return self
    async def __aexit__(self, *a): return False
    def __add__(self, o): return self
    def __radd__(self, o): return self
    def __sub__(self, o): return self
    def __mul__(self, o): return self
    def __rmul__(self, o): return self
    def __truediv__(self, o): return self
    def __eq__(self, o): return isinstance(o, _AutoMock)
    def __hash__(self): return hash(self._name)
    def __lt__(self, o): return True
    def __le__(self, o): return True
    def __gt__(self, o): return False
    def __ge__(self, o): return False
    def __mro_entries__(self, bases): return (object,)

def Counter(*_a, **_kw):
    print('-> Counter() called')
    return _AutoMock('Counter()')

most_common(n) gives you the top-n in one call — pass n when you only want a slice, not the whole sorted list:

python

print(c.most_common(2))         # [('the', 3), ('quick', 1)]

+ setup added so this can run · defines c

# Lightweight mock for objects whose attributes/methods aren't critical
class _AutoMock:
    def __init__(self, name='mock'): self._name = name
    def __getattr__(self, k): return _AutoMock(self._name + '.' + k)
    def __call__(self, *a, **kw):
        print('-> ' + self._name + '() called')
        return _AutoMock(self._name + '()')
    def __repr__(self): return '<mock ' + self._name + '>'
    def __str__(self): return '<mock ' + self._name + '>'
    def __bool__(self): return True
    def __iter__(self): return iter([])
    def __len__(self): return 0
    def __getitem__(self, k): return _AutoMock(self._name + '[...]')
    def __setitem__(self, k, v): pass
    def __enter__(self): return self
    def __exit__(self, *a): return False
    async def __aenter__(self): return self
    async def __aexit__(self, *a): return False
    def __add__(self, o): return self
    def __radd__(self, o): return self
    def __sub__(self, o): return self
    def __mul__(self, o): return self
    def __rmul__(self, o): return self
    def __truediv__(self, o): return self
    def __eq__(self, o): return isinstance(o, _AutoMock)
    def __hash__(self): return hash(self._name)
    def __lt__(self, o): return True
    def __le__(self, o): return True
    def __gt__(self, o): return False
    def __ge__(self, o): return False
    def __mro_entries__(self, bases): return (object,)

c = _AutoMock('c')

Counters also support arithmetic:

python

a = Counter("aabbc")            # a:2 b:2 c:1
b = Counter("abbcc")            # a:1 b:2 c:2

print(a + b)                    # union sum  → a:3 b:4 c:3
print(a - b)                    # subtract, drop zero/negative → a:1
print(a & b)                    # min (intersection) → a:1 b:2 c:1
print(a | b)                    # max (union)        → a:2 b:2 c:2

+ setup added so this can run · defines Counter

# Lightweight mock for objects whose attributes/methods aren't critical
class _AutoMock:
    def __init__(self, name='mock'): self._name = name
    def __getattr__(self, k): return _AutoMock(self._name + '.' + k)
    def __call__(self, *a, **kw):
        print('-> ' + self._name + '() called')
        return _AutoMock(self._name + '()')
    def __repr__(self): return '<mock ' + self._name + '>'
    def __str__(self): return '<mock ' + self._name + '>'
    def __bool__(self): return True
    def __iter__(self): return iter([])
    def __len__(self): return 0
    def __getitem__(self, k): return _AutoMock(self._name + '[...]')
    def __setitem__(self, k, v): pass
    def __enter__(self): return self
    def __exit__(self, *a): return False
    async def __aenter__(self): return self
    async def __aexit__(self, *a): return False
    def __add__(self, o): return self
    def __radd__(self, o): return self
    def __sub__(self, o): return self
    def __mul__(self, o): return self
    def __rmul__(self, o): return self
    def __truediv__(self, o): return self
    def __eq__(self, o): return isinstance(o, _AutoMock)
    def __hash__(self): return hash(self._name)
    def __lt__(self, o): return True
    def __le__(self, o): return True
    def __gt__(self, o): return False
    def __ge__(self, o): return False
    def __mro_entries__(self, bases): return (object,)

def Counter(*_a, **_kw):
    print('-> Counter() called')
    return _AutoMock('Counter()')

Useful for anything from word-count diffs to inventory reconciliation in two lines.

2. `defaultdict` — Auto-Creating Values

The classic "group items by key" loop in plain dict:

python

people = [("eng", "Alice"), ("eng", "Bob"), ("design", "Carol"), ("eng", "Dan")]

by_team = {}
for team, name in people:
    by_team.setdefault(team, []).append(name)

defaultdict(list) makes it cleaner:

python

from collections import defaultdict

by_team = defaultdict(list)
for team, name in people:
    by_team[team].append(name)              # no setdefault — first access creates []

print(dict(by_team))                        # {'eng': ['Alice', 'Bob', 'Dan'], 'design': ['Carol']}

+ setup added so this can run · defines people

people = [("alpha", 1), ("beta", 2), ("gamma", 3)]

The argument is a factory — a callable that returns the default value. list for grouping, set for de-duped grouping, int for counting:

python

counts = defaultdict(int)
for word in "to be or not to be".split():
    counts[word] += 1                       # int() returns 0 — no key-check needed
print(dict(counts))                         # {'to': 2, 'be': 2, 'or': 1, 'not': 1}

+ setup added so this can run · defines defaultdict

# Lightweight mock for objects whose attributes/methods aren't critical
class _AutoMock:
    def __init__(self, name='mock'): self._name = name
    def __getattr__(self, k): return _AutoMock(self._name + '.' + k)
    def __call__(self, *a, **kw):
        print('-> ' + self._name + '() called')
        return _AutoMock(self._name + '()')
    def __repr__(self): return '<mock ' + self._name + '>'
    def __str__(self): return '<mock ' + self._name + '>'
    def __bool__(self): return True
    def __iter__(self): return iter([])
    def __len__(self): return 0
    def __getitem__(self, k): return _AutoMock(self._name + '[...]')
    def __setitem__(self, k, v): pass
    def __enter__(self): return self
    def __exit__(self, *a): return False
    async def __aenter__(self): return self
    async def __aexit__(self, *a): return False
    def __add__(self, o): return self
    def __radd__(self, o): return self
    def __sub__(self, o): return self
    def __mul__(self, o): return self
    def __rmul__(self, o): return self
    def __truediv__(self, o): return self
    def __eq__(self, o): return isinstance(o, _AutoMock)
    def __hash__(self): return hash(self._name)
    def __lt__(self, o): return True
    def __le__(self, o): return True
    def __gt__(self, o): return False
    def __ge__(self, o): return False
    def __mro_entries__(self, bases): return (object,)

def defaultdict(*_a, **_kw):
    print('-> defaultdict() called')
    return _AutoMock('defaultdict()')

For pure counting, Counter is still nicer. defaultdict(int) shines when the value transforms beyond a simple += 1.

3. `deque` — Fast at Both Ends

A list is great for indexing and appending. Pop from the front (list.pop(0)) and it's O(n) — Python has to shift every remaining element. A deque ("deck") is O(1) at both ends.

python

from collections import deque

queue = deque(["a", "b", "c"])
queue.append("d")               # right side
queue.appendleft("z")           # left side
print(queue)                    # deque(['z', 'a', 'b', 'c', 'd'])

print(queue.popleft())          # 'z'   — FIFO queue
print(queue.pop())              # 'd'   — LIFO stack

Use cases:

FIFO queues — producer/consumer patterns.

Sliding windows — pass maxlen and oldest items drop off automatically:

python

recent = deque(maxlen=3)
for n in [1, 2, 3, 4, 5]:
    recent.append(n)
    print(list(recent))
# [1] → [1,2] → [1,2,3] → [2,3,4] → [3,4,5]

+ setup added so this can run · defines deque

# Lightweight mock for objects whose attributes/methods aren't critical
class _AutoMock:
    def __init__(self, name='mock'): self._name = name
    def __getattr__(self, k): return _AutoMock(self._name + '.' + k)
    def __call__(self, *a, **kw):
        print('-> ' + self._name + '() called')
        return _AutoMock(self._name + '()')
    def __repr__(self): return '<mock ' + self._name + '>'
    def __str__(self): return '<mock ' + self._name + '>'
    def __bool__(self): return True
    def __iter__(self): return iter([])
    def __len__(self): return 0
    def __getitem__(self, k): return _AutoMock(self._name + '[...]')
    def __setitem__(self, k, v): pass
    def __enter__(self): return self
    def __exit__(self, *a): return False
    async def __aenter__(self): return self
    async def __aexit__(self, *a): return False
    def __add__(self, o): return self
    def __radd__(self, o): return self
    def __sub__(self, o): return self
    def __mul__(self, o): return self
    def __rmul__(self, o): return self
    def __truediv__(self, o): return self
    def __eq__(self, o): return isinstance(o, _AutoMock)
    def __hash__(self): return hash(self._name)
    def __lt__(self, o): return True
    def __le__(self, o): return True
    def __gt__(self, o): return False
    def __ge__(self, o): return False
    def __mro_entries__(self, bases): return (object,)

def deque(*_a, **_kw):
    print('-> deque() called')
    return _AutoMock('deque()')

Tail-of-log — keep the last N lines without holding the whole file in memory.

If you ever find yourself writing lst.pop(0) in a hot loop, swap to deque.

4. `namedtuple` — Tuples With Field Names

A tuple is a great lightweight record — until you find yourself writing point[0] and point[1] and forgetting which is which. namedtuple keeps the immutability and the tuple-ness, but adds names:

python

from collections import namedtuple

Point = namedtuple("Point", ["x", "y"])
p = Point(3, 4)
print(p.x, p.y)                 # 3 4
print(p[0], p[1])               # 3 4   — still indexable
print(p)                        # Point(x=3, y=4)

It's hashable, immutable, and unpacks normally:

python

x, y = p                        # tuple unpacking works
points = {Point(0, 0): "origin"}   # works as a dict key

+ setup added so this can run · defines p, Point

# Lightweight mock for objects whose attributes/methods aren't critical
class _AutoMock:
    def __init__(self, name='mock'): self._name = name
    def __getattr__(self, k): return _AutoMock(self._name + '.' + k)
    def __call__(self, *a, **kw):
        print('-> ' + self._name + '() called')
        return _AutoMock(self._name + '()')
    def __repr__(self): return '<mock ' + self._name + '>'
    def __str__(self): return '<mock ' + self._name + '>'
    def __bool__(self): return True
    def __iter__(self): return iter([])
    def __len__(self): return 0
    def __getitem__(self, k): return _AutoMock(self._name + '[...]')
    def __setitem__(self, k, v): pass
    def __enter__(self): return self
    def __exit__(self, *a): return False
    async def __aenter__(self): return self
    async def __aexit__(self, *a): return False
    def __add__(self, o): return self
    def __radd__(self, o): return self
    def __sub__(self, o): return self
    def __mul__(self, o): return self
    def __rmul__(self, o): return self
    def __truediv__(self, o): return self
    def __eq__(self, o): return isinstance(o, _AutoMock)
    def __hash__(self): return hash(self._name)
    def __lt__(self, o): return True
    def __le__(self, o): return True
    def __gt__(self, o): return False
    def __ge__(self, o): return False
    def __mro_entries__(self, bases): return (object,)

p = _AutoMock('p')
def Point(*_a, **_kw):
    print('-> Point() called')
    return _AutoMock('Point()')

Modern preference: use typing.NamedTuple for the class-based syntax with type hints. Recommend this in any new code:

python

from typing import NamedTuple

class Point(NamedTuple):
    x: float
    y: float
    label: str = ""             # defaults supported

p = Point(3, 4, "origin")
print(p.label)                  # origin

Rule of thumb: NamedTuple for ultra-light, immutable value objects (coordinates, RGB colours, records read from a CSV). Use @dataclass(frozen=True) when you need richer behaviour — methods, mutation, __post_init__ validation, comparison overrides.

5. `ChainMap` — Layered Lookups

ChainMap searches multiple dicts in order — perfect for layered config (defaults → file → environment → CLI args):

python

from collections import ChainMap

defaults = {"host": "localhost", "port": 8000, "debug": False}
env      = {"port": 9000}
cli      = {"debug": True}

config = ChainMap(cli, env, defaults)       # leftmost wins
print(config["host"])                       # 'localhost'  (from defaults)
print(config["port"])                       # 9000         (env beats defaults)
print(config["debug"])                      # True         (cli beats env)

Writes go to the first map only — the underlying dicts aren't merged or mutated:

python

config["host"] = "remote"
print(cli)                                  # {'debug': True, 'host': 'remote'}
print(defaults)                             # unchanged

+ setup added so this can run · defines config, cli, defaults

# Lightweight mock for objects whose attributes/methods aren't critical
class _AutoMock:
    def __init__(self, name='mock'): self._name = name
    def __getattr__(self, k): return _AutoMock(self._name + '.' + k)
    def __call__(self, *a, **kw):
        print('-> ' + self._name + '() called')
        return _AutoMock(self._name + '()')
    def __repr__(self): return '<mock ' + self._name + '>'
    def __str__(self): return '<mock ' + self._name + '>'
    def __bool__(self): return True
    def __iter__(self): return iter([])
    def __len__(self): return 0
    def __getitem__(self, k): return _AutoMock(self._name + '[...]')
    def __setitem__(self, k, v): pass
    def __enter__(self): return self
    def __exit__(self, *a): return False
    async def __aenter__(self): return self
    async def __aexit__(self, *a): return False
    def __add__(self, o): return self
    def __radd__(self, o): return self
    def __sub__(self, o): return self
    def __mul__(self, o): return self
    def __rmul__(self, o): return self
    def __truediv__(self, o): return self
    def __eq__(self, o): return isinstance(o, _AutoMock)
    def __hash__(self): return hash(self._name)
    def __lt__(self, o): return True
    def __le__(self, o): return True
    def __gt__(self, o): return False
    def __ge__(self, o): return False
    def __mro_entries__(self, bases): return (object,)

config = _AutoMock('config')
cli = _AutoMock('cli')
defaults = _AutoMock('defaults')

It's a lightweight alternative to merging three dicts every time you need a config value.

6. `OrderedDict` — Mostly Obsolete

Before Python 3.7, regular dict made no order guarantees. OrderedDict preserved insertion order. Since 3.7, dict preserves insertion order in the language spec — so OrderedDict is mostly historical.

Two methods make it still useful:

python

from collections import OrderedDict

od = OrderedDict([("a", 1), ("b", 2), ("c", 3)])

od.move_to_end("a")             # move existing key to the end
print(od)                       # OrderedDict([('b', 2), ('c', 3), ('a', 1)])

od.move_to_end("c", last=False) # ...or to the front
print(od)                       # OrderedDict([('c', 3), ('b', 2), ('a', 1)])

print(od.popitem(last=False))   # ('c', 3) — pop from the front

These are handy for LRU-cache style structures. For everything else, reach for plain dict.

7. Which Container, When?

Need	Reach for
Count occurrences of hashable things	`Counter`
Group items by key into lists/sets	`defaultdict(list / set)`
FIFO queue or sliding window	`deque(maxlen=...)`
Lightweight immutable record	`typing.NamedTuple`
Record with methods, defaults, mutation	`@dataclass`
Layered config lookup	`ChainMap`
LRU-style "move recent to end"	`OrderedDict.move_to_end`
Anything else	plain `dict` / `list`

When in doubt, write it with plain types first. Swap in a specialized container only when the pattern is obvious or the performance matters.

Common Mistakes

dict.setdefault(k, []).append(v) everywhere — works, but defaultdict(list) is shorter and faster. Reserve setdefault for one-off cases.
Counter.most_common() without an n — returns all items sorted, which is wasteful if you only want the top 5. Pass n.
deque without maxlen when you wanted a sliding window — it'll grow unbounded. The maxlen is the whole point of the sliding-window pattern.
Treating namedtuple like a mutable class — it's immutable. Trying to assign p.x = 5 raises AttributeError. For state that changes, use a dataclass.
Reaching for OrderedDict when plain dict works — modern Python preserves insertion order natively. Only use OrderedDict if you need move_to_end or popitem(last=False).
Forgetting Counter keys can be negative after subtraction — Counter arithmetic with + drops zero/negative counts, but bare c["x"] -= 1 leaves negatives. Use +unary_counter to clean them up: +c drops zero/negative.

🎯 Your Turn — Word Counter and Rolling Average

Two small functions, both of which use containers you just met.

Part 1: top_words(text, n=10) returns the top-n most common words in text. Lowercase, only "word characters" (letters, digits, underscore — use re.findall(r"\w+", ...)), no stopword filtering needed.

Part 2: rolling_average(stream, window=5) is a generator that consumes an iterable of numbers and yields the running average over the last window items. For the first window-1 items, yield the partial average (average of what you have so far).

python

import re
from collections import Counter, deque

def top_words(text, n=10):
    """Return list of (word, count) for the n most common words."""
    # TODO 1: tokenize with re.findall(r"\w+", text.lower())
    # TODO 2: Counter the tokens
    # TODO 3: return .most_common(n)
    pass

def rolling_average(stream, window=5):
    """Yield the running mean over the last `window` items from `stream`."""
    # TODO 1: deque with maxlen=window
    # TODO 2: for each item, append, then yield sum(buf) / len(buf)
    pass

# Tests
text = "The rain in Spain falls mainly on the plain. The plain is plain."
print(top_words(text, 3))
# Expected: [('the', 3), ('plain', 3), ('in', 1)]   (order between ties may vary)

avgs = list(rolling_average([10, 20, 30, 40, 50], window=3))
print(avgs)
# Expected: [10.0, 15.0, 20.0, 30.0, 40.0]
#            └─ 10/1   (10+20)/2   (10+20+30)/3   (20+30+40)/3   (30+40+50)/3

Hint 1 — Tokenizing

re.findall(r"\w+", text.lower()) returns a list of all word-character runs, lowercased. Punctuation and whitespace are skipped automatically.

Hint 2 — Sliding mean

A deque(maxlen=window) automatically drops the oldest item when you append beyond maxlen. So sum(buf) / len(buf) always gives the average of the most recent ≤ window items, even before the window is full.

Show full solution

python

import re
from collections import Counter, deque

def top_words(text, n=10):
    tokens = re.findall(r"\w+", text.lower())
    return Counter(tokens).most_common(n)

def rolling_average(stream, window=5):
    buf = deque(maxlen=window)
    for item in stream:
        buf.append(item)
        yield sum(buf) / len(buf)

# Tests
text = "The rain in Spain falls mainly on the plain. The plain is plain."
print(top_words(text, 3))
# [('the', 3), ('plain', 3), ('in', 1)]

print(list(rolling_average([10, 20, 30, 40, 50], window=3)))
# [10.0, 15.0, 20.0, 30.0, 40.0]

Five lines of real logic across both functions. That's the point of collections — push the loop-counter bookkeeping into the container and let your code state the intent.

A more numerically stable version of rolling_average would maintain a running sum and add/subtract on each step (O(1) per yield instead of O(window)). For small windows the difference is negligible; for window sizes in the thousands, optimise.

What You Learned

Counter(iterable) — tally hashables; most_common(n); arithmetic (+ - & |) for combining tallies.
defaultdict(factory) — auto-create missing values; list/set/int are the common factories.
deque — O(1) at both ends; maxlen gives you a sliding window for free.
namedtuple / typing.NamedTuple — immutable records with field names; prefer the typed class form. Use @dataclass when you need behaviour.
ChainMap — layered lookup across multiple dicts; ideal for cascading configs.
OrderedDict — historical, but move_to_end and popitem(last=False) keep it relevant for LRU patterns.
The decision table above is worth memorising — picking the right container is half of writing clean Python.

Next: itertools — the iteration toolbelt that pairs perfectly with these containers (chain, groupby, islice, accumulate, combinations).

1 · The lesson

1. `Counter` — The Universal Tally

2. `defaultdict` — Auto-Creating Values

3. `deque` — Fast at Both Ends

4. `namedtuple` — Tuples With Field Names

5. `ChainMap` — Layered Lookups

6. `OrderedDict` — Mostly Obsolete

7. Which Container, When?

Common Mistakes

🎯 Your Turn — Word Counter and Rolling Average

What You Learned

Practice this

collections: The Stdlib's Hidden Power

1. Counter — The Universal Tally

2. defaultdict — Auto-Creating Values

3. deque — Fast at Both Ends

4. namedtuple — Tuples With Field Names

5. ChainMap — Layered Lookups

6. OrderedDict — Mostly Obsolete

7. Which Container, When?

Common Mistakes

🎯 Your Turn — Word Counter and Rolling Average

What You Learned

1. `Counter` — The Universal Tally

2. `defaultdict` — Auto-Creating Values

3. `deque` — Fast at Both Ends

4. `namedtuple` — Tuples With Field Names

5. `ChainMap` — Layered Lookups

6. `OrderedDict` — Mostly Obsolete