Take from the front of a deque, not a list

A list is fast at the end: append and pop() are cheap. Take from the front with pop(0) and every item behind it has to move up one place. For a queue, where you add at the back and take from the front, that turns into a lot of shuffling. collections.deque is built for both ends.

Measure it

import timeit
from collections import deque

def drain_list():
    queue = list(range(50_000))
    while queue:
        queue.pop(0)

def drain_deque():
    queue = deque(range(50_000))
    while queue:
        queue.popleft()

t_list = timeit.timeit(drain_list, number=1)
t_deque = timeit.timeit(drain_deque, number=1)
print(f"list.pop(0): {t_list * 1000:.0f} ms   deque.popleft(): {t_deque * 1000:.1f} ms")
print(f"the deque was about {t_list / t_deque:.0f}x faster")

list.pop(0): 136 ms   deque.popleft(): 2.1 ms
the deque was about 65x faster

About 60–70× in desktop Python for 50,000 items, and about 14× when I ran it here in the browser. The exact number depends on where it runs, but the gap does not go away. The list's time grows with the square of the queue length: twice the items means roughly four times the work. The deque's grows in a straight line.

The pattern in real code

A queue of jobs, processed in the order they arrived:

from collections import deque

jobs = deque(["resize photo.png", "email Ada", "backup notes"])
jobs.append("email Linus")          # new work goes on the back

while jobs:
    job = jobs.popleft()            # oldest first
    print("doing:", job)

doing: resize photo.png
doing: email Ada
doing: backup notes
doing: email Linus

A deque can also keep only the most recent items, which is handy for "the last N readings":

from collections import deque

last_three = deque(maxlen=3)
for temp in [21.4, 22.0, 23.1, 22.7, 21.9]:
    last_three.append(temp)
print(list(last_three))

[23.1, 22.7, 21.9]

Why it works

A list is one block of slots in order, so removing slot 0 means copying every slot after it. A deque is a chain of small blocks with a pointer at each end, so taking from the front just moves the pointer.

When not to use it

If you mostly index into the middle (items[2500]) or slice, stay with a list: a deque is slower there. And for a handful of items pop(0) is fine. The problem only shows once the queue is long and busy.