Take from the front of a deque, not a list

A list is fast at the end: append and pop() are cheap. Take from the front with pop(0) and every item behind it has to move up one place. For a queue, where you add at the back and take from the front, that turns into a lot of shuffling. collections.deque is built for both ends.

Measure it

import timeit
from collections import deque

def drain_list():
    queue = list(range(50_000))
    while queue:
        queue.pop(0)

def drain_deque():
    queue = deque(range(50_000))
    while queue:
        queue.popleft()

t_list = timeit.timeit(drain_list, number=1)
t_deque = timeit.timeit(drain_deque, number=1)
print(f"list.pop(0): {t_list * 1000:.0f} ms   deque.popleft(): {t_deque * 1000:.1f} ms")
print(f"the deque was about {t_list / t_deque:.0f}x faster")

list.pop(0): 136 ms   deque.popleft(): 2.1 ms
the deque was about 65x faster

About 60–70× in desktop Python for 50,000 items, and about 14× when I ran it here in the browser: the exact number depends on where it runs, the gap does not go away. The list's time grows with the square of the queue length: twice the items means roughly four times the work. The deque's grows in a straight line.

The pattern in real code

A queue of jobs, processed in the order they arrived:

from collections import deque

jobs = deque(["resize photo.png", "email Ada", "backup notes"])
jobs.append("email Linus")          # new work goes on the back

while jobs:
    job = jobs.popleft()            # oldest first
    print("doing:", job)

doing: resize photo.png
doing: email Ada
doing: backup notes
doing: email Linus

A deque can also keep only the most recent items, which is handy for "the last N readings":

from collections import deque

last_three = deque(maxlen=3)
for temp in [21.4, 22.0, 23.1, 22.7, 21.9]:
    last_three.append(temp)
print(list(last_three))

[23.1, 22.7, 21.9]

Why it works

A list is one block of slots in order, so removing slot 0 means copying every slot after it. A deque is a chain of small blocks with a pointer at each end; taking from the front just moves the pointer.

When not to use it

If you mostly index into the middle (items[2500]) or slice, stay with a list: a deque is slower there. And for a handful of items pop(0) is fine. The problem only shows once the queue is long and busy.