Tips & tricks

Python tips that change how you write code

Short, practical, and every one runnable: the before, the after, why it works, and when not to use it. Every example on these pages is run when the site is built, so the output you see is the output Python gives.

In the wild

real projects, the decision that mattered, and the tip you can take from it

DropboxHow Dropbox moved its desktop app to Python 3The tip: Migrate in stages, behind a switch you can turn back, and fix every regression before widening the rollout DropboxHow Dropbox type-checked 4 million lines of PythonThe tip: Add type hints gradually, starting where code is hardest to read CPythonPython 3.11: an average 25% faster without changing your codeThe tip: Upgrade Python before you rewrite code for speed, and stop avoiding try blocks for performance CPythonPre-fork Python servers: how Python stopped wasting their shared memoryThe tip: In a pre-fork server, build shared state once in the parent, then gc.freeze() it before forking SpotifyHow Spotify runs thousands of pipeline tasks a day with LuigiThe tip: Make each pipeline step skip work whose output already exists, and only publish finished results

Lessons from incidents

what went wrong, why, and the habit that would have caught it

CloudflareThe regular expression that took Cloudflare down for 27 minutesThe tip: Before a regex meets untrusted input, time it on a long string that nearly matches but doesn't GitLabGitLab deleted its production database, and the backups weren't thereThe tip: A backup you have never restored is a hope, not a backup: restore it automatically and check what comes back PyPIThe PyPI package whose GitHub repository was cleanThe tip: Pin exact versions with hashes, so a new release can't reach your machine until you choose to take it

Did you know?

Python behaviour that surprises people who have used it for years.

Comparisons chain, and that can surprise you0 <= x < 10 works like maths, because a < b < c means a < b and b < c. The same rule makes 1 < 2 == 2 True and x < y > z say less than it looks.0.1 + 0.2 is not 0.3, and round(2.5) is 2Floats are binary approximations, and round() rounds halves to the even number. Use math.isclose to compare and Decimal for money.Common mistakeis checks identity, == checks value== asks whether two values are equal; is asks whether they are the same object. Small-int caching hides the difference until it doesn't.Common mistakeFunctions made in a loop all see the last valueA lambda or inner function looks up loop variables when it's called, not when it's made, so every one sees the final value. Bind it with a default.Common mistakeA default list is shared by every callA default like tags=[] is built once, when the function is defined, and every call shares it. Use None and create the list inside the function.

Idioms

The shorter, clearer way experienced Python developers write it.

Replace flag loops with any() and all()any() and all() with a generator replace the found = False loop, stop at the first answer, and read like the question you're asking.Common mistakeWrite if ready:, not if ready == True:if ready == True: is longer and misses values that are true in every sense but not the value True. Test truthiness directly, and use 'is True' only on purpose.Common mistakeLoop with enumerate and zip, not range(len())enumerate gives the position and the item together, start=1 numbers from one, and zip(strict=True) stops unequal lists silently dropping data.Format numbers and dates inside the f-stringAfter the colon in an f-string: ,.2f for 1,234.57, .1% for percentages, <14 and >6 for columns, %d %b %Y for dates, and = for debugging.Sort by anything with key=, and by two things at oncesorted(key=...) sorts by any field, a tuple key sorts by several, negation reverses one of them, and Python's stable sort keeps ties in order.Unpack a row instead of indexing itReplace row[0], row[1], row[2:] with one line: date, city, *temps = row. Names instead of positions, and the rest collected into a list.

Industry practice

What working teams do, and why they bother.

Use a dataclass for records, not a dict@dataclass writes __init__, __repr__ and __eq__ for you. Typos become errors, fields have names, and frozen=True makes a record you can't change by accident.Common mistakeLog, don't print, in code that other code callsprint() can't be turned down, filtered or sent anywhere else. logging gives every message a level and a source, and one line changes what you see.Guard script code with if __name__ == "__main__"Code at the top level of a file runs on every import. Put the script part under if __name__ == "__main__" and the same file works as a script and a module.Build file paths with pathlib, not string joinspathlib.Path joins with /, splits a name into stem and suffix, and swaps extensions in one call. Fewer string bugs, and it reads like the path it builds.Common mistakeCatch the error you expect, not every errorA bare except hides typos and real bugs behind a friendly message. Catch the one error you expect, keep the try small, and re-raise with from.

Standard-library gems

Already installed, and replaces code you were about to write.

Count anything with collections.CounterCounter replaces the count-with-a-dict loop, answers most_common(n) directly, returns 0 for missing keys, and supports + and - between counts.Group rows with defaultdict(list)defaultdict(list) creates the empty list the first time a key is seen, so grouping rows by city, customer or date is two lines with no key check.Cache a slow pure function with one decorator@functools.cache remembers a function's results by its arguments, so repeated calls return instantly. Count the calls before and after, and know when not to.Get the top few with heapq.nlargest, not a full sortTo get the 5 biggest orders you don't need all million in order. heapq.nlargest keeps only the top N as it reads, with a key= just like sorted().Chunk with batched, compare neighbours with pairwiseitertools.batched splits a sequence into fixed-size chunks and pairwise walks neighbouring pairs, replacing fiddly index arithmetic with one clear call.

Performance

Measured, not guessed: run the timing yourself.

Look things up in a sorted list with bisectA chain of if/elif, or a scan through a sorted list, can be one call to bisect. It halves the search each step and keeps the list in order as you insert.Common mistakeTake from the front of a deque, not a listlist.pop(0) shifts every remaining item along by one. deque.popleft() doesn't. Measure it on a 50,000-item queue, in the browser.Use a generator when you only need one passsum([x for x in data]) builds a whole list first; sum(x for x in data) doesn't. Drop the brackets when you only read the values once.Common mistakeBuild strings with join, not += in a loopAdding to a string in a loop can copy it again and again. Collect the pieces and call join once. Measure the difference yourself, in the browser.Common mistakeCheck membership in a set, not a listx in some_list checks items one by one; x in some_set jumps straight to the answer. Measure the difference yourself, in the browser.

Debugging

Find the problem faster than print() and hope.

Add context to an error with add_note()An error deep in a loop says what went wrong but not where in your data. exc.add_note() (3.11+) attaches the row or file, and the traceback shows it.Label your debug prints with f"{x=}"f"{price=}" prints price=4.5: the name and the value, with no typing it twice. It works on whole expressions and with format specs like f"{price=:.2f}".Read nested data with pprintprint() puts a nested dict on one endless line. pprint.pp breaks it up by structure, and depth= and width= let you see the shape before the detail.Common mistakeKeep the whole traceback, not just print(e)print(e) keeps the message and throws away where it happened. traceback.print_exc() or logging.exception() keep the full trace while your program carries on.Debug with repr(), not print()print() shows '12' and 12 identically and hides trailing spaces and tabs. repr() shows the quotes and the whitespace, so the bug is visible.