PythonMastery

How do I read a file line by line in Python?

Loop over the open file: for line in f. It reads one line at a time, so a 10 GB log costs the same memory as a 10-line one. Strip the newline with rstrip().

Loop over the file object itself. Python hands you one line per pass and never loads the whole file.

python
# Make a file to read. Skip this part if you already have one.
with open("server.log", "w", encoding="utf-8") as f:
    f.write("09:00 INFO started\n09:02 ERROR disk full\n09:05 INFO retrying\n")

with open("server.log", encoding="utf-8") as f:
    for line in f:
        print(line.rstrip("\n"))
output
09:00 INFO started
09:02 ERROR disk full
09:05 INFO retrying

Each line still ends with its \n. Leave it on and print adds a second one, so you get a blank line between every row. rstrip("\n") takes it off. Plain strip() also removes spaces at both ends, which is usually what you want for data and not what you want for indented code.

With line numbers, and only the lines you care about

python
with open("server.log", "w", encoding="utf-8") as f:
    f.write("09:00 INFO started\n09:02 ERROR disk full\n09:05 INFO retrying\n")

with open("server.log", encoding="utf-8") as f:
    for number, line in enumerate(f, start=1):
        if "ERROR" in line:
            print(f"line {number}: {line.strip()}")
output
line 2: 09:02 ERROR disk full

When you want a list instead

f.read().splitlines() gives you every line in a list, with the newlines already gone:

python
with open("server.log", "w", encoding="utf-8") as f:
    f.write("09:00 INFO started\n09:02 ERROR disk full\n")

with open("server.log", encoding="utf-8") as f:
    lines = f.read().splitlines()

print(lines)
print(len(lines), "lines")
output
['09:00 INFO started', '09:02 ERROR disk full']
2 lines

It reads the whole file into memory, which is fine for a config file and a bad idea for a log that grows all day.

Skip readlines() in a loop

for line in f.readlines(): reads every line into a list first, then loops. It looks the same as for line in f: and uses as much memory as the file is big. The plain loop is shorter and doesn't.

Go deeper: File I/O, Pathlib, and Large Files, Files: Reading and Writing to Disk, FileNotFoundError: [Errno 2] No such file or directory

More short answers

all questions