Working With Your Own Data Files

Every example dataset in the world is clean. Yours is not. It has a stray semicolon delimiter, a byte order mark Excel left behind, a column of numbers stored as text, and 40,000 rows when the tutorial assumed twelve.

This lesson is about that file, the one on your machine right now, whether it is a CSV, a JSON export, a spreadsheet or a PDF. It shows how to get the file into the editor, read it correctly, and get your results back out. The formats themselves are covered in CSV, JSON, and JSONL. This is about the part before and after: the upload, the encoding, the size, and the download.

0. A File to Practise On

Every block below has a Run button, and every one of them reads a file. If you have your own CSV to hand, upload it and change the names as you go. If you do not, run this once. It writes a deliberately awkward file to the same place where uploads go, so the rest of the lesson has a realistic file to work with:

rows = [
    "Order ID;Region;Units;Unit Price",
    "ORD-0000001;north;12;39,95",
    "ORD-0000002;south;3;129,00",
    "ORD-0000003;north;7;18,50",
    "ORD-0000004;west;21;7,25",
]
with open("sales.csv", "w", encoding="utf-8-sig", newline="") as f:
    for line in rows:
        f.write(line + "\n")

# Two more, for the JSON examples later on.
import json

json.dump({"north": "Manchester", "south": "Bristol", "west": "Cardiff"},
          open("regions.json", "w", encoding="utf-8"))

with open("events.jsonl", "w", encoding="utf-8") as f:
    for order_id, status in [("ORD-0000001", "shipped"), ("ORD-0000002", "packing")]:
        f.write(json.dumps({"order": order_id, "status": status}) + "\n")

print("wrote sales.csv with", len(rows) - 1, "orders, plus regions.json and events.jsonl")

Semicolons, decimal commas and a byte order mark, on purpose. It is closer to a real export than anything with neat commas, and it will break the obvious code in Section 3 exactly the way a real file does.

1. Where Your File Goes

Press Upload, pick a .csv, and it appears as a chip on the data row above the editor. Then this works:

rows = open("sales.csv").read()

No folder. No C:\Users\.... No /home/. Just the name.

That surprises people, so here is exactly what happened. Python here runs inside your browser, compiled to WebAssembly. It has a filesystem, but it is its own: a set of folders the browser keeps, with no connection to the drives on your machine. Your upload was copied into that filesystem's working directory, which is why the bare filename resolves.

Your uploads are kept. Close the tab, quit the browser, come back tomorrow: the data row still lists them and open("sales.csv") still works. They are stored on your own device, in your browser's storage, and are never sent anywhere. They stay until you remove them with the × on a chip, the Clear data files button, or by clearing this site's data in your browser settings. The privacy policy lists every other way they can disappear, including a browser reclaiming the space on its own.

Two consequences follow, and both catch people:

Your real disk is invisible. open("C:/Users/ada/sales.csv") fails with FileNotFoundError no matter how certain you are that the file is there. The browser cannot reach your disk, which is the same protection that stops every other web page reading your documents. Uploading is how a file crosses that line, and it crosses in one direction only: into your browser, never to a server.

The names are exactly what you uploaded. A file called Q3 sales (final) .csv keeps its spaces and brackets, so quote it properly and check the chip if a read fails. Each chip has a copy code button that gives you the correct line with the correct name already in it.

import os
print(os.listdir("."))
# ['main.py', 'sales.csv', 'regions.json', 'events.jsonl']

os.listdir(".") is the fastest way to check whether a file is really loaded.

2. Look Before You Parse

Most people start parsing straight away. Wait ten seconds first. Most of the problems in this lesson come from files that were almost what the reader assumed, and printing three lines rules out many of them.

with open("sales.csv", encoding="utf-8", errors="replace") as f:
    for i, line in enumerate(f):
        print(repr(line))
        if i == 2:
            break

Use repr(), not print(line). repr shows you the things that matter and that plain printing hides:

'\ufeffOrder ID;Region;Units;Unit Price\n'
'ORD-0000001;north;12;39,95\n'

That one line tells you four things. There's a BOM at the front (\ufeff). The delimiter is a semicolon, not a comma. The decimal separator is a comma. And the headers have spaces in them. If you parse this file with the defaults, you get one column of wrong data, and no error, which is worse.

errors="replace" is there on purpose for this first look: it makes sure the first look itself cannot crash, so you can see the file even when the encoding is wrong. Take it off once you know what you're dealing with.

3. Reading It, By Type

The copy code button on each chip gives you the right opening line for that file. Here is what it gives you and why.

The whole file, as text: the fastest way to see what you have:

print(open("sales.csv").read())

That is a good starting point for a small file, and it needs no imports. For anything above a few hundred rows, print a slice instead of everything (print(open("sales.csv").read()[:2000])), because a console with 40,000 lines in it is no easier to read than the file was.

CSV, parsed: use DictReader so columns are named, not numbered:

import csv

with open("sales.csv", newline="", encoding="utf-8") as f:
    rows = list(csv.DictReader(f))

print(len(rows), "rows")
print(rows[0])

Run it on the practice file and the result is wrong:

{'Order ID;Region;Units;Unit Price': 'ORD-0000001;north;12;39', None: ['95']}

One key holds the entire header, and one value holds most of the row. There is no exception and no warning, just four rows of wrong data. A later row["Units"] then raises KeyError, three functions away from the real cause. That is the normal failure mode for someone else's CSV, and Sections 4 and 5 are the two reasons it happens.

newline="" is not optional either. Without it, a quoted field containing a line break gets split into two rows on some inputs, and the bug shows up as a handful of corrupted records in the middle of an otherwise fine file.

JSON: one function:

import json

config = json.load(open("regions.json", encoding="utf-8"))
print(type(config), len(config))

If you get JSONDecodeError: Extra data: line 2 column 1, the file is usually not broken JSON. It is usually JSON Lines (one object per line, no surrounding array), which needs a loop instead:

import json

records = [json.loads(line) for line in open("events.jsonl") if line.strip()]
print(records)

Binary (images, spreadsheets, databases): always use "rb":

# Any real PNG starts with these eight bytes. Write one, then read it back.
open("chart.png", "wb").write(bytes([0x89, 0x50, 0x4E, 0x47, 0x0D, 0x0A, 0x1A, 0x0A]))

raw = open("chart.png", "rb").read()
print(len(raw), "bytes, starts with", raw[:4])
# 8 bytes, starts with b'\x89PNG'

Open a PNG without "rb" and you get UnicodeDecodeError on the very first byte (0x89). That error means "this is not text", not "this file is damaged".

The last two need a file only you have. Section 0 wrote a CSV and some JSON for you, but a fake spreadsheet or PDF would not teach you much. Press Upload, add one of your own, and change the filename in the block. If you run them without a file, you get a short note saying the file is not here, instead of a stack trace.

Excel needs openpyxl, which the editor installs the first time you import it. Note that .xlsx is itself a zip of XML files, but it is treated as a data file here, not unpacked as an archive:

import pandas as pd
df = pd.read_excel("q3.xlsx", sheet_name=0)

# Or without pandas, if you only want a few cells
import openpyxl
ws = openpyxl.load_workbook("q3.xlsx")["Sales"]
print(ws["A1"].value, ws["B2"].value)

PDF needs pypdf, installed the same way:

from pypdf import PdfReader

reader = PdfReader("invoice.pdf")
print(len(reader.pages), "pages")
print(reader.pages[0].extract_text())

There is one limitation, and it catches many people. extract_text() reads the text layer of a PDF. A document produced by software (an invoice generated by an accounting system, a report exported from Word) has one, and you get clean text. A scanned document does not: it is a photograph of a page, and extract_text() returns an empty string. Nothing is broken. There is no text in the file to find.

The fix for a scan is OCR, and that is the one thing that does not work here. Python's usual OCR package (pytesseract) needs the Tesseract program installed on your computer, and Python in this page can't call it. If extract_text() comes back empty, check whether you are holding a picture:

from pypdf import PdfReader

page = PdfReader("invoice.pdf").pages[0]
text = page.extract_text().strip()
print("characters of text:", len(text))
print("images on the page :", len(page.images))
# text 0 and images 1 means a scanned page. It needs OCR, not a parser.

That distinction is not a browser limitation. pypdf returns the same empty string on a scanned PDF on your laptop.

4. The Encoding Wall

This is the most common failure with a real file, and the error message points at the wrong thing.

UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe9 in position 4721

The file is fine. Your assumption was wrong. Byte 0xe9 is é in Latin-1 and cp1252, and it is invalid on its own in UTF-8. So this file was probably written by an older Windows program. Files like this are very common in exports from accounting and retail systems.

Try these in order:

import csv

# 1. Excel-exported UTF-8, with a byte order mark  (the right answer here)
with open("sales.csv", newline="", encoding="utf-8-sig") as f:
    print("utf-8-sig ->", csv.DictReader(f, delimiter=";").fieldnames)

# 2. Plain utf-8 leaves the BOM stuck to the first column name
with open("sales.csv", newline="", encoding="utf-8") as f:
    print("utf-8     ->", csv.DictReader(f, delimiter=";").fieldnames)

# 3. Western European legacy encoding, for files older Windows tools wrote
with open("sales.csv", newline="", encoding="cp1252") as f:
    print("cp1252    ->", csv.DictReader(f, delimiter=";").fieldnames)

Run that. The first prints ['Order ID', 'Region', ...]. The second prints ['Order ID', ...]: a column name that looks the same on screen but fails every lookup you write. The third shows what a mis-guessed encoding does to the same bytes.

For a file that really is not UTF-8, errors="replace" is the last resort:

import csv

rows = list(csv.DictReader(
    open("sales.csv", newline="", encoding="utf-8", errors="replace"), delimiter=";"))
print(len(rows), "rows, but check what was lost before you trust them")

Use utf-8-sig by default for anything that has ever been near Excel. It reads plain UTF-8 correctly and strips the BOM if one is there. The cost of getting this wrong is a first column named '\ufeffOrder ID' that fails every lookup you write, with KeyError: 'Order ID' and a printed header that looks identical to what you typed.

errors="replace" belongs in exploration, not in a pipeline. It turns unreadable bytes into � and continues, so you get a corrupted row with no warning instead of a clear error. If you do use it in real code, at least count how many characters it replaced.

5. Semicolons and Decimal Commas

Much of the world writes 1.234,56 where others write 1,234.56. Because the comma is taken, those locales export CSVs delimited with semicolons. Python's defaults assume the other convention, and the result is not an error. It is one column that contains your whole row as a string.

import csv

rows = list(csv.DictReader(open("sales.csv", newline="", encoding="utf-8-sig"), delimiter=";"))

for r in rows[:3]:
    units = int(r["Units"])
    price = float(r["Unit Price"].replace(",", "."))   # 39,95 becomes 39.95
    print(r["Order ID"], units, "x", price, "=", round(units * price, 2))

Let csv.Sniffer guess when you are handling files from more than one source:

import csv

with open("sales.csv", newline="", encoding="utf-8-sig") as f:
    sample = f.read(4096)
    f.seek(0)
    dialect = csv.Sniffer().sniff(sample, delimiters=",;\t|")
    rows = list(csv.DictReader(f, dialect=dialect))

print("sniffed delimiter:", repr(dialect.delimiter), "->", len(rows), "rows")

Pass delimiters explicitly. Without it, the sniffer sometimes decides that a letter, such as e, is the delimiter.

Seeing your rows, once they parse

With the encoding and the delimiter right, the rows are real dictionaries and printing them is ordinary Python. Three levels, in the order you will want them:

import csv

rows = list(csv.DictReader(open("sales.csv", newline="", encoding="utf-8-sig"), delimiter=";"))

# 1. One row per line, fine for a quick look
for row in rows:
    print(row)

# 2. Just the columns you care about
for row in rows:
    print(row["Order ID"], row["Region"], row["Units"])

# 3. Lined up, so you can actually read it
cols = ["Order ID", "Region", "Units", "Unit Price"]
print(" | ".join(c.ljust(12) for c in cols))
for row in rows:
    print(" | ".join(row[c].ljust(12) for c in cols))

Level 3 only needs str.ljust: pad every field to the same width, and the columns line up in any console. If you have pandas installed, print(df.head(20)) does the same job in one line and handles the widths for you. For a handful of rows, stdlib is faster than the install.

Print a slice, not the file. for row in rows[:20] while you are exploring. Forty thousand printed rows will freeze the output pane and tell you nothing that the first twenty did not.

6. How Big Is Too Big

The tab is not a server. There is a real ceiling, and it helps to know roughly where it sits.

Uploads here are capped at 25 MB per data file, and a zip is refused if its contents would expand past 32 MB. Those limits exist because the alternative is a frozen tab with no explanation.

Below that, speed is rarely the problem. A 7.5 MB, 200,000-row CSV streams through csv.DictReader in 0.69 s in this editor, reading every row and doing arithmetic on it. That is fast enough that you do not need to plan around it.

Memory is the real limit, and the two ways of reading use very different amounts:

import csv

# Streaming: one row in memory at a time, so memory use stays flat
total = 0.0
with open("sales.csv", newline="", encoding="utf-8-sig") as f:
    for row in csv.DictReader(f, delimiter=";"):
        total += int(row["Units"]) * float(row["Unit Price"].replace(",", "."))
print("revenue:", round(total, 2))

# Building a list: every row is in memory at once
rows = list(csv.DictReader(open("sales.csv", newline="", encoding="utf-8-sig"), delimiter=";"))
print("held in memory:", len(rows), "rows")

Measured in this editor on that same 7.5 MB file, with tracemalloc watching:

A 7.5 MB file becomes 46.9 MB as a list of dicts, about six times the file size. Every row is a dict and every field is a separate string object, and each object needs much more memory than the text inside it. Streaming holds one row at a time, so it barely registers.

Three things follow. Stream when you only need a total, a count or a filter, because it uses almost no memory. Use pandas when you need the whole table, because it stores columns as arrays rather than millions of small objects: half the memory of the list of dicts, even though it took four times as long to load. And fix your dtypes: turning one repeated text column into a category cut 5.5 MB, a quarter of the frame, in one line.

If a file is too big to upload, cut it down before it gets here. Ten thousand representative rows will teach you the same things about your data as ten million, and they will do it in a second rather than not at all.

7. Getting Results Back Out

Reading is half the job. A file you write inside the tab goes into the same browser filesystem. Python can see it, but your operating system can't until you download it.

import csv

summary = {}
with open("sales.csv", newline="", encoding="utf-8-sig") as f:
    for row in csv.DictReader(f, delimiter=";"):
        region = row["Region"]
        summary[region] = summary.get(region, 0) + int(row["Units"])

with open("summary.csv", "w", newline="", encoding="utf-8") as f:
    w = csv.writer(f)
    w.writerow(["region", "total_units"])
    for region, units in sorted(summary.items(), key=lambda kv: -kv[1]):
        w.writerow([region, units])

print(open("summary.csv").read())

To keep that file, open it in a tab and press Download, or press Download⁺ to get every open tab at once as a zip.

Two things to know about writing here. Anything you write sits alongside your uploads, so a script that writes sales.csv overwrites the file you uploaded. Give your outputs different names. And written files are not kept the way uploads are: refresh the page and summary.csv is gone. Download anything you want to keep, or regenerate it.

8. What Does Not Work Here, and Why

These are the limits, so you do not spend an evening on one:

• requests.get("https://...") to an arbitrary site. The browser blocks cross-origin requests the page has no permission for. This is not a Python limitation. It is the same rule that applies to every web page. Download the data yourself and upload the file.
• Absolute paths to your real disk. Covered in Section 1. Upload instead.
• os.system, subprocess. There is no operating system underneath to call.
• PyTorch and TensorFlow. No WebAssembly build exists. scikit-learn, XGBoost, LightGBM and statsmodels all run here, so most machine learning on tables is available. Deep learning frameworks are not.
• Files above 25 MB. Sample them down first.

Everything else in this lesson (the reads, the encodings, the writes) is ordinary Python. Run it here to learn it, and the same code works unchanged on your own machine.

🎯 Your Turn: Handle a Difficult Export

Write load_export(path) that reads a CSV you know nothing about and returns a list of dicts, without crashing on the usual traps.

Requirements:

• Try utf-8-sig first, then fall back to cp1252. Do not use errors="replace".
• Detect the delimiter from the first 4 KB with csv.Sniffer, restricted to ,;\t|.
• Strip whitespace from header names so " Unit Price " becomes "Unit Price".
• Return [] and print a readable explanation if the file cannot be parsed at all, instead of a bare traceback.
• Print a one-line report: rows loaded, encoding used, delimiter found.

Test it against a file you deliberately break: export something from a spreadsheet as "CSV UTF-8", then re-save it as "CSV (semicolon delimited)", and make sure both load.

# Expected shape of the output
# loaded 1,482 rows | encoding=utf-8-sig | delimiter=';'

What You Learned

• Uploaded files live in the tab's own filesystem, so open("sales.csv") works with no path. Your real disk stays out of reach, and that is on purpose.
• os.listdir(".") shows whether a file is really loaded.
• Excel and PDF need a package: openpyxl and pypdf, installed the first time you import them. .xlsx is a zip internally but is treated as data, not unpacked.
• A scanned PDF has no text layer, so extract_text() returns "". Check len(page.images) to tell a scan from a parsing problem. OCR isn't available to Python in this page, and pypdf behaves the same way on your laptop.
• print(open("file.csv").read()) is the fastest look at a small file. Slice it ([:2000]) for a big one.
• Peek with repr() before parsing. It reveals BOMs, delimiters and decimal commas that plain printing hides.
• Print rows with str.ljust to line the columns up, or df.head(20) if pandas is already installed. Always a slice while exploring.
• newline="" on every CSV open, or quoted fields containing line breaks silently split rows.
• utf-8-sig is the right default for anything that has touched Excel, and cp1252 is the usual answer to can't decode byte 0xe9.
• errors="replace" is for exploring, not for pipelines. It turns a clear error into data that is corrupted without any warning.
• Semicolon-delimited files with decimal commas are normal, not broken. delimiter=";" and .replace(",", ".").
• csv.Sniffer guesses the dialect. Always pass delimiters explicitly.
• Stream for totals and filters, and materialise only for random access. Measured here: a 7.5 MB, 200,000-row file streams in 0.69 s at ~0 MB, becomes 46.9 MB as a list of dicts, and 22.3 MB as a DataFrame (16.8 MB once one column is a category). Memory, not speed, sets the limit.
• Write files, then Download, or use Download⁺ for every tab as a zip. Written files do not survive a refresh. Uploads do: they are kept in your browser's storage until you clear them.
• No arbitrary network fetches, no real paths, no subprocess. These are browser rules, not Python ones.

Next: CSV, JSON, and JSONL. It covers the formats themselves in depth, including streaming multi-gigabyte files once you are back on a real machine.