Working With Your Own Data Files

Every example dataset in the world is clean. Yours is not. It has a stray semicolon delimiter, a byte order mark Excel left behind, a column of numbers stored as text, and 40,000 rows when the tutorial assumed twelve.

This lesson is about that file — the one on your machine right now, whether it is a CSV, a JSON export, a spreadsheet or a PDF — and how to get it into the editor, read it correctly, and get your results back out. The formats themselves are covered in CSV, JSON, and JSONL. This is about the part before and after: the upload, the encoding, the size, and the download.

0. A File to Practice On

Every block below has a Run button, and every one of them reads a file. If you have your own CSV to hand, upload it and change the names as you go. If you do not, run this once — it writes a deliberately awkward file into the same place an upload lands, so the rest of the lesson has something real to chew on:

rows = [
    "Order ID;Region;Units;Unit Price",
    "ORD-0000001;north;12;39,95",
    "ORD-0000002;south;3;129,00",
    "ORD-0000003;north;7;18,50",
    "ORD-0000004;west;21;7,25",
]
with open("sales.csv", "w", encoding="utf-8-sig", newline="") as f:
    for line in rows:
        f.write(line + "\n")

# Two more, for the JSON examples later on.
import json

json.dump({"north": "Manchester", "south": "Bristol", "west": "Cardiff"},
          open("regions.json", "w", encoding="utf-8"))

with open("events.jsonl", "w", encoding="utf-8") as f:
    for order_id, status in [("ORD-0000001", "shipped"), ("ORD-0000002", "packing")]:
        f.write(json.dumps({"order": order_id, "status": status}) + "\n")

print("wrote sales.csv —", len(rows) - 1, "orders, plus regions.json and events.jsonl")

Semicolons, decimal commas and a byte order mark, on purpose. It is closer to a real export than anything with neat commas, and it will break the obvious code in Section 3 exactly the way a real file does.

1. Where Your File Actually Goes

Press Upload, pick a .csv, and it appears as a chip on the data row above the editor. Then this works:

rows = open("sales.csv").read()

No folder. No C:\Users\.... No /home/. Just the name.

That surprises people, so here is precisely what happened. Python here runs inside your browser, compiled to WebAssembly. It has a filesystem, but its own — a set of folders the browser keeps, with no connection to the drives on your machine. Your upload was copied into that filesystem's working directory, which is why the bare filename resolves.

Your uploads are kept. Close the tab, quit the browser, come back tomorrow: the data row still lists them and open("sales.csv") still works. They are stored on your own device, in your browser's storage — never sent anywhere — and they stay until you remove them with the × on a chip, the Clear data files button, or by clearing this site's data in your browser settings. The privacy policy lists every other way they can disappear, including a browser reclaiming the space on its own.

Two consequences follow, and both catch people:

Your real disk is invisible. open("C:/Users/ada/sales.csv") fails with FileNotFoundError no matter how certain you are that the file is there. The browser cannot reach your disk, which is the same protection that stops every other web page reading your documents. Uploading is how a file crosses that line, and it crosses in one direction only: into your browser, never to a server.

The names are exactly what you uploaded. A file called Q3 sales (final) .csv keeps its spaces and brackets, so quote it properly and check the chip if a read fails. Each chip has a copy code button that hands you the correct line with the correct name already in it.

import os
print(os.listdir("."))
# ['main.py', 'sales.csv', 'regions.json']

os.listdir(".") is the fastest way to settle an argument about whether a file is really loaded.

2. Look Before You Parse

The instinct is to reach for a parser immediately. Resist it for ten seconds. Most of the pain in this lesson comes from files that were almost what the reader assumed, and you can rule out half of it by printing three lines.

with open("sales.csv", encoding="utf-8", errors="replace") as f:
    for i, line in enumerate(f):
        print(repr(line))
        if i == 2:
            break

Use repr(), not print(line). repr shows you the things that matter and that plain printing hides:

'\ufeffOrder ID;Region;Units;Unit Price\n'
'ORD-0000001;north;12;39,95\n'

That one line tells you four things. There's a BOM at the front (\ufeff). The delimiter is a semicolon, not a comma. The decimal separator is a comma. And the headers have spaces in them. Parse this file with the defaults and you get one column of nonsense — with no error, which is worse.

errors="replace" is there on purpose for this first look: it guarantees the peek itself cannot crash, so you can see the file even when the encoding is wrong. Take it off once you know what you're dealing with.

3. Reading It, By Type

The copy code button on each chip gives you the right opening line for that file. Here is what it hands you and why.

The whole file, as text — the fastest way to see what you actually have:

print(open("sales.csv").read())

That is the honest starting point for a small file, and it needs nothing imported. For anything above a few hundred rows, print a slice instead of the lot — print(open("sales.csv").read()[:2000]) — because a console with 40,000 lines in it is no easier to read than the file was.

CSV, parsed — use DictReader so columns are named, not numbered:

import csv

with open("sales.csv", newline="", encoding="utf-8") as f:
    rows = list(csv.DictReader(f))

print(len(rows), "rows")
print(rows[0])

Run it on the practice file and the result is wrong:

{'Order ID;Region;Units;Unit Price': 'ORD-0000001;north;12;39', None: ['95']}

One key holding the entire header, one value holding the entire row. No exception, no warning — just four rows of nonsense that a later row["Units"] will turn into KeyError three functions away from the cause. That is the normal failure mode for someone else's CSV, and Sections 4 and 5 are the two reasons it happens.

newline="" is not optional either. Without it, a quoted field containing a line break gets split into two rows on some inputs, and the bug shows up as a handful of corrupted records in the middle of an otherwise fine file.

JSON — one function:

import json

config = json.load(open("regions.json", encoding="utf-8"))
print(type(config), len(config))

If you get JSONDecodeError: Expecting value: line 1 column 1, the file is almost never broken JSON. It is usually JSON Lines — one object per line, no wrapping array — which needs a loop instead:

import json

records = [json.loads(line) for line in open("events.jsonl") if line.strip()]
print(records)

Binary — images, spreadsheets, databases — "rb", always:

# Any real PNG starts with these eight bytes. Write one, then read it back.
open("chart.png", "wb").write(bytes([0x89, 0x50, 0x4E, 0x47, 0x0D, 0x0A, 0x1A, 0x0A]))

raw = open("chart.png", "rb").read()
print(len(raw), "bytes, starts with", raw[:4])
# 8 bytes, starts with b'\x89PNG'

Open a PNG without "rb" and you get UnicodeDecodeError on roughly the fourth byte. That error means "this is not text", not "this file is damaged".

The last two need a file only you have. Section 0 wrote a CSV and some JSON for you, but a spreadsheet and a PDF are not things worth faking — press Upload, add one of your own, and change the filename in the block. Run them without that and you get a note saying the file is not here, which is the truth rather than a stack trace.

Excel needs openpyxl, which the editor installs the first time you import it. Note that .xlsx is itself a zip of XML — it is treated as a data file here, not unpacked as an archive:

import pandas as pd
df = pd.read_excel("q3.xlsx", sheet_name=0)

# Or without pandas, if you only want a few cells
import openpyxl
ws = openpyxl.load_workbook("q3.xlsx")["Sales"]
print(ws["A1"].value, ws["B2"].value)

PDF needs pypdf, installed the same way:

from pypdf import PdfReader

reader = PdfReader("invoice.pdf")
print(len(reader.pages), "pages")
print(reader.pages[0].extract_text())

One honest caveat, and it is the one that catches people. extract_text() reads the text layer of a PDF. A document produced by software — an invoice generated by an accounting system, a report exported from Word — has one, and you get clean text. A scanned document does not: it is a photograph of a page, and extract_text() returns an empty string. Nothing is broken; there is simply no text in the file to find.

The fix for a scan is OCR, and that is the one thing genuinely off the table here — Tesseract has no WebAssembly build, so no browser can do it. If extract_text() comes back empty, check whether you are holding a picture:

from pypdf import PdfReader

page = PdfReader("invoice.pdf").pages[0]
text = page.extract_text().strip()
print("characters of text:", len(text))
print("images on the page :", len(page.images))
# text 0 and images 1 means a scan — OCR territory, not a parsing problem

That distinction is not a browser limitation. pypdf returns the same empty string on a scanned PDF on your laptop.

4. The Encoding Wall

This is the single most common failure with a real file, and the error message points at the wrong thing.

UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe9 in position 4721

The file is fine. Your assumption was wrong. Byte 0xe9 is é in Latin-1 and cp1252, and invalid on its own in UTF-8 — so this is a file written by an older Windows program, and it is very common in exports from accounting and retail systems.

Try these in order:

import csv

# 1. Excel-exported UTF-8, with a byte order mark  <- the right answer here
with open("sales.csv", newline="", encoding="utf-8-sig") as f:
    print("utf-8-sig ->", csv.DictReader(f, delimiter=";").fieldnames)

# 2. Plain utf-8 leaves the BOM stuck to the first column name
with open("sales.csv", newline="", encoding="utf-8") as f:
    print("utf-8     ->", csv.DictReader(f, delimiter=";").fieldnames)

# 3. Western European legacy encoding, for files older Windows tools wrote
with open("sales.csv", newline="", encoding="cp1252") as f:
    print("cp1252    ->", csv.DictReader(f, delimiter=";").fieldnames)

Run that. The first prints ['Order ID', 'Region', ...]. The second prints ['Order ID', ...] — a column name that looks identical on screen and fails every lookup you write. The third shows what a mis-guessed encoding does to the same bytes.

For a file that genuinely is not UTF-8, errors="replace" is the last resort:

import csv

rows = list(csv.DictReader(
    open("sales.csv", newline="", encoding="utf-8", errors="replace"), delimiter=";"))
print(len(rows), "rows — but check what was lost before trusting them")

Use utf-8-sig by default for anything that has ever been near Excel. It reads plain UTF-8 correctly and strips the BOM if one is there. The cost of getting this wrong is a first column named '\ufeffOrder ID' that fails every lookup you write, with KeyError: 'Order ID' and a printed header that looks identical to what you typed.

errors="replace" belongs in exploration, not in a pipeline. It turns unreadable bytes into � and marches on, which means a silently corrupted row instead of a loud failure. If you ship it, at least count what it destroyed.

5. Semicolons and Decimal Commas

Much of the world writes 1.234,56 where others write 1,234.56. Because the comma is taken, those locales export CSVs delimited with semicolons. Python's defaults assume the other convention, and the result is not an error — it is one column, containing your whole row as a string.

import csv

rows = list(csv.DictReader(open("sales.csv", newline="", encoding="utf-8-sig"), delimiter=";"))

for r in rows[:3]:
    units = int(r["Units"])
    price = float(r["Unit Price"].replace(",", "."))   # 39,95 -> 39.95
    print(r["Order ID"], units, "x", price, "=", round(units * price, 2))

Let csv.Sniffer guess when you are handling files from more than one source:

import csv

with open("sales.csv", newline="", encoding="utf-8-sig") as f:
    sample = f.read(4096)
    f.seek(0)
    dialect = csv.Sniffer().sniff(sample, delimiters=",;\t|")
    rows = list(csv.DictReader(f, dialect=dialect))

print("sniffed delimiter:", repr(dialect.delimiter), "->", len(rows), "rows")

Pass delimiters explicitly. Left to itself the sniffer will occasionally decide your file is delimited by the letter e, and it is not joking.

Seeing your rows, once they parse

With the encoding and the delimiter right, the rows are real dictionaries and printing them is ordinary Python. Three levels, in the order you will want them:

import csv

rows = list(csv.DictReader(open("sales.csv", newline="", encoding="utf-8-sig"), delimiter=";"))

# 1. One row per line — fine for a quick look
for row in rows:
    print(row)

# 2. Just the columns you care about
for row in rows:
    print(row["Order ID"], row["Region"], row["Units"])

# 3. Lined up, so you can actually read it
cols = ["Order ID", "Region", "Units", "Unit Price"]
print(" | ".join(c.ljust(12) for c in cols))
for row in rows:
    print(" | ".join(row[c].ljust(12) for c in cols))

str.ljust is the whole trick for level 3 — pad every field to the same width and the columns line up in any console. If you have pandas installed, print(df.head(20)) does the same job in one line and handles the widths for you; for a handful of rows, stdlib is faster than the install.

Print a slice, not the file. for row in rows[:20] while you are exploring. Forty thousand printed rows will freeze the output pane and tell you nothing that the first twenty did not.

6. How Big Is Too Big

The tab is not a server. There is a real ceiling, and it helps to know roughly where it sits.

Uploads here are capped at 25 MB per data file, and a zip is refused if its contents would expand past 32 MB. Those limits exist because the alternative is a frozen tab with no explanation.

Below that, speed is rarely the problem. A 7.5 MB, 200,000-row CSV streams through csv.DictReader in 0.69 s in this editor — reading and doing arithmetic on every row. That is not a number to design around.

Memory is the thing that bites, and the gap between the two ways of reading is large:

import csv

# Streaming — one row alive at a time, flat memory
total = 0.0
with open("sales.csv", newline="", encoding="utf-8-sig") as f:
    for row in csv.DictReader(f, delimiter=";"):
        total += int(row["Units"]) * float(row["Unit Price"].replace(",", "."))
print("revenue:", round(total, 2))

# Materialising — every row alive at once
rows = list(csv.DictReader(open("sales.csv", newline="", encoding="utf-8-sig"), delimiter=";"))
print("held in memory:", len(rows), "rows")

Measured in this editor on that same 7.5 MB file, with tracemalloc watching:

A 7.5 MB file becomes 46.9 MB as a list of dicts — six times the file — because every row is a dict and every field a separate string object, and the per-object overhead dwarfs the text. Streaming holds one row at a time, so it barely registers.

Three things follow. Stream when you only need a total, a count or a filter — it is free. Reach for pandas when you need the whole table, because it stores columns as arrays rather than millions of small objects: half the memory of the list of dicts, even though it took four times as long to load. And fix your dtypes — turning one repeated text column into a category cut 5.5 MB, a quarter of the frame, in one line.

If a file is too big to upload, cut it down before it gets here. Ten thousand representative rows will teach you the same things about your data as ten million, and they will do it in a second rather than not at all.

7. Getting Results Back Out

Reading is half the job. A file you write inside the tab lands in the same pretend filesystem — so it is real to Python, and invisible to your operating system until you bring it across.

import csv

summary = {}
with open("sales.csv", newline="", encoding="utf-8-sig") as f:
    for row in csv.DictReader(f, delimiter=";"):
        region = row["Region"]
        summary[region] = summary.get(region, 0) + int(row["Units"])

with open("summary.csv", "w", newline="", encoding="utf-8") as f:
    w = csv.writer(f)
    w.writerow(["region", "total_units"])
    for region, units in sorted(summary.items(), key=lambda kv: -kv[1]):
        w.writerow([region, units])

print(open("summary.csv").read())

To keep that file, open it in a tab and press Download — or Download⁺ to take every open tab at once as a zip.

Two things to know about writing here. Anything you write sits alongside your uploads, so a script that writes sales.csv overwrites the file you uploaded — name your outputs distinctly. And written files are not kept the way uploads are: refresh the page and summary.csv is gone. Download anything you want to keep, or regenerate it.

8. What Does Not Work Here, and Why

Honest limits, so you do not spend an evening on one:

• requests.get("https://...") to an arbitrary site. The browser blocks cross-origin requests the page has no permission for. This is not a Python limitation; it is the same rule that applies to every web page. Download the data yourself and upload the file.
• Absolute paths to your real disk. Covered in Section 1. Upload instead.
• os.system, subprocess. There is no operating system underneath to call.
• PyTorch and TensorFlow. No WebAssembly build exists. scikit-learn, XGBoost, LightGBM and statsmodels all run here, so most tabular machine learning is available — deep learning frameworks are not.
• Files above 25 MB. Sample them down first.

Everything else in this lesson — the reads, the encodings, the writes — is ordinary Python. Run it here to learn it, and the same code works unchanged on your own machine.

🎯 Your Turn — Survive a Hostile Export

Write load_export(path) that reads a CSV you know nothing about and returns a list of dicts, without crashing on the usual traps.

Requirements:

• Try utf-8-sig first, fall back to cp1252 — do not reach for errors="replace".
• Detect the delimiter from the first 4 KB with csv.Sniffer, restricted to ,;\t|.
• Strip whitespace from header names so " Unit Price " becomes "Unit Price".
• Return [] and print a readable explanation if the file cannot be parsed at all — no bare traceback.
• Print a one-line report: rows loaded, encoding used, delimiter found.

Test it against a file you deliberately break: export something from a spreadsheet as "CSV UTF-8", then re-save it as "CSV (semicolon delimited)", and make sure both load.

# Expected shape of the output
# loaded 1,482 rows | encoding=utf-8-sig | delimiter=';'

What You Learned

• Uploaded files live in the tab's own filesystem, so open("sales.csv") works with no path — and your real disk stays unreachable, which is the point.
• os.listdir(".") settles any question about whether a file is actually loaded.
• Excel and PDF need a package — openpyxl and pypdf, installed the first time you import them. .xlsx is a zip internally but is treated as data, not unpacked.
• A scanned PDF has no text layer, so extract_text() returns "". Check len(page.images) to tell a scan from a parsing problem. OCR cannot run in any browser — and pypdf behaves the same on your laptop.
• print(open("file.csv").read()) is the fastest look at a small file; slice it ([:2000]) for a big one.
• Peek with repr() before parsing. It reveals BOMs, delimiters and decimal commas that plain printing hides.
• Print rows with str.ljust to line the columns up, or df.head(20) if pandas is already installed. Always a slice while exploring.
• newline="" on every CSV open, or quoted fields containing line breaks silently split rows.
• utf-8-sig is the right default for anything that has touched Excel; cp1252 is the usual answer to can't decode byte 0xe9.
• errors="replace" is for exploring, not for pipelines — it converts a loud failure into a quiet corruption.
• Semicolon-delimited files with decimal commas are normal, not broken. delimiter=";" and .replace(",", ".").
• csv.Sniffer guesses the dialect — always pass delimiters explicitly.
• Stream for totals and filters; materialise only for random access. Measured here: a 7.5 MB, 200,000-row file streams in 0.69 s at ~0 MB, becomes 46.9 MB as a list of dicts, and 22.3 MB as a DataFrame — 16.8 MB once one column is a category. Memory, not speed, sets the limit.
• Write files, then Download — or Download⁺ for every tab as a zip. Written files do not survive a refresh; uploads do — they are kept in your browser's storage until you clear them.
• No arbitrary network fetches, no real paths, no subprocess — browser rules, not Python ones.

Next: CSV, JSON, and JSONL — the formats themselves, in depth, including streaming multi-gigabyte files once you are back on a real machine.