Project: URL Shortener

You'll build a local bit.ly — turn long URLs into 6-char codes, look them up, count clicks. No web framework, no server, no signup. By the end you'll graduate from a dict to JSON to SQLite, and ship a real CLI with add, lookup, list, stats subcommands.

What you'll practice: random.choices, hashlib, JSON persistence, sqlite3, argparse subparsers, URL validation.

Step 1 — The Bare-Bones Version

Two functions, one dict. That's it.

python

import random
import string

MAPPINGS = {}      # code → long URL

ALPHABET = string.ascii_letters + string.digits      # 62 chars

def shorten(url, length=6):
    code = "".join(random.choices(ALPHABET, k=length))
    MAPPINGS[code] = url
    return code

def expand(code):
    return MAPPINGS.get(code)

# Demo
code1 = shorten("https://docs.python.org/3/library/sqlite3.html")
code2 = shorten("https://github.com/python/cpython")

print(f"  {code1} → {expand(code1)}")
print(f"  {code2} → {expand(code2)}")
print(f"  {'nope42':<8} → {expand('nope42')}")    # None

Six characters from 62 possibilities gives 62⁶ ≈ 56.8 billion unique codes — more than enough for personal use.

Step 2 — Persistence with JSON

A dict in memory dies when the process exits. Load on start, save on every change. (Same shape as you saw in csvjson.)

python

import json
import random
import string
from pathlib import Path

DB_FILE = Path("urls.json")
ALPHABET = string.ascii_letters + string.digits

def load():
    if not DB_FILE.exists():
        return {}
    return json.loads(DB_FILE.read_text(encoding="utf-8"))

def save(mappings):
    DB_FILE.write_text(json.dumps(mappings, indent=2), encoding="utf-8")

def shorten(url, mappings):
    code = "".join(random.choices(ALPHABET, k=6))
    mappings[code] = url
    save(mappings)
    return code

# Demo
mappings = load()
code = shorten("https://example.com/some/very/long/path", mappings)
print(f"created: {code}")
print(f"now stored: {len(mappings)} mappings")

The "load-mutate-save" pattern works fine for hundreds of entries. It collapses around tens of thousands because you rewrite the whole file every time — which is exactly the problem SQLite was built to solve.

Step 3 — Collision Handling

Random codes collide. Rarely, but they do. The fix: retry on collision, and if you keep colliding, grow the alphabet.

python

import random
import string

ALPHABET = string.ascii_letters + string.digits

def shorten(url, mappings, length=6, max_retries=5):
    for attempt in range(max_retries):
        code = "".join(random.choices(ALPHABET, k=length))
        if code not in mappings:
            mappings[code] = url
            return code
    # All retries collided — grow the keyspace
    return shorten(url, mappings, length=length + 1, max_retries=max_retries)

# Stress-test the collision logic with a tiny alphabet & length
small_alpha = "ab"      # only 2 chars
def shorten_tiny(url, mappings, length=2, max_retries=3):
    for _ in range(max_retries):
        code = "".join(random.choices(small_alpha, k=length))
        if code not in mappings:
            mappings[code] = url
            return code
    return shorten_tiny(url, mappings, length=length + 1, max_retries=max_retries)

# Fill the 2-char namespace (only 4 codes possible) and watch the length grow
tiny = {}
for i in range(8):
    code = shorten_tiny(f"https://example.com/{i}", tiny)
    print(f"  {i}: {code:<5} (length {len(code)})")

The math: with 62⁶ ≈ 5.7×10¹⁰ codes and a million existing entries, your collision probability per insert is roughly 1.7×10⁻⁵. Five retries makes it astronomically unlikely.

Step 4 — Hash-Based Codes (Deterministic)

Random codes mean the same URL gets two different short codes if you submit it twice. Sometimes you want the same input to always produce the same output — idempotent behaviour. Hashing gives you that.

python

import hashlib

def shorten_hashed(url, length=6):
    digest = hashlib.sha256(url.encode("utf-8")).hexdigest()
    return digest[:length]

# Demo
for url in [
    "https://docs.python.org",
    "https://docs.python.org",       # same input → same code
    "https://github.com/python/cpython",
]:
    print(f"  {shorten_hashed(url)}  ← {url}")

Pros of hashing:

Deterministic — re-submit the same URL, get the same code. No duplicate entries.

No need to track existing codes to detect collisions on identical input.

Cons:

Hex chars only (0-9, a-f) — keyspace shrinks from 62ⁿ to 16ⁿ. To match random.choices(62chars, k=6) you'd need 16⁹ ≈ 6-9 hex chars.

Subtle privacy leak: anyone with the URL can compute the code. For public short URLs you usually want random IDs.

Real systems often mix both — hash for the default, allow custom random IDs for privacy-sensitive cases.

Step 5 — Move to SQLite

JSON breaks down with concurrent access or large volumes. Time to graduate to a real database. (Background in databases.)

python

import sqlite3
from datetime import datetime

DB = sqlite3.connect("urls.db")
DB.execute("""
    CREATE TABLE IF NOT EXISTS urls (
        code        TEXT PRIMARY KEY,
        url         TEXT NOT NULL,
        created_at  TEXT NOT NULL,
        clicks      INTEGER NOT NULL DEFAULT 0
    )
""")
DB.commit()

def add(code, url):
    DB.execute(
        "INSERT INTO urls (code, url, created_at) VALUES (?, ?, ?)",
        (code, url, datetime.utcnow().isoformat(timespec="seconds")),
    )
    DB.commit()

def lookup(code):
    row = DB.execute(
        "SELECT url, clicks FROM urls WHERE code = ?", (code,)
    ).fetchone()
    if not row:
        return None
    # Increment the click counter
    DB.execute("UPDATE urls SET clicks = clicks + 1 WHERE code = ?", (code,))
    DB.commit()
    return row[0]

def stats(code):
    return DB.execute(
        "SELECT url, created_at, clicks FROM urls WHERE code = ?", (code,)
    ).fetchone()

# Demo
add("py3docs", "https://docs.python.org/3/")
print("  resolves to:", lookup("py3docs"))
print("  resolves to:", lookup("py3docs"))     # increments clicks again
print("  stats:", stats("py3docs"))

Three wins over JSON:
1. Concurrent writes safe via SQLite's locking.
2. Indexed lookups are O(log n) on the primary key — JSON is O(n) at best.
3. Atomic updates — incrementing clicks is a single SQL statement, no read-modify-write race.

Parameterised queries (? placeholders, never string formatting) — this is also your SQL-injection defence.

Step 6 — Polished Final Version

A real CLI with subcommands: add, lookup, list, stats. (Argparse subparsers — see cli.)

python

import argparse
import hashlib
import random
import sqlite3
import string
import sys
from datetime import datetime
from pathlib import Path
from urllib.parse import urlparse

DB_PATH = Path.home() / ".local" / "share" / "shortener" / "urls.db"
ALPHABET = string.ascii_letters + string.digits

def db():
    DB_PATH.parent.mkdir(parents=True, exist_ok=True)
    conn = sqlite3.connect(DB_PATH)
    conn.execute("""
        CREATE TABLE IF NOT EXISTS urls (
            code        TEXT PRIMARY KEY,
            url         TEXT NOT NULL,
            created_at  TEXT NOT NULL,
            clicks      INTEGER NOT NULL DEFAULT 0
        )
    """)
    return conn

def is_valid(url):
    p = urlparse(url)
    return p.scheme in ("http", "https") and "." in (p.netloc or "")

def generate_code(conn, length=6, retries=5):
    for _ in range(retries):
        code = "".join(random.choices(ALPHABET, k=length))
        if not conn.execute("SELECT 1 FROM urls WHERE code = ?", (code,)).fetchone():
            return code
    return generate_code(conn, length=length + 1, retries=retries)

def cmd_add(args):
    if not is_valid(args.url):
        print(f"error: invalid URL: {args.url!r}", file=sys.stderr)
        sys.exit(1)
    conn = db()
    code = args.alias or generate_code(conn)
    try:
        conn.execute(
            "INSERT INTO urls (code, url, created_at) VALUES (?, ?, ?)",
            (code, args.url, datetime.utcnow().isoformat(timespec="seconds")),
        )
        conn.commit()
    except sqlite3.IntegrityError:
        print(f"error: code {code!r} already taken", file=sys.stderr)
        sys.exit(1)
    print(f"  {code}  →  {args.url}")

def cmd_lookup(args):
    conn = db()
    row = conn.execute("SELECT url FROM urls WHERE code = ?", (args.code,)).fetchone()
    if not row:
        print(f"error: unknown code: {args.code!r}", file=sys.stderr)
        sys.exit(1)
    conn.execute("UPDATE urls SET clicks = clicks + 1 WHERE code = ?", (args.code,))
    conn.commit()
    print(row[0])

def cmd_list(args):
    conn = db()
    rows = conn.execute(
        "SELECT code, url, clicks FROM urls ORDER BY created_at DESC LIMIT ?",
        (args.limit,),
    ).fetchall()
    if not rows:
        print("(no URLs yet)")
        return
    print(f"  {'CODE':<10}{'CLICKS':>8}  URL")
    for code, url, clicks in rows:
        # Truncate long URLs for display
        display_url = url if len(url) <= 60 else url[:57] + "..."
        print(f"  {code:<10}{clicks:>8}  {display_url}")

def cmd_stats(args):
    conn = db()
    row = conn.execute(
        "SELECT url, created_at, clicks FROM urls WHERE code = ?", (args.code,)
    ).fetchone()
    if not row:
        print(f"error: unknown code: {args.code!r}", file=sys.stderr)
        sys.exit(1)
    url, created, clicks = row
    print(f"  code:    {args.code}")
    print(f"  url:     {url}")
    print(f"  created: {created}")
    print(f"  clicks:  {clicks}")

def main():
    p = argparse.ArgumentParser(description="Local URL shortener.")
    sub = p.add_subparsers(dest="cmd", required=True)

    p_add = sub.add_parser("add", help="shorten a URL")
    p_add.add_argument("url")
    p_add.add_argument("--alias", help="use a custom code instead of random")
    p_add.set_defaults(func=cmd_add)

    p_look = sub.add_parser("lookup", help="resolve a code")
    p_look.add_argument("code")
    p_look.set_defaults(func=cmd_lookup)

    p_list = sub.add_parser("list", help="show recent codes")
    p_list.add_argument("--limit", type=int, default=20)
    p_list.set_defaults(func=cmd_list)

    p_stat = sub.add_parser("stats", help="show stats for a code")
    p_stat.add_argument("code")
    p_stat.set_defaults(func=cmd_stats)

    args = p.parse_args()
    args.func(args)

if __name__ == "__main__":
    main()

Real-world usage:

bash

python shortener.py add "https://docs.python.org/3/library/sqlite3.html"
# → aB3xK9  →  https://docs.python.org/3/library/sqlite3.html

python shortener.py add "https://github.com/python" --alias gh-py
python shortener.py lookup gh-py
python shortener.py list --limit 5
python shortener.py stats gh-py

Stretch Goals

1. Expiry dates: add an expires_at column. Reject lookups after expiry. python shortener.py add URL --ttl 7d parses durations like 7d, 2h.
2. Custom alias support: already in the polished version — extend it to reject reserved words (add, lookup, stats) and disallow profanity.
3. QR code generation: python shortener.py qr CODE prints a QR code in the terminal (pip install qrcode — uses Unicode block chars for terminal output).
4. Flask web wrapper: serve the lookups at http://localhost:5000/CODE so the shortener actually redirects browsers. ~30 lines of Flask.
5. Click-through analytics: extra table clicks with code, clicked_at, user_agent, referer. Now you can plot when each link gets hit.

🎯 Your Turn — `validate_url(s)`

A urlparse check is fine for a hobby script but accepts a lot of junk. Build a stricter validator that returns a (bool, error_message) tuple — True/None if valid, False/"reason" if not. This is the pattern for any "validate at the boundary, return rich errors" function.

python

from urllib.parse import urlparse

def validate_url(s):
    """Strict URL validation.

    Rules:
      - must be a non-empty string
      - scheme must be 'http' or 'https'
      - netloc (the 'example.com' part) must contain at least one dot
      - no spaces anywhere
      - reject 'http://.com', 'http://com.', or domains starting/ending with a dot

    Returns:
      (True, None) on success
      (False, "reason") on failure
    """
    # TODO 1: handle empty / non-string input
    # TODO 2: reject if any whitespace is present
    # TODO 3: parse with urlparse — check scheme and netloc
    # TODO 4: check netloc has a dot and doesn't start/end with one
    pass


# Test
TESTS = [
    "https://example.com",                  # valid
    "http://docs.python.org/3/",            # valid
    "ftp://files.example.com",              # invalid: wrong scheme
    "https://no-dot-here",                  # invalid: no dot
    "https:// has spaces .com",             # invalid: whitespace
    "https://.com",                         # invalid: starts with dot
    "https://com.",                         # invalid: ends with dot
    "",                                     # invalid: empty
    None,                                   # invalid: not a string
]
for t in TESTS:
    ok, err = validate_url(t)
    mark = "✓" if ok else "✗"
    print(f"  {mark}  {str(t)[:40]:<40}  {err or ''}")

Hint 1 — Validate at the boundaries first

Reject early: if not isinstance(s, str) or not s.strip(): return (False, "empty url"). Then check whitespace before urlparse: if any(c.isspace() for c in s): return (False, "contains whitespace").

Hint 2 — Use urlparse, then inspect the parts

p = urlparse(s) gives you p.scheme and p.netloc. Then a sequence of guards: if p.scheme not in ("http", "https"): ..., if "." not in p.netloc: ..., if p.netloc.startswith(".") or p.netloc.endswith("."): ....

Show full solution

python

from urllib.parse import urlparse

def validate_url(s):
    if not isinstance(s, str) or not s.strip():
        return False, "empty url"
    if any(c.isspace() for c in s):
        return False, "contains whitespace"
    p = urlparse(s)
    if p.scheme not in ("http", "https"):
        return False, f"bad scheme: {p.scheme or '(none)'}"
    if not p.netloc:
        return False, "missing domain"
    if "." not in p.netloc:
        return False, "domain has no dot"
    if p.netloc.startswith(".") or p.netloc.endswith("."):
        return False, "domain starts or ends with a dot"
    return True, None


TESTS = [
    "https://example.com",
    "http://docs.python.org/3/",
    "ftp://files.example.com",
    "https://no-dot-here",
    "https:// has spaces .com",
    "https://.com",
    "https://com.",
    "",
    None,
]
for t in TESTS:
    ok, err = validate_url(t)
    print(f"  {'✓' if ok else '✗'}  {str(t)[:40]:<40}  {err or ''}")

The "return a tuple" pattern is gold for validation — callers know whether it succeeded AND why it failed, without raising exceptions for expected outcomes. Use it for form fields, config validation, and anywhere a user-friendly error message matters.

What You Learned

random.choices with a custom alphabet — the simplest cryptographic-grade-ish ID generator (use secrets.choice for real tokens).
hashlib.sha256 for deterministic IDs and the trade-off vs random.
JSON → SQLite evolution — the natural growth curve of any persistence layer.
sqlite3 with parameterised queries (no string formatting in SQL, ever).
argparse subparsers — the standard CLI shape for tools with multiple commands.
urlparse for cheap URL sanity-checking, plus what it doesn't cover.

You now have a real persistence layer skill: choose JSON for tiny data, SQLite when you need real queries, and PostgreSQL/MySQL only when you outgrow SQLite (which is later than you think — SQLite handles billions of rows fine).

Next: Markdown to HTML — build a parser, learn state machines, ship a real text-processing tool.

1 · The lesson

Step 1 — The Bare-Bones Version

Step 2 — Persistence with JSON

Step 3 — Collision Handling

Step 4 — Hash-Based Codes (Deterministic)

Step 5 — Move to SQLite

Step 6 — Polished Final Version

Stretch Goals

🎯 Your Turn — `validate_url(s)`

What You Learned

Practice this

Project: URL Shortener

Step 1 — The Bare-Bones Version

Step 2 — Persistence with JSON

Step 3 — Collision Handling

Step 4 — Hash-Based Codes (Deterministic)

Step 5 — Move to SQLite

Step 6 — Polished Final Version

Stretch Goals

🎯 Your Turn — validate_url(s)

What You Learned

🎯 Your Turn — `validate_url(s)`