Project: URL Shortener

You'll build a local version of bit.ly. It turns long URLs into 6-character codes, looks them up and counts clicks. You need no web framework, no server and no account. You'll store the data first in a dict, then in JSON, then in SQLite. At the end you'll have a real CLI with the subcommands add, lookup, list and stats.

What you'll practise: random.choices, hashlib, JSON persistence, sqlite3, argparse subparsers, URL validation.

Step 1: The Simplest Version

This version has two functions and one dict.

python

import random
import string

MAPPINGS = {}      # maps each code to its long URL

ALPHABET = string.ascii_letters + string.digits      # 62 characters

def shorten(url, length=6):
    code = "".join(random.choices(ALPHABET, k=length))
    MAPPINGS[code] = url
    return code

def expand(code):
    return MAPPINGS.get(code)

# Demo
code1 = shorten("https://docs.python.org/3/library/sqlite3.html")
code2 = shorten("https://github.com/python/cpython")

print(f"  {code1} → {expand(code1)}")
print(f"  {code2} → {expand(code2)}")
print(f"  {'nope42':<8} → {expand('nope42')}")    # None

Six characters, each with 62 possible values, give 62⁶ ≈ 56.8 billion unique codes. That is more than enough for personal use.

Step 2: Persistence with JSON

A dict in memory is lost when the program exits. So load the data at the start, and save it after every change. (This is the same pattern you saw in csvjson.)

python

import json
import random
import string
from pathlib import Path

DB_FILE = Path("urls.json")
ALPHABET = string.ascii_letters + string.digits

def load():
    if not DB_FILE.exists():
        return {}
    return json.loads(DB_FILE.read_text(encoding="utf-8"))

def save(mappings):
    DB_FILE.write_text(json.dumps(mappings, indent=2), encoding="utf-8")

def shorten(url, mappings):
    code = "".join(random.choices(ALPHABET, k=6))
    mappings[code] = url
    save(mappings)
    return code

# Demo
mappings = load()
code = shorten("https://example.com/some/very/long/path", mappings)
print(f"created: {code}")
print(f"now stored: {len(mappings)} mappings")

The "load, change, save" pattern works well for hundreds of entries. As the file grows, it gets slower, because you rewrite the whole file on every change. A database such as SQLite solves this problem.

Step 3: Collision Handling

Sometimes two random codes are the same (a collision). It is rare, but it happens. The fix: if the code is already taken, try again. If it keeps happening, make the codes one character longer.

python

import random
import string

ALPHABET = string.ascii_letters + string.digits

def shorten(url, mappings, length=6, max_retries=5):
    for attempt in range(max_retries):
        code = "".join(random.choices(ALPHABET, k=length))
        if code not in mappings:
            mappings[code] = url
            return code
    # Every retry collided, so make the codes longer
    return shorten(url, mappings, length=length + 1, max_retries=max_retries)

# Stress-test the collision logic with a tiny alphabet and length
small_alpha = "ab"      # only 2 characters
def shorten_tiny(url, mappings, length=2, max_retries=3):
    for _ in range(max_retries):
        code = "".join(random.choices(small_alpha, k=length))
        if code not in mappings:
            mappings[code] = url
            return code
    return shorten_tiny(url, mappings, length=length + 1, max_retries=max_retries)

# Fill the 2-char namespace (only 4 codes possible) and watch the length grow
tiny = {}
for i in range(8):
    code = shorten_tiny(f"https://example.com/{i}", tiny)
    print(f"  {i}: {code:<5} (length {len(code)})")

The maths: with 62⁶ ≈ 5.7×10¹⁰ codes and a million existing entries, the chance of a collision for each new code is about 1.7×10⁻⁵. The chance that all five tries collide is extremely small.

Step 4: Hash-Based Codes (Deterministic)

With random codes, the same URL gets two different short codes if you submit it twice. Sometimes you want the same input to always give the same output. This is called idempotent behaviour, and hashing gives you that.

python

import hashlib

def shorten_hashed(url, length=6):
    digest = hashlib.sha256(url.encode("utf-8")).hexdigest()
    return digest[:length]

# Demo
for url in [
    "https://docs.python.org",
    "https://docs.python.org",       # the same input gives the same code
    "https://github.com/python/cpython",
]:
    print(f"  {shorten_hashed(url)}  ← {url}")

Pros of hashing:

Deterministic: if you submit the same URL again, you get the same code. There are no duplicate entries.

No need to track existing codes to detect collisions on identical input.

Cons:

Hex characters only (0-9, a-f), so the number of possible codes drops from 62ⁿ to 16ⁿ. To get at least as many codes as 6 random characters from 62, you need 9 hex characters (16⁹ ≈ 6.9×10¹⁰).

A small privacy problem: anyone who knows a URL can compute its code. For public short URLs, you usually want random IDs.

Some systems use both: a hash by default, and a random ID when privacy matters.

Step 5: Move to SQLite

A JSON file causes problems when several programs write to it at the same time, or when it gets very large. So let's move to a real database. (The basics are in databases.)

python

import sqlite3
from datetime import datetime, timezone

DB = sqlite3.connect("urls.db")
DB.execute("""
    CREATE TABLE IF NOT EXISTS urls (
        code        TEXT PRIMARY KEY,
        url         TEXT NOT NULL,
        created_at  TEXT NOT NULL,
        clicks      INTEGER NOT NULL DEFAULT 0
    )
""")
DB.commit()

def add(code, url):
    DB.execute(
        "INSERT INTO urls (code, url, created_at) VALUES (?, ?, ?)",
        (code, url, datetime.now(timezone.utc).isoformat(timespec="seconds")),
    )
    DB.commit()

def lookup(code):
    row = DB.execute(
        "SELECT url, clicks FROM urls WHERE code = ?", (code,)
    ).fetchone()
    if not row:
        return None
    # Increment the click counter
    DB.execute("UPDATE urls SET clicks = clicks + 1 WHERE code = ?", (code,))
    DB.commit()
    return row[0]

def stats(code):
    return DB.execute(
        "SELECT url, created_at, clicks FROM urls WHERE code = ?", (code,)
    ).fetchone()

# Demo
add("py3docs", "https://docs.python.org/3/")
print("  resolves to:", lookup("py3docs"))
print("  resolves to:", lookup("py3docs"))     # increments clicks again
print("  stats:", stats("py3docs"))

Three advantages over JSON:
1. Concurrent writes are safe, because SQLite locks the database during a write.
2. Indexed lookups on the primary key are O(log n), and SQLite reads only the rows it needs. With JSON, you must load and parse the whole file first, which is O(n).
3. Atomic updates: increasing clicks is a single SQL statement, so two programs can't overwrite each other's changes.

Always use parameterised queries (? placeholders), never string formatting. They also protect you against SQL injection.

Step 6: Polished Final Version

A real CLI with the subcommands add, lookup, list and stats. (It uses argparse subparsers. See cli.)

python

import argparse
import hashlib
import random
import sqlite3
import string
import sys
from datetime import datetime, timezone
from pathlib import Path
from urllib.parse import urlparse

DB_PATH = Path.home() / ".local" / "share" / "shortener" / "urls.db"
ALPHABET = string.ascii_letters + string.digits

def db():
    DB_PATH.parent.mkdir(parents=True, exist_ok=True)
    conn = sqlite3.connect(DB_PATH)
    conn.execute("""
        CREATE TABLE IF NOT EXISTS urls (
            code        TEXT PRIMARY KEY,
            url         TEXT NOT NULL,
            created_at  TEXT NOT NULL,
            clicks      INTEGER NOT NULL DEFAULT 0
        )
    """)
    return conn

def is_valid(url):
    p = urlparse(url)
    return p.scheme in ("http", "https") and "." in (p.netloc or "")

def generate_code(conn, length=6, retries=5):
    for _ in range(retries):
        code = "".join(random.choices(ALPHABET, k=length))
        if not conn.execute("SELECT 1 FROM urls WHERE code = ?", (code,)).fetchone():
            return code
    return generate_code(conn, length=length + 1, retries=retries)

def cmd_add(args):
    if not is_valid(args.url):
        print(f"error: invalid URL: {args.url!r}", file=sys.stderr)
        sys.exit(1)
    conn = db()
    code = args.alias or generate_code(conn)
    try:
        conn.execute(
            "INSERT INTO urls (code, url, created_at) VALUES (?, ?, ?)",
            (code, args.url, datetime.now(timezone.utc).isoformat(timespec="seconds")),
        )
        conn.commit()
    except sqlite3.IntegrityError:
        print(f"error: code {code!r} already taken", file=sys.stderr)
        sys.exit(1)
    print(f"  {code}  →  {args.url}")

def cmd_lookup(args):
    conn = db()
    row = conn.execute("SELECT url FROM urls WHERE code = ?", (args.code,)).fetchone()
    if not row:
        print(f"error: unknown code: {args.code!r}", file=sys.stderr)
        sys.exit(1)
    conn.execute("UPDATE urls SET clicks = clicks + 1 WHERE code = ?", (args.code,))
    conn.commit()
    print(row[0])

def cmd_list(args):
    conn = db()
    rows = conn.execute(
        "SELECT code, url, clicks FROM urls ORDER BY created_at DESC LIMIT ?",
        (args.limit,),
    ).fetchall()
    if not rows:
        print("(no URLs yet)")
        return
    print(f"  {'CODE':<10}{'CLICKS':>8}  URL")
    for code, url, clicks in rows:
        # Truncate long URLs for display
        display_url = url if len(url) <= 60 else url[:57] + "..."
        print(f"  {code:<10}{clicks:>8}  {display_url}")

def cmd_stats(args):
    conn = db()
    row = conn.execute(
        "SELECT url, created_at, clicks FROM urls WHERE code = ?", (args.code,)
    ).fetchone()
    if not row:
        print(f"error: unknown code: {args.code!r}", file=sys.stderr)
        sys.exit(1)
    url, created, clicks = row
    print(f"  code:    {args.code}")
    print(f"  url:     {url}")
    print(f"  created: {created}")
    print(f"  clicks:  {clicks}")

def main():
    p = argparse.ArgumentParser(description="Local URL shortener.")
    sub = p.add_subparsers(dest="cmd", required=True)

    p_add = sub.add_parser("add", help="shorten a URL")
    p_add.add_argument("url")
    p_add.add_argument("--alias", help="use a custom code instead of random")
    p_add.set_defaults(func=cmd_add)

    p_look = sub.add_parser("lookup", help="resolve a code")
    p_look.add_argument("code")
    p_look.set_defaults(func=cmd_lookup)

    p_list = sub.add_parser("list", help="show recent codes")
    p_list.add_argument("--limit", type=int, default=20)
    p_list.set_defaults(func=cmd_list)

    p_stat = sub.add_parser("stats", help="show stats for a code")
    p_stat.add_argument("code")
    p_stat.set_defaults(func=cmd_stats)

    args = p.parse_args()
    args.func(args)

if __name__ == "__main__":
    main()

Real-world usage:

bash

python shortener.py add "https://docs.python.org/3/library/sqlite3.html"
# → aB3xK9  →  https://docs.python.org/3/library/sqlite3.html

python shortener.py add "https://github.com/python" --alias gh-py
python shortener.py lookup gh-py
python shortener.py list --limit 5
python shortener.py stats gh-py

Stretch Goals

1. Expiry dates: add an expires_at column. Reject lookups after expiry. python shortener.py add URL --ttl 7d parses durations like 7d, 2h.
2. Custom alias support: the polished version already has it. Extend it to reject reserved words (add, lookup, stats) and rude words.
3. QR code generation: python shortener.py qr CODE prints a QR code in the terminal (pip install qrcode. It uses Unicode block characters to draw in the terminal).
4. Flask web wrapper: serve the lookups at http://localhost:5000/CODE so that the shortener redirects real browsers.
5. Click-through analytics: add a table clicks with code, clicked_at, user_agent and referer. Then you can plot when each link is used.

🎯 Your Turn: `validate_url(s)`

A simple urlparse check is fine for a hobby script, but it accepts many invalid URLs. Build a stricter validator that returns a (bool, error_message) tuple: True/None if the URL is valid, False/"reason" if not. You can use the same pattern for any function that checks input and must explain what is wrong.

python

from urllib.parse import urlparse

def validate_url(s):
    """Strict URL validation.

    Rules:
      - must be a non-empty string
      - scheme must be 'http' or 'https'
      - netloc (the 'example.com' part) must contain at least one dot
      - no spaces anywhere
      - reject 'http://.com', 'http://com.', or domains starting/ending with a dot

    Returns:
      (True, None) on success
      (False, "reason") on failure
    """
    # TODO 1: handle empty / non-string input
    # TODO 2: reject if any whitespace is present
    # TODO 3: parse with urlparse, then check scheme and netloc
    # TODO 4: check netloc has a dot and doesn't start or end with one
    pass


# Test
TESTS = [
    "https://example.com",                  # valid
    "http://docs.python.org/3/",            # valid
    "ftp://files.example.com",              # invalid: wrong scheme
    "https://no-dot-here",                  # invalid: no dot
    "https:// has spaces .com",             # invalid: whitespace
    "https://.com",                         # invalid: starts with dot
    "https://com.",                         # invalid: ends with dot
    "",                                     # invalid: empty
    None,                                   # invalid: not a string
]
for t in TESTS:
    ok, err = validate_url(t)
    mark = "✓" if ok else "✗"
    print(f"  {mark}  {str(t)[:40]:<40}  {err or ''}")

Hint 1: Reject bad input first

Reject early: if not isinstance(s, str) or not s.strip(): return (False, "empty url"). Then check whitespace before urlparse: if any(c.isspace() for c in s): return (False, "contains whitespace").

Hint 2: Use urlparse, then check the parts

p = urlparse(s) gives you p.scheme and p.netloc. Then a sequence of guards: if p.scheme not in ("http", "https"): ..., if "." not in p.netloc: ..., if p.netloc.startswith(".") or p.netloc.endswith("."): ....

Show full solution

python

from urllib.parse import urlparse

def validate_url(s):
    if not isinstance(s, str) or not s.strip():
        return False, "empty url"
    if any(c.isspace() for c in s):
        return False, "contains whitespace"
    p = urlparse(s)
    if p.scheme not in ("http", "https"):
        return False, f"bad scheme: {p.scheme or '(none)'}"
    if not p.netloc:
        return False, "missing domain"
    if "." not in p.netloc:
        return False, "domain has no dot"
    if p.netloc.startswith(".") or p.netloc.endswith("."):
        return False, "domain starts or ends with a dot"
    return True, None


TESTS = [
    "https://example.com",
    "http://docs.python.org/3/",
    "ftp://files.example.com",
    "https://no-dot-here",
    "https:// has spaces .com",
    "https://.com",
    "https://com.",
    "",
    None,
]
for t in TESTS:
    ok, err = validate_url(t)
    print(f"  {'✓' if ok else '✗'}  {str(t)[:40]:<40}  {err or ''}")

Returning a tuple works well for validation. The caller learns whether the check passed AND why it failed, and you don't need to raise exceptions for results you expect. Use it for form fields, config files, and anywhere a clear error message matters.

What You Learned

random.choices with a custom alphabet: a simple ID generator. It is not secure, so use secrets.choice for tokens that must stay secret.
hashlib.sha256 for deterministic IDs, and the trade-offs compared with random IDs.
JSON → SQLite: when to move from a file to a database.
sqlite3 with parameterised queries (no string formatting in SQL, ever).
argparse subparsers: the standard way to build a CLI tool with several commands.
urlparse for a quick URL check, and what it doesn't check.

You now know how to choose where to store data: JSON for small data, SQLite when you need real queries, and a server database such as PostgreSQL or MySQL only when SQLite is no longer enough (for example, when many users write at the same time).

Next: Markdown to HTML, where you build a parser, learn about state machines and make a real text-processing tool.

1 · The lesson

Step 1: The Simplest Version

Step 2: Persistence with JSON

Step 3: Collision Handling

Step 4: Hash-Based Codes (Deterministic)

Step 5: Move to SQLite

Step 6: Polished Final Version

Stretch Goals

🎯 Your Turn: `validate_url(s)`

What You Learned

Practice this

Project: URL Shortener

Step 1: The Simplest Version

Step 2: Persistence with JSON

Step 3: Collision Handling

Step 4: Hash-Based Codes (Deterministic)

Step 5: Move to SQLite

Step 6: Polished Final Version

Stretch Goals

🎯 Your Turn: validate_url(s)

What You Learned

🎯 Your Turn: `validate_url(s)`