One system. Every day.

Practical system
design, documented.

Clear breakdowns of real-world systems—the requirements, architecture, data model, bottlenecks, and trade-offs that matter.

Latest · #006

Data-intensive systems

Design a Distributed Web Crawler

Discover and fetch billions of pages while respecting websites, avoiding duplicate work, and keeping the crawl frontier healthy.

13 min readRead document

Archive

All documents

6 notes published

The premise

Learn one architecture
at a time.

New design documents are published daily. No filler—just constraints, decisions, diagrams, and honest trade-offs.