When an interviewer asks you to design a URL‑based unique ID generator, they are looking for a clear thought process, not a perfect diagram. The problem is essentially a thin service that turns an arbitrary payload (often a long URL) into a short, globally unique token that can be resolved back to the original URL. Think of it as a modern "tiny URL" service, but the focus is on the ID generation pipeline rather than analytics or user accounts.
1. Clarify the scope and requirements
Start by asking clarifying questions. This shows you care about constraints and helps you avoid over‑engineering.
- Functional:
- Create: Given a long URL, return a short token (e.g.,
https://short.io/abc123). - Read: Given a token, retrieve the original URL.
- Idempotent: The same URL should map to the same token if requested repeatedly (optional).
- Expiration: Should tokens expire after a period? If yes, what happens on reuse?
- Create: Given a long URL, return a short token (e.g.,
- Non‑functional:
- Throughput: How many creates/reads per second does the system need to handle? Use a placeholder like
Rreads/sec andWwrites/sec. - Latency: Target sub‑100 ms response for both operations.
- Availability: Aim for "five nines" if the service is core; otherwise, a lower SLA may be acceptable.
- Durability: Data loss is unacceptable – persisted storage is required.
- Scalability: The design should grow horizontally as traffic increases.
- Security: Prevent abuse (e.g., rate‑limit per IP) and ensure tokens cannot be guessed.
- Throughput: How many creates/reads per second does the system need to handle? Use a placeholder like
By listing these, you set a decision framework for the rest of the discussion.
2. Core entities and data model
| Entity | Fields | Reason |
|---|---|---|
UrlMapping | token (string, primary key), original_url (string), created_at (timestamp), expires_at (timestamp, optional) | Represents the one‑to‑one mapping. |
Counter | shard_id (int), value (bigint) | Holds a monotonically increasing counter per shard for deterministic ID generation. |
The token is the short identifier. It can be a base‑62 encoded integer, a hash, or a random string depending on the chosen generation strategy.
3. API contract
POST /api/v1/shorten
Content-Type: application/json
{ "url": "https://example.com/very/long/path" }
Response (201):
{ "token": "abc123", "short_url": "https://short.io/abc123" }
GET /abc123
Response (302 redirect) or JSON payload with the original URL.
Both endpoints should be idempotent: repeated POST with the same URL may return the same token if you implement a deterministic mapping.
4. High‑level architecture
+-------------------+ +-------------------+ +-------------------+
| Load Balancer | <----> | API Servers | <----> | Cache Layer |
+-------------------+ +-------------------+ +-------------------+
| |
v v
+-------------------+ +-------------------+
| ID Generator | | Persistence DB |
+-------------------+ +-------------------+
- Load Balancer distributes traffic across stateless API servers.
- API Servers handle request validation, call the ID generator, and write/read from the DB.
- Cache Layer (e.g., Redis) stores recent token → URL mappings to meet latency goals.
- ID Generator can be a microservice that either:
- Allocates a block of integers from a distributed counter and encodes them, or
- Generates a cryptographic hash and truncates it.
- Persistence DB (e.g., a sharded relational store or a wide‑column DB) holds the authoritative mapping.
5. Deep dive: ID generation strategies
5.1 Monotonic counter + base‑62 encoding
- Sharding the counter: Split the global counter into
Nshards. Each shard owns a range of values, reducing contention. - Allocation: When a request arrives, the API server picks a shard (e.g., round‑robin) and fetches the next value atomically.
- Encoding: Convert the integer to a base‑62 string (
0‑9,a‑z,A‑Z). This yields a short token whose length grows logarithmically with the number of generated IDs.
Pros:
- Predictable length; easy to decode if needed.
- Simple to implement; no collisions.
Cons:
- Tokens are sequential, making enumeration attacks easier.
- Requires coordination to avoid duplicate allocation across shards.
5.2 Hash‑based (e.g., MD5/SHA‑256) with truncation
- Compute a hash of the original URL plus a random salt.
- Take the first
kcharacters (e.g., 6‑8) and encode in base‑62. - Check for collisions in the DB; on conflict, re‑hash with a different salt.
Pros:
- Tokens appear random, mitigating enumeration.
- No need for a global counter.
Cons:
- Collision handling adds latency; probability of collision grows with the number of IDs (birthday paradox).
- Token length is bounded, so you may need to increase
kover time.
5.3 Hybrid approach (recommended for interviews)
Use a monotonic counter for the first M characters (ensuring uniqueness) and append a random suffix for extra entropy. This balances predictability and security.
6. Storage layout and sharding
A common pattern is to shard the UrlMapping table by the prefix of the token (e.g., first two characters). This yields roughly 62^2 ≈ 3.8k shards, which distributes load evenly without needing a separate routing layer.
Table schema (relational example):
CREATE TABLE url_mapping (
token VARCHAR(12) PRIMARY KEY,
original_url TEXT NOT NULL,
created_at TIMESTAMP NOT NULL,
expires_at TIMESTAMP NULL
) PARTITION BY HASH (token);
Each partition lives on a different DB node; queries are directed by the API server using the token prefix.
7. Handling reads efficiently
- Cache first: On a GET request, look up the token in Redis. If a miss, fall back to the DB, then populate the cache.
- Cache eviction: Use a TTL matching the token's expiration or a LRU policy if you expect many short‑lived tokens.
- Read‑through pattern simplifies code and keeps cache warm for popular tokens.
8. Trade‑offs and discussion points
| Aspect | Counter‑based | Hash‑based |
|---|---|---|
| Collision | None (by definition) | Possible; requires retry logic |
| Predictability | High (sequential) | Low (random) |
| Coordination overhead | Needs atomic increment per shard | None (stateless) |
| Scalability | Good if shards are many | Excellent; fully parallel |
| Implementation complexity | Moderate | Low to moderate |
During the interview, you can argue that a hybrid approach gives you the best of both worlds: low collision risk, decent randomness, and simple scaling.
9. Follow‑up questions interviewers often ask
- How would you support custom aliases (e.g.,
short.io/mybrand)?- Store a separate namespace table mapping custom strings to URLs; enforce uniqueness via a unique index.
- What if the service must survive a data‑center outage?
- Replicate the DB across regions; use a consensus protocol (e.g., Raft) for the counter shards, or rely on a globally consistent store like Spanner.
- How do you prevent abuse (spam, DDoS) on the
POST /shortenendpoint?- Rate‑limit per IP, require CAPTCHA for high‑volume callers, and monitor for patterns of rapid token generation.
- Can you make the token generation deterministic so the same URL always yields the same token?
- Yes, by hashing the URL without a random salt and using a collision‑resolution table; trade‑off is loss of privacy for the URL.
- How would you migrate from a hash‑based scheme to a counter‑based scheme without breaking existing links?
- Keep both schemes active; store a migration flag in the DB and redirect old tokens to a lookup service that resolves them to the new format.
10. How to practice this
- Sketch the design on paper: Write the API, draw the component diagram, and list trade‑offs without looking at any reference.
- Explain each decision aloud: Use a tool like Call Assistant to rehearse your answer, ensuring you stay within a 45‑90 second window for each section.
- Iterate with variations: Change a requirement (e.g., add analytics) and redo the design quickly to build flexibility.
FAQ
- Q: Do I need a distributed database for this problem?
- A: Not necessarily for a prototype, but a production‑grade service benefits from sharding or a wide‑column store to handle high write throughput and to keep latency low.
- Q: How long should the token be?
- A: Aim for 6‑8 characters; this yields millions of unique IDs while keeping the URL short. Adjust length if you anticipate billions of entries.
- Q: Is it okay to store the original URL in plain text?
- A: Yes, unless you have privacy concerns. If needed, encrypt the column at rest and decrypt on read.
- Q: What monitoring metrics matter most?
- A: Track request latency, error rates, cache hit ratio, and counter shard utilization to spot bottlenecks early.
Frequently asked questions
Do I need a distributed database for this problem?
Not necessarily for a prototype, but a production‑grade service benefits from sharding or a wide‑column store to handle high write throughput and to keep latency low.
How long should the token be?
Aim for 6‑8 characters; this yields millions of unique IDs while keeping the URL short. Adjust length if you anticipate billions of entries.
Is it okay to store the original URL in plain text?
Yes, unless you have privacy concerns. If needed, encrypt the column at rest and decrypt on read.
What monitoring metrics matter most?
Track request latency, error rates, cache hit ratio, and counter shard utilization to spot bottlenecks early.
#system design#url shortener#unique id#scalability#interview prep#a URL-based unique ID generator