When the interview asks you to design an ad click aggregator, the goal is to show that you can translate a vague product idea into a concrete system that meets reliability, latency, and scalability constraints. The conversation should flow from high‑level requirements to data model, API, architecture, and finally to the hard parts that interviewers love to probe.

1. Clarify the scope and requirements

Functional requirements

  • Record every click event with timestamp, user identifier, ad identifier, campaign identifier, and optional metadata (device, geo, etc.).
  • Provide real‑time counters per ad, per campaign, and per advertiser.
  • Support historical queries (e.g., clicks per hour for the last 30 days).
  • Expose an API for insertion and for read‑only aggregation queries.

Non‑functional requirements

  • Throughput: Must ingest a large volume of clicks (think millions per second in peak hours). Use variables like clickRate to keep the discussion abstract.
  • Latency: Real‑time dashboards should reflect a click within a few seconds.
  • Durability: No click should be lost, even if a node crashes.
  • Scalability: System should grow horizontally with traffic.
  • Consistency: Slightly stale counts are acceptable for dashboards, but exact counts are needed for billing.

Ask the interviewer clarifying questions: expected peak clickRate, SLA for latency, tolerance for eventual consistency, and whether the system needs to support deduplication of duplicate clicks.

2. Core entities and data model

EntityKey fieldsReason for inclusion
ClickclickId, timestamp, adId, campaignId, userId, metadataImmutable event, primary write unit
AdadId, advertiserId, campaignIdEnables grouping by advertiser or campaign
CampaigncampaignId, advertiserId, budgetUseful for quota enforcement
AggregatedCounterentityId (ad/campaign/advertiser), windowStart, windowEnd, clickCountStores pre‑computed counts for fast reads

The click record is append‑only; counters are materialized views built from the stream of clicks.

3. API design

POST /clicks
Content-Type: application/json
{
  "clickId": "uuid",
  "timestamp": 1698451200,
  "adId": "ad-123",
  "campaignId": "camp-45",
  "userId": "user-9",
  "metadata": {"device":"mobile","geo":"US"}
}
GET /metrics?entityId=ad-123&granularity=hour&start=1698447600&end=1698451200
Response:
{
  "entityId": "ad-123",
  "granularity": "hour",
  "buckets": [{"windowStart":1698447600,"clickCount":124}, …]
}

The write endpoint must be idempotent; the read endpoint returns pre‑aggregated counters.

4. High‑level architecture

+----------------+      +----------------+      +-------------------+
|  Client Apps   | -->  | Load Balancer  | -->  | Ingestion Service |
+----------------+      +----------------+      +-------------------+
                                                         |
                                                         v
                                         +----------------------------+
                                         | Write‑Ahead Log (Kafka)    |
                                         +----------------------------+
                                            |            |
               +----------------------------+            +----------------------------+
               | Stream Processor (Flink)  |            | Batch Processor (Spark)   |
               +----------------------------+            +----------------------------+
               |   Real‑time counters      |            |   Daily aggregates         |
               +----------------------------+            +----------------------------+
                                            |            |
                                            v            v
                                   +----------------+  +-------------------+
                                   | KV Store (Redis|  | OLAP DB (Snowflake) |
                                   |  for hot data) |  +-------------------+
                                   +----------------+

Key components

  • Load balancer distributes write traffic across ingestion nodes.
  • Ingestion service validates payload and writes to a durable log (Kafka or equivalent). The log provides replay capability for downstream processors.
  • Stream processor consumes the log, updates in‑memory counters (Redis or a custom in‑process map), and writes the delta to a durable KV store for fast reads.
  • Batch processor runs nightly to recompute aggregates for longer windows and stores them in an OLAP database for reporting.
  • Read API pulls from the KV store for recent windows and falls back to the OLAP store for older data.

5. Deep dive: Handling high write throughput

5.1 Partitioning strategy

Use a composite key of adId (or campaignId) and a time bucket (e.g., minute) to partition the log. This spreads load across partitions and enables parallel consumers.

5.2 Write‑ahead log (WAL)

A WAL guarantees durability: each click is appended before any processing. If a consumer crashes, it can resume from the last committed offset. The log also decouples producers from consumers, allowing you to scale each independently.

5.3 Idempotency & deduplication

Clicks may be retried; include a unique clickId and let the ingestion service check a Bloom filter or a small deduplication cache before writing to the log.

6. Deep dive: Real‑time vs. batch aggregation

  • Real‑time: Stream processor maintains sliding windows (e.g., per minute) and updates a hot cache. Latency is a few seconds, suitable for dashboards.
  • Batch: Nightly jobs recompute exact counts for longer windows (hourly, daily). This ensures billing accuracy and provides data for ML models.

The trade‑off is consistency: the hot cache may be slightly out‑of‑date, but that is acceptable for monitoring. Billing queries should always hit the batch layer.

7. Trade‑offs and alternatives

DecisionOption AOption BWhen to choose
Storage for countersIn‑memory KV (Redis)Distributed column store (Cassandra)KV is simpler and cheaper for hot data; column store scales better for very large cardinalities
Stream processingFlinkKafka StreamsFlink offers richer windowing; Kafka Streams is lighter if you already run Kafka
Consistency modelEventual (hot cache) + Strong (batch)Strong everywhere (two‑phase commit)Eventual is fine for dashboards; strong everywhere adds latency and complexity

Interviewers often ask: "What if the click volume spikes 10× overnight?" Discuss autoscaling ingestion nodes, increasing partition count, and back‑pressure handling in the stream processor.

8. Typical follow‑up questions

  1. How would you support deduplication across multiple data centers?
    • Use a globally unique clickId and a distributed cache (e.g., DynamoDB with TTL) that all ingestion nodes consult.
  2. What if advertisers need per‑country breakdowns?
    • Add geo to the partition key and extend the stream processor to emit separate counters per country.
  3. How do you ensure low latency for read queries?
    • Keep the most recent windows in a hot KV store; serve older windows from the OLAP layer.
  4. Can you expose a pub‑sub API for external dashboards?
    • Yes; stream the aggregated counters to a topic that downstream services can subscribe to.
  5. Where does Call Assistant fit in your interview prep?
    • It can help you rehearse the answer aloud, keep follow‑ups on track, and ground your story in concrete resume examples.

How to practice this

  1. Sketch the diagram on paper – start with the load balancer and add components one by one, explaining each trade‑off.
  2. Write a mock API – implement the POST /clicks endpoint in a language of your choice, focusing on idempotency.
  3. Run a mini‑simulation – use a local Kafka instance and a simple stream processor (e.g., Faust) to ingest fake click events and update an in‑memory counter. Observe latency and think about scaling.

FAQ

  • What is the main bottleneck in an ad click aggregator? The ingestion path—validating, deduplicating, and persisting each click—must handle the highest sustained throughput. Partitioning and a durable log relieve downstream pressure.
  • Do I need to store raw click events forever? Not necessarily. Raw events can be retained for a limited window (e.g., 30 days) for debugging; aggregated results are stored long‑term.
  • How do I choose between Redis and Cassandra for counters? If cardinality is moderate and you need sub‑second reads, Redis is simpler. For billions of distinct keys, a column store like Cassandra provides better horizontal scaling.
  • What consistency guarantees should I promise to advertisers? Real‑time dashboards can be eventually consistent. Billing should rely on batch‑computed, strongly consistent aggregates.

Frequently asked questions

What is the main bottleneck in an ad click aggregator?

The ingestion pipeline is the bottleneck because it must validate, deduplicate, and persist each click at peak rates. Partitioning, a write‑ahead log, and autoscaling ingestion nodes help keep it from choking.

Do I need to store raw click events forever?

Usually not. Keep raw events for a limited retention period (e.g., 30 days) for debugging or compliance, and rely on aggregated counters for long‑term analytics.

How do I decide between Redis and Cassandra for storing counters?

Redis is quick and easy for hot, moderate‑cardinality data. Cassandra (or another column store) scales better when you have billions of distinct keys and need durability across clusters.

What consistency guarantees should I give to advertisers?

Allow eventual consistency for real‑time dashboards, but guarantee strong consistency for billing‑critical aggregates by using batch‑computed results.

#system design#ad click aggregator#scalability#real-time aggregation#interview prep#an ad click aggregator