System Design Studio

Architecture

Real-time bidding engine at scale

A DSP bidder assembled one layer at a time - from the exchange fan-out down to the spend ledger. Arrow keys or click to advance.

01

The exchange fan-out hits managed ingress

flowchart LR EX([SSP and Exchanges]) -->|OpenRTB BidRequest| RF[AWS RTB Fabric\nrate limit and filter]

SSPs and exchanges fan one impression out to every DSP in parallel. We meet them at AWS RTB Fabric - single-digit-ms private networking with an inline Rate Limiter (per-partner QPS caps), an inline OpenRTB Filter that drops wrong-geo and wrong-format requests before they cost a CPU cycle, and Error Masking. Supply-chain checks (sellers.json, ads.txt) run in the bidder app, not the Fabric.

02

Stateless bidder fleet

flowchart LR EX([SSP and Exchanges]) -->|BidRequest| RF[AWS RTB Fabric] RF --> BID[Bidder Fleet\nECS Fargate Graviton4] BID -->|BidResponse or 204| RF

Filtered, qualifying requests reach a stateless C++ bidder fleet on ECS Fargate / ARM (Graviton). Any node answers any request; we scale out on the filtered request rate, not raw ingress, so a 10x inflation surge does not 10x the fleet.

03

Sub-1 ms profile and in-process model

flowchart TD EX([SSP and Exchanges]) -->|BidRequest| RF[AWS RTB Fabric] RF --> BID[Bidder Fleet ECS Fargate] BID -->|BidResponse or 204| RF subgraph DATA[Data plane] DAX[DAX cache] --> DDB[(DynamoDB profiles and config)] end BID -->|profile read| DAX SM[SageMaker training] -.->|GBDT artifact| BID

The bidder reads the user profile and campaign config from DynamoDB fronted by DAX - sub-1 ms on cache hits, 3-5 ms on the 5 percent miss tail, both inside the 10 ms wall. Bid-price and bid-shading inference runs in-process from a SageMaker-trained compressed GBDT, no RPC in the hot path.

04

Frequency caps and budget chunks

flowchart TD EX([SSP and Exchanges]) -->|BidRequest| RF[AWS RTB Fabric] RF --> BID[Bidder Fleet ECS Fargate] BID -->|BidResponse or 204| RF subgraph DATA[Data plane] DAX[DAX cache] --> DDB[(DynamoDB profiles and config)] EC[(ElastiCache Valkey 8\ncluster 5 shards freq caps)] end BID -->|profile read| DAX BID -->|freq check| EC SM[SageMaker training] -.->|GBDT artifact| BID subgraph CTRL[Control plane] PACE[Pacing Lambda\nPID controller] -->|token chunk lease| BID end

Frequency caps read from ElastiCache for Valkey 8 replicas (cluster mode, 5 shards, 1-5 ms lag, 5-10 percent over-cap accepted). Budgets are never read in the hot path - a control-plane Lambda hands each node a token-chunk lease against a PID pacing curve; the node spends locally and fails closed (fast 204 within 8 ms) if it cannot read its allowance or the lease expires.

05

Spend ledger, effectively-once on burl

flowchart TD EX([SSP and Exchanges]) -->|BidRequest| RF[AWS RTB Fabric] RF --> BID[Bidder Fleet ECS Fargate] BID -->|BidResponse or 204| RF EX -->|burl billing notice| KIN[Kinesis Data Streams] subgraph DATA[Data plane] DAX[DAX cache] --> DDB[(DynamoDB profiles and config)] EC[(ElastiCache Valkey 8 cluster)] LED[(Spend ledger Snowflake IDs)] end BID -->|profile read| DAX BID -->|freq check| EC SM[SageMaker training] -.->|GBDT artifact| BID subgraph CTRL[Control plane] PACE[Pacing and reconcile Lambda] -->|token chunk lease| BID CB[Circuit breaker Lambda] end KIN -->|BatchWindow 1s| PACE PACE --> LED KIN --> CB -->|pause campaign| BID BID -.->|EMF spend velocity| CB

Billing notices (burl, not nurl) flow through Kinesis Data Streams (BatchWindow 1s, bisect-on-error, DLQ), deduplicated by the stable SSP burl transaction id, keyed by Snowflake ledger IDs into the ledger. Reconciliation every 1-5 s sums spend and re-allocates chunk leases; a circuit-breaker Lambda - fed by both Kinesis and the EMF spend-velocity metric - pauses any campaign spending over 3x target for 60 s.

06

Global sync, observability, compliance

flowchart TD EX([SSP and Exchanges]) -->|BidRequest| RF[AWS RTB Fabric] RF --> BID[Bidder Fleet ECS Fargate] BID -->|BidResponse or 204| RF RF -->|BidResponse or 204| EX EX -->|burl| KIN[Kinesis Data Streams] subgraph DATA[Data plane] DAX[DAX] --> DDB[(DynamoDB profiles config)] EC[(ElastiCache Valkey 8)] LED[(Spend ledger)] end BID -->|profile| DAX BID -->|freq| EC SM[SageMaker training] -.->|GBDT artifact| BID subgraph CTRL[Control plane] PACE[Pacing and reconcile Lambda] -->|chunk lease| BID CB[Circuit breaker Lambda] -->|pause| BID end KIN --> PACE --> LED KIN --> CB BID -.->|EMF spend velocity| CB DDB -->|Global Tables config only| GT[(Cross-region sync)] BID -->|EMF tiered buckets| CW[CloudWatch] LED --> S3[(S3 Object Lock audit trail)]

DynamoDB Global Tables sync campaign and budget config (not user profiles - those stay regional) across the six RTB Fabric regions, with per-region allocation capped at total/active_regions. CloudWatch EMF carries latency at bounded advertiser-tier cardinality plus spend velocity. The ledger lands in S3 under Object Lock as a tamper-evident audit trail for SOC 2 and advertiser billing.