Design Leetcode
System design for a programming platform like Leetcode

On this page
- 1. Requirements
- Functional Requirements
- Out of Scope
- Non-Functional Requirements
- 2. Capacity Estimation
- 3. API Gateway
- 4. User Profile Service
- Data Model
- Caching Strategy
- 5. Problem Catalog Service
- Data Model
- Pagination and Filtering
- Test Case Caching
- Bottlenecks in the Catalog Service
- 6. Code Execution Pipeline
- Security Isolation Stack
- Two Separate Pipelines: Run Code vs Submit
- Full Submit Flow
- WebSocket Multi-Server Problem
- Submissions Table (PostgreSQL)
- Bottlenecks in the Code Execution Pipeline
- 7. Contest Service
- Data Model
- Contest Start - Handling the Read Spike
- Real-time Leaderboard with Redis Sorted Set
- Submission Flow During Contest
- Wrong Answer Penalty Tracking
- Contest End Flow
- Bottlenecks in the Contest Service
- 8. Discussion Service
- Data Model
- 9. Full Data Flow Summary
- 10. Resilience and Fault Tolerance
- 11. Technology Choices Summary
1. Requirements
Functional Requirements
- Problem catalog (list, filter by difficulty, category, company tag, paginated)
- Code editor with multi-language support
- Run Code against visible test cases
- Code submission and judging against hidden test cases
- Contest management (timed submissions, real-time leaderboard)
- User profile (submission history, solved problems, rating, streak, contest history)
- Discussion forum per problem (comments and solutions)
Out of Scope
- Premium subscription management
- Mock interviews
- Study plans
- Video explanations
- Company-specific question sets
- Judge0 internals (treated as a black box)
Non-Functional Requirements
- Strong isolation for arbitrary code execution, user code must never affect host infrastructure
- Low latency for Run Code, results within 1-2 seconds
- Acceptable latency for Submit, verdict within 2-5 seconds
- Real-time leaderboard updates during contests
- High availability for problem catalog and editor, read-heavy and latency-sensitive
- Handle contest write spikes of ~170 submissions per second
- Exactly once submission processing, a submission must never be judged twice
2. Capacity Estimation
| Metric | Value |
|---|---|
| DAU | 5 million |
| Active problems | 4000 |
| Submission rate | ~20% of DAU submit at least once = 1 million submissions per day |
| Run Code rate | ~5× submissions = 5 million run executions per day |
| Code size per submission | ~10 KB |
| Submission storage per day | 1M × 10 KB = ~10 GB per day |
| Test cases | Average 50 test cases per problem × 4000 problems = 200,000 test case files in S3 |
| Contest participants | ~50,000 concurrent users per major contest |
| Contest submission rate | 50,000 users submitting once every 5 minutes = ~170 submissions per second at peak |
| Leaderboard size | 50,000 entries per contest sorted set in Redis |
The dominant systems challenges are code execution isolation and the contest write spike. Storage numbers are modest compared to media platforms. The hardest problem is running untrusted arbitrary code safely and returning verdicts in near real-time.
3. API Gateway
All client requests enter through the API Gateway which handles authentication, rate limiting, and routing. LeetCode's API Gateway has an important additional responsibility during contests: contest-specific rate limiting. A user should not be able to spam submissions to game the judge queue. A per-user rate limit of one submission per problem per 30 seconds prevents queue flooding without affecting legitimate usage.
4. User Profile Service
Manages user identity, submission history, solved problem tracking, contest participation history, and rating.
Data Model
Users Table (PostgreSQL)
Users
user_id UUID, primary key
username VARCHAR, unique
email VARCHAR, unique
password_hash VARCHAR
profile_pic_url TEXT
rating INTEGER (starts at 1500, Elo-style)
max_rating INTEGER
problems_solved INTEGER
easy_solved INTEGER
medium_solved INTEGER
hard_solved INTEGER
submission_count INTEGER
acceptance_rate FLOAT
streak_days INTEGER
last_active TIMESTAMP
created_at TIMESTAMP
problems_solved, easy_solved, medium_solved, hard_solved, submission_count, and acceptance_rate are all denormalized counters updated asynchronously via Kafka consumers when submissions are judged. Counting solved problems by scanning the Submissions table on every profile load would be too slow.
ContestHistory Table (PostgreSQL)
ContestHistory
history_id UUID, primary key
user_id UUID, foreign key -> Users
contest_id UUID, foreign key -> Contests
rank INTEGER
problems_solved INTEGER
total_penalty INTEGER
rating_change INTEGER
attended_at TIMESTAMP
SolvedProblems Table (PostgreSQL)
SolvedProblems
user_id UUID, foreign key -> Users
problem_id UUID, foreign key -> Problems
solved_at TIMESTAMP
language VARCHAR
PRIMARY KEY (user_id, problem_id)
The primary key on (user_id, problem_id) ensures a problem is only marked solved once regardless of how many accepted submissions a user has for it.
Caching Strategy
User profiles are cached in Redis with a short TTL. Profile pages are among the most frequently viewed pages on LeetCode. Solved problem sets are also cached per user since they are checked on every problem list page to show the green checkmark on previously solved problems.
5. Problem Catalog Service
The Problem Catalog Service is the most read-heavy service in the system. Users browse problems constantly (filtering, sorting, searching), far more often than they submit solutions.
Data Model
Problems Table (PostgreSQL)
Problems
problem_id UUID, primary key
problem_number INTEGER, unique
title VARCHAR
slug VARCHAR, unique
description TEXT
difficulty ENUM('easy', 'medium', 'hard')
acceptance_rate FLOAT
submission_count INTEGER
accepted_count INTEGER
is_premium BOOLEAN
is_active BOOLEAN
created_at TIMESTAMP
ProblemTopics Table (PostgreSQL)
ProblemTopics
problem_id UUID, foreign key -> Problems
topic ENUM('array', 'string', 'dp', 'graph', 'tree', 'binary_search', ...)
PRIMARY KEY (problem_id, topic)
ProblemCompanies Table (PostgreSQL)
ProblemCompanies
problem_id UUID, foreign key -> Problems
company VARCHAR
frequency ENUM('very_high', 'high', 'medium', 'low')
PRIMARY KEY (problem_id, company)
TestCases Table (S3 + PostgreSQL reference)
TestCases
testcase_id UUID, primary key
problem_id UUID, foreign key -> Problems
input_s3_url TEXT
output_s3_url TEXT
is_visible BOOLEAN (true for example test cases shown to users)
order_index INTEGER
Test case content is stored in S3 as files. The TestCases table stores references (S3 URLs) not the actual content. Large inputs (graph problems with 100,000 nodes) can be megabytes, storing them in PostgreSQL as text would bloat the database unnecessarily.
Pagination and Filtering
Problem list requests follow the pattern:
GET /v1/problems?page=1&limit=50&difficulty=medium&topic=dp&company=google&status=unsolved
A composite index on (difficulty, is_active, is_premium) covers the most common filter combinations. Topic and company filters use the join tables with indexed foreign keys.
For the paginated problem list, the full result set is cached in Redis with a short TTL since the problem list changes rarely (new problems added infrequently):
Key: problems:list:{filter_hash}:{page}
Value: serialized list of problem metadata
TTL: 10 minutes
filter_hash is a hash of the filter parameters so different filter combinations get separate cache keys.
The status filter (solved / unsolved) is the exception. It depends on who is asking, so it can't be part of a cache key shared by everyone. The shared list is cached without it, and the Catalog Service then filters it against the user's own cached solved set.
Test Case Caching
Hot problems (daily challenge, contest problems, NeetCode 150, Blind 75) have their test cases preloaded into Redis:
Key: testcases:{problem_id}
Value: [{input: "...", expected_output: "..."}, ...]
TTL: 24 hours for hot problems, 1 hour for cold problems
On cache miss, the Judge Worker fetches test cases from S3 and populates Redis. Subsequent submissions for the same problem hit Redis directly. Cold problems (obscure hard problems rarely attempted) are fetched from S3 on demand.
Bottlenecks in the Catalog Service
The problem list is read-heavy and rarely changes. Redis caching at the list level absorbs the vast majority of catalog read traffic. The main concern is cache invalidation when a problem is updated (description fix, new test case added): a problem.updated Kafka event triggers cache invalidation for all cached list pages containing that problem.
Finding "every cached page that contains this problem" across all filter combinations is awkward, so a simpler trick is common: put a catalog version number in every list key (problems:list:v42:{filter_hash}:{page}) and bump it on any update. All old pages instantly stop being used and expire on their own.
6. Code Execution Pipeline
This is the most unique service across all the systems we have designed. User-submitted code must be executed safely, correctly, and quickly on shared infrastructure.
Security Isolation Stack
Running arbitrary user code on your servers is one of the most dangerous operations a system can perform. A multi-layer security stack is applied to every execution:
Layer 1 - Non-root process: Code runs as a non-privileged user with no sudo access inside the container.
Layer 2 - seccomp syscall filter: A Linux kernel feature that restricts which system calls the process can make. Dangerous syscalls are blocked at the kernel level:
- Process creation limited (prevents fork bombs). Rather than banning
fork/cloneoutright, which would break runtimes like the JVM and Go that create threads, a process limit (the cgrouppidslimit, e.g.--pids-limit=64) caps how many can exist socketblocked (prevents network connections at kernel level)execveblocked once the program is running (prevents spawning other programs)ptraceblocked (prevents debugging other processes)
Layer 3 - No network interface: The container has zero network connectivity. No outbound, no inbound. Code cannot call external APIs or exfiltrate data.
Layer 4 - CPU and memory hard limits:
--memory="256m" # max 256MB RAM
--cpus="0.5" # max 50% of one CPU core
If the submission exceeds these, the container is killed and returns Memory Limit Exceeded.
Layer 5 - Wall clock timeout: A hard execution time limit (typically 2-5 seconds depending on problem). Infinite loops are killed and return Time Limit Exceeded.
Layer 6 - Read-only filesystem: Container filesystem is read-only except for a single /tmp scratch directory with a size cap. Code cannot write to system directories or accumulate disk usage.
Layer 7 - gVisor sandbox (optional, highest security): Google's gVisor is a sandboxed container runtime that intercepts all syscalls in userspace rather than passing them to the host kernel. Because user code never talks to the host kernel directly, a kernel bug can no longer be used to escape the container, which removes the biggest class of container escapes.
Two Separate Pipelines: Run Code vs Submit
Run Code and Submit have fundamentally different requirements and must not share the same pipeline.
| Run Code (lightweight, interactive) | Submit (heavyweight, authoritative) | |
|---|---|---|
| Triggered by | Clicking "Run" in the editor | Clicking "Submit" |
| Test cases | Only the 3-5 visible example test cases, hardcoded in the Problems metadata cache in Redis, no S3 fetch needed | All hidden test cases (50-100+ per problem), from Redis (hot problems) or S3 (cold problems) |
| Expected latency | 1-2 seconds | 2-5 seconds |
| Kafka topic | code.run | code.submitted |
| Workers | Dedicated Run Workers with lighter resource limits and faster container spinup | Dedicated Judge Workers with the full security stack |
| Result | Output per example, returned via WebSocket | Full verdict (Accepted, Wrong Answer, TLE, MLE, Runtime Error) + runtime ms + memory KB, returned via WebSocket |
Keeping them separate means a flood of Run clicks can never slow down Submit judging, which matters most during a contest.
Full Submit Flow
sequenceDiagram
participant C as Client
participant CS as Code Service
participant K as Kafka
participant J as Judge Worker
participant P as Redis Pub/Sub
participant WS as WebSocket Server
C->>CS: POST /submissions
CS->>CS: validate, save PENDING, code to S3
CS-->>C: submission_id
CS->>K: code.submitted
C->>WS: listen on submission:{id}
K->>J: consume
J->>J: claim job, run in sandbox
J->>K: submission.judged
K->>P: PUBLISH verdict
P->>WS: verdict
WS-->>C: Accepted / Wrong Answer / ...
Step 1 - Submission Received
User clicks Submit. Client sends POST /submissions with { user_id, problem_id, language, source_code, contest_id (if in contest) } to the Code Service via API Gateway.
Step 2 - Server-side Validation
Code Service validates:
- Language is supported
- Source code size is within limits (max 50 KB)
- If in contest: submission timestamp is within contest window (server-side check, not client-side)
- User is not rate-limited (one submission per problem per 30 seconds)
Step 3 - Submission Persisted
Source code is uploaded to S3, and a Submission record is created in PENDING state in the Submissions Table. Submission ID returned to client immediately.
Step 4 - Kafka Event Published
code.submitted event published to Kafka with submission ID, problem ID, language, and S3 URL of the source code.
Step 5 - WebSocket Connection Opened
Client opens a WebSocket connection subscribed to the submission:{submission_id} channel via Redis Pub/Sub. The user sees a spinner while waiting.
Redis Pub/Sub is fire-and-forget: a message published before anyone subscribes is simply lost. A fast Run can finish before the client's subscription is in place, so once subscribed, the WebSocket server also checks whether the verdict is already stored and sends it straight away if so. (The Chat System uses the same safety net.)
Step 6 - Judge Worker Picks Up Job
Judge Worker consumes the Kafka event. Fetches:
- Source code from S3
- Test cases from Redis (cache hit) or S3 (cache miss, then populates Redis)
Kafka delivers messages at least once, so after a worker crash or a rebalance, the same code.submitted event can arrive twice. To meet the "never judged twice" requirement, a worker first claims the submission with a conditional update, and only the worker whose update succeeds goes on to judge it:
UPDATE Submissions
SET status = 'running'
WHERE submission_id = $1 AND status = 'pending';
-- 1 row updated: this worker owns the job. 0 rows: someone else already has it, skip.
(This is the same idea as the idempotency keys and inbox pattern from the Microservices series. A worker that dies mid-judging leaves the row in running, so a sweeper resets jobs stuck in running for too long back to pending.)
Step 7 - Container Spun Up
Judge Worker spins up a language-specific Docker container with the full security stack applied. The container image is pre-warmed (not pulled fresh on each submission) to minimize startup latency.
Step 8 - Compilation
For compiled languages (C++, Java), source code is compiled first. Compilation error terminates immediately without running any test cases. Compile Error verdict returned.
Step 9 - Test Case Execution
Test cases are run sequentially. On first failure (Wrong Answer, TLE, MLE, Runtime Error), execution stops and the failing test case is reported. On all test cases passing, verdict is Accepted.
Step 10 - Container Destroyed
Container is destroyed immediately after execution completes. No state persists between submissions.
Step 11 - Result Published
Judge Worker publishes submission.judged event to Kafka with full verdict details.
Step 12 - Result Delivered to User
Kafka consumer publishes result to Redis Pub/Sub channel submission:{submission_id}. WebSocket Server subscribed to that channel receives the result and pushes it to the user's open WebSocket connection. Spinner disappears, verdict appears.
Step 13 - Async Database Update
Kafka consumer updates the Submissions Table with final verdict. If Accepted, publishes problem.solved event which triggers:
- User profile counter updates (
problems_solved,acceptance_rate) - SolvedProblems table insert
- If in contest: leaderboard update
WebSocket Multi-Server Problem
WebSocket connections are stateful. A user connected to WebSocket Server 1 cannot receive a push from WebSocket Server 3. The solution is Redis Pub/Sub:
Judge Worker -> Kafka (submission.judged)
Kafka consumer -> Redis PUBLISH submission:{submission_id} {verdict}
WebSocket Server (subscribed to submission:{submission_id}) -> push to user connection
Every WebSocket server subscribes to Redis Pub/Sub channels for all active submissions on that server. When the verdict lands in Redis Pub/Sub, the correct server instance receives it and pushes to the user. No direct server-to-server communication needed.
Submissions Table (PostgreSQL)
Submissions
submission_id UUID, primary key
user_id UUID, foreign key -> Users
problem_id UUID, foreign key -> Problems
contest_id UUID (null if not in contest)
language VARCHAR
source_code_url TEXT (S3 URL)
status ENUM('pending', 'running', 'accepted', 'wrong_answer',
'time_limit_exceeded', 'memory_limit_exceeded',
'runtime_error', 'compile_error')
runtime_ms INTEGER
memory_kb INTEGER
test_cases_passed INTEGER
total_test_cases INTEGER
submitted_at TIMESTAMP
judged_at TIMESTAMP
Source code is stored in S3, not in the database. Storing code as text in PostgreSQL would bloat the database and make it slow. The Submissions Table stores only the S3 reference URL.
Bottlenecks in the Code Execution Pipeline
Container startup latency is the primary bottleneck. Pulling a Docker image from scratch takes seconds. This is mitigated by pre-warming a pool of language-specific containers on each Judge Worker node. When a submission arrives, a pre-warmed container is used immediately and a new one is started to replace it in the background. Container startup latency goes from seconds to milliseconds.
The second bottleneck is Judge Worker autoscaling during contests. At 170 submissions per second, the worker pool must scale horizontally. Kafka decouples the submission rate from the processing rate: submissions queue in Kafka and workers process at their own pace. Kubernetes horizontal pod autoscaling adds Judge Worker pods based on Kafka consumer lag.
7. Contest Service
The Contest Service manages contest lifecycle, problem sets, and the real-time leaderboard.
Data Model
Contests Table (PostgreSQL)
Contests
contest_id UUID, primary key
title VARCHAR
start_time TIMESTAMP
end_time TIMESTAMP
duration_mins INTEGER
status ENUM('upcoming', 'active', 'ended')
participant_count INTEGER
created_at TIMESTAMP
ContestProblems Table (PostgreSQL)
ContestProblems
contest_id UUID, foreign key -> Contests
problem_id UUID, foreign key -> Problems
position INTEGER (1st, 2nd, 3rd, 4th problem in contest)
points INTEGER
PRIMARY KEY (contest_id, problem_id)
ContestParticipants Table (PostgreSQL)
ContestParticipants
contest_id UUID, foreign key -> Contests
user_id UUID, foreign key -> Users
registered_at TIMESTAMP
final_rank INTEGER (null until contest ends)
PRIMARY KEY (contest_id, user_id)
Contest Start - Handling the Read Spike
When a contest starts, 50,000 users simultaneously request the contest problems. This is a coordinated read spike similar to Netflix's new season release.
The mitigation is proactive cache warming, not reactive scaling:
- 30 minutes before contest start, Contest Service pre-loads all contest problems and their visible test cases into Redis
- All 50,000 users hit Redis on contest start, zero spike reaches PostgreSQL
- Additional API server pods are pre-scaled before the contest based on registered participant count
Key: contest:{contest_id}:problems
Value: full problem data including descriptions and visible test cases
TTL: contest duration + 1 hour
Real-time Leaderboard with Redis Sorted Set
The contest leaderboard is powered by a Redis Sorted Set:
- Key:
leaderboard:{contest_id} - Member:
user_id - Score:
(problems_solved * 10000) - total_penalty_minutes
Multiplying problems solved by 10000 ensures solving more problems always outranks a faster time on fewer problems. Penalty minutes are subtracted within that band. Each solved problem adds the minutes elapsed from the contest start until it was solved, plus 5 minutes for every wrong submission before it (compile errors usually don't count). This is the classic ICPC-style rule. LeetCode's own variant counts finish time differently, but the leaderboard mechanics are identical.
Score operations:
ZINCRBY leaderboard:contest_123 9938 user_A # problem solved at minute 47 after 3 wrong answers: 10000 - (47 + 15)
ZREVRANGE leaderboard:contest_123 0 49 WITHSCORES # top 50 users
ZREVRANK leaderboard:contest_123 user_A # user's current rank
All operations are O(log N), extremely fast even with 50,000 participants. Applying the solve and its penalty as a single ZINCRBY (rather than one for the solve and another for the penalty) means no one ever sees a score with the problem counted but the penalty not yet applied.
Submission Flow During Contest
When user A solves problem 3 at 47 minutes into the contest:
- Submission received by Code Service with
contest_idattached - Server-side validation:
submitted_at <= contest.end_time(enforced server-side, not client-side) - Judge Worker processes submission normally
- On Accepted verdict:
submission.judgedevent published to Kafka withcontest_id - Contest Leaderboard Consumer picks up event:
- Checks if this problem was already solved by this user (idempotency check in Redis, for example
SADD contest:123:solved:user_A problem_3, which returns 0 if it was already there) - If not already solved: reads the wrong answer count for this problem and applies the solve and penalty together:
ZINCRBY leaderboard:contest_123 {10000 - penalty_minutes} user_A
- Checks if this problem was already solved by this user (idempotency check in Redis, for example
- User sees green checkmark on problem 3 via WebSocket
- Leaderboard updates visible to all participants within seconds
Wrong Answer Penalty Tracking
Key: contest:{contest_id}:penalties:{user_id}:{problem_id}
Value: wrong_answer_count
TTL: contest duration + 1 hour
When a submission is judged Wrong Answer: INCR contest:123:penalties:user_A:problem_3
When the problem is eventually solved, penalty minutes = wrong_answer_count * 5 (plus the solve time) are applied to the leaderboard score. Wrong answers on a problem that is already solved are ignored.
Contest End Flow
When the contest ends:
contest.endedevent published to Kafka- Contest Service updates contest status to ended in PostgreSQL
- Final leaderboard snapshot taken from Redis and persisted to ContestHistory Table in PostgreSQL
- Rating Calculation Worker computes new Elo-style ratings for all participants based on final rankings and pre-contest ratings
- User ratings updated in PostgreSQL Users Table
- Redis leaderboard key TTL set to 7 days (for viewing past results) then evicted
- Notification Service sends contest results summary to all participants
- User Profile Service updates contest history, badges, and rating change display
Bottlenecks in the Contest Service
The leaderboard Redis Sorted Set handles 170 score updates per second comfortably since each ZINCRBY is O(log N). The read spike at contest start is handled by proactive cache warming. The rating calculation after contest end is a batch job and its latency (a few minutes) is acceptable. The main risk is Redis availability during the contest: if Redis goes down mid-contest, the leaderboard is unavailable. This is mitigated by Redis Sentinel or Redis Cluster with automatic failover, and because every accepted submission is also in Kafka and PostgreSQL, the leaderboard can always be rebuilt from scratch.
8. Discussion Service
Each problem has a discussion forum where users post solutions, ask questions, and comment. This is architecturally similar to Reddit's comment system.
Data Model
Posts Table (PostgreSQL)
Posts
post_id UUID, primary key
problem_id UUID, foreign key -> Problems
user_id UUID, foreign key -> Users
title VARCHAR
content TEXT
post_type ENUM('solution', 'question', 'discussion')
language VARCHAR (null for non-solution posts)
upvotes INTEGER
view_count INTEGER
created_at TIMESTAMP
Comments Table (PostgreSQL)
Comments
comment_id UUID, primary key
post_id UUID, foreign key -> Posts
parent_id UUID, foreign key -> Comments (null for top-level)
user_id UUID, foreign key -> Users
content TEXT
upvotes INTEGER
created_at TIMESTAMP
Same Adjacency List pattern as Reddit for nested comments. parent_id = NULL means top-level comment on a post. Non-null parent_id means reply to another comment. Lazy loading with depth limits prevents unbounded recursive fetches.
Hot posts and top comments for popular problems are cached in Redis with a short TTL. The daily challenge problem's discussion section sees enormous traffic and benefits heavily from caching.
9. Full Data Flow Summary
Problem Catalog Read Flow:
Client -> API Gateway -> Catalog Service
-> Redis (paginated list cache with filter hash key)
-> PostgreSQL (cache miss, query with composite index)
-> Redis MGET (hydrate problem metadata)
-> Redis (solved problems per user for green checkmarks and the status filter)
Run Code Flow:
Client clicks Run -> Code Service
-> Validate input (size, language, rate limit)
-> Publish to Kafka (code.run topic)
-> WebSocket connection opened (subscription to run:{run_id} Redis channel)
-> Run Worker picks up job
-> Spin up lightweight container
-> Fetch visible test cases from Redis metadata cache (hardcoded, no S3)
-> Execute code against 3-5 visible test cases
-> Destroy container
-> Publish result to Redis Pub/Sub (run:{run_id})
-> WebSocket Server pushes result to user (1-2 seconds total)
Submit Flow:
Client clicks Submit -> Code Service
-> Validate input + rate limit check + contest timing check (server-side)
-> Upload source code to S3
-> Create Submission record (PENDING) in PostgreSQL
-> Publish to Kafka (code.submitted topic)
-> WebSocket connection opened (subscription to submission:{submission_id} Redis channel)
-> Judge Worker picks up job
-> Claim it: UPDATE ... SET status = 'running' WHERE status = 'pending'
-> Fetch source code from S3
-> Fetch test cases from Redis (hot) or S3 (cold, then populate Redis)
-> Spin up language-specific container with full security stack
-> Compile (if compiled language) -> Compile Error if fails
-> Run all test cases sequentially -> stop on first failure
-> Destroy container
-> Publish submission.judged to Kafka
-> Kafka consumer -> Redis PUBLISH submission:{submission_id} {verdict}
-> WebSocket Server pushes verdict to user (2-5 seconds total)
-> Async Kafka consumers:
-> Update Submissions Table (PostgreSQL)
-> If Accepted: insert SolvedProblems, update user counters
-> If contest submission: update leaderboard sorted set in Redis
Contest Flow:
30 mins before start -> Contest Service pre-warms Redis cache with all contest problems
Contest starts -> 50,000 users hit Redis (zero DB pressure)
User submits -> Code Service (with contest_id) -> same Submit pipeline
Accepted verdict -> Contest Leaderboard Consumer
-> Idempotency check (already solved this problem?)
-> ZINCRBY leaderboard:{contest_id} {10000 - penalty_minutes} user_id
-> User sees green checkmark via WebSocket
Contest ends -> contest.ended event -> Kafka
-> Snapshot leaderboard from Redis -> PostgreSQL ContestHistory
-> Rating Calculation Worker -> update user ratings
-> Notification Service -> results email to all participants
-> Redis leaderboard TTL set to 7 days
Discussion Flow:
Client loads problem discussion -> Discussion Service
-> Redis (hot posts cache for popular problems)
-> PostgreSQL (top posts WHERE problem_id = X ORDER BY upvotes DESC LIMIT 20)
-> Lazy load comments (top-level first, replies on demand)
10. Resilience and Fault Tolerance
- Kafka submission queue absorbs contest write spikes. At 170 submissions per second, the queue depth grows during peaks and Judge Workers process at their own pace. No submissions are lost even if workers are temporarily overloaded. Kubernetes autoscaling adds Judge Worker pods based on Kafka consumer lag.
- Pre-warmed container pools on Judge Worker nodes eliminate container startup latency. A pool of idle containers per language is maintained on each worker node. Used containers are destroyed and replaced in the background.
- Redis Pub/Sub for verdict delivery ensures results reach the correct WebSocket server regardless of which server the user is connected to. WebSocket servers are stateless from the perspective of verdict routing.
- Proactive contest cache warming eliminates the read spike at contest start. All 50,000 users hit Redis simultaneously with no database pressure.
- Exactly-once judging comes from the conditional
pending -> runningclaim: a re-delivered Kafka event finds the submission already claimed and is skipped. - Idempotency checks for contest scoring prevent a single accepted submission from being counted multiple times even if the Kafka consumer retries the event. Once a problem is marked solved for a user in a contest, subsequent processing of the same event is a no-op.
- seccomp + no-network + resource limits provide defense in depth for code execution. Even if one isolation layer is bypassed, others prevent damage to the host system.
- PostgreSQL read replicas for problem catalog and user profiles ensure high availability for the read-heavy browsing experience. A primary failure does not affect problem catalog reads.
- Redis Cluster with automatic failover for the contest leaderboard ensures the leaderboard remains available even if a Redis node fails mid-contest.
11. Technology Choices Summary
| Component | Technology | Why |
|---|---|---|
| API Gateway | Kong / AWS API Gateway | Auth, rate limiting, per-user submission rate limiting |
| Problem Catalog DB | PostgreSQL sharded + replicas | Problem metadata, composite indexes for filtering |
| Problem List Cache | Redis | Paginated filtered lists, short TTL, invalidated on problem updates |
| Test Case Storage | AWS S3 | Hidden test case files, fetched by Judge Workers |
| Test Case Cache | Redis | Hot problem test cases preloaded, cold problems cached on first fetch |
| Submissions DB | PostgreSQL | Submission records, verdict history, contest submission tracking, job claiming |
| Source Code Storage | AWS S3 | Submitted source code files, referenced by S3 URL in Submissions Table |
| Run Code Queue | Apache Kafka | code.run topic, separate lightweight pipeline |
| Submit Queue | Apache Kafka | code.submitted topic, full judge pipeline |
| Run Workers | Docker containers with lightweight security stack | Visible test cases only, 1-2s latency |
| Judge Workers | Docker containers with full seccomp + gVisor security stack | All hidden test cases, 2-5s latency |
| Container Orchestration | Kubernetes | Horizontal pod autoscaling based on Kafka consumer lag |
| Verdict Delivery | Redis Pub/Sub + WebSockets | Multi-server verdict routing to correct client connection |
| Contest Leaderboard | Redis Sorted Sets | ZINCRBY for real-time score updates, ZREVRANGE for rankings |
| Penalty Tracking | Redis counters per user per problem per contest | Wrong answer count for penalty calculation |
| Contest Problem Cache | Redis | Pre-warmed 30 mins before contest start, absorbs 50k concurrent reads |
| User Profile Cache | Redis | Solved problems, ratings, counters for profile page |
| Discussion DB | PostgreSQL Adjacency List | Nested comments, lazy loaded by depth |
| Rating Calculation | Batch worker triggered by contest.ended Kafka event | Elo-style rating updates |
| Notification Delivery | APNs + FCM + AWS SES | Contest results and reminders |