Design a Payment Gateway System | System Design Question

"Design a payment gateway" is the interview question that looks like a CRUD app and turns out to be one of the hardest correctness problems you can be handed in 60 minutes. Nobody dies if a photo-sharing app shows a stale like count. If a payment system charges a customer twice, or moves money and forgets to record it, that is real money and a real lawsuit. So the whole design is bent around one idea that never lets up: move money exactly once, and be able to prove every cent later.

Let me walk it the way I would in a real interview: scope it, size it, build the architecture up from numbers, deep-dive the one part that actually matters (never double-charging), and then answer the follow-ups that always come.

If you want the interview process itself broken down separately, I have written that up in the 60-minute system design framework; here I will just run it.

First, the words: gateway, processor, PSP, and the four banks

Before scoping I need to define the vocabulary, because "payment gateway" means slightly different things to different people and interviewers will test whether you actually know the players. A card payment is a four-party model with a few pieces of software glued in front:

• Cardholder - the customer paying.
• Merchant - the business getting paid (an online store).
• Issuing bank (issuer) - the customer's bank, the one that issued their card and holds their money.
• Acquiring bank (acquirer) - the merchant's bank, the one that will receive the funds.
• Card network (scheme) - Visa, Mastercard, RuPay, Amex. The rails that route messages between the two banks and enforce the rules.

Now the software terms people blur together:

• Payment gateway - the piece that securely captures the payment details from the customer and passes them onward. Historically it just meant "the thing at the front that takes the card number." Think of it as the front door.
• Payment processor - the piece that actually talks to the acquirer and the card network to move the transaction through. The engine room.
• PSP (Payment Service Provider) - a company like Stripe, Adyen, or Razorpay that bundles the gateway and the processor (and often the acquiring relationship) into one API so you never touch a bank directly.

In an interview, "design a payment gateway" almost always means: design the system a company like an e-commerce platform runs in-house to orchestrate payments - accept the request, talk to one or more PSPs, keep an accurate internal record of the money, and reconcile it all - not "reimplement Visa." We are building the orchestration layer, and we treat the PSP as an external service with its own failure modes. That framing is the first thing I would say out loud.

How the money actually moves (authorization vs settlement)

Here is the single most important real-world fact, because half the design falls out of it: the approval and the money are two different events, hours or days apart.

Four stages, in the language the industry uses:

1. Authorization - the issuer confirms the card is valid and the funds exist, and holds (reserves) the amount. Response code 00 means approved. No money has moved yet; the customer just sees a pending hold. This is the fast, synchronous, online step (a second or two).
2. Capture - the merchant tells the issuer "yes, actually take the money I reserved." For many businesses auth and capture happen together at checkout; for others (say, something that ships later) capture happens on shipment. An authorization can expire if never captured.
3. Clearing - at end of day the acquirer bundles all captured transactions into a batch, the network exchanges the details with each issuer, everyone agrees on what happened, and fees (interchange, network, acquirer) get calculated.
4. Settlement - the real inter-bank money movement, usually T+1 or T+2 (one or two business days later). Funds leave the issuer, the network nets everything, and the acquirer deposits the balance (minus fees) into the merchant's account.

Keep this split in your head. It is why we need a ledger and reconciliation: our system says "paid" at authorization, but the bank's money-truth only finalizes at settlement, and those two views must be forced to agree.

Step 1: Scope it - and be ready for messy answers

Now the dialogue. In a real interview you do not get clean requirements handed to you; you have to pull them out, and the interviewer will sometimes be helpful, sometimes punt, and sometimes bolt on scope to see if you panic. Here is roughly how it goes:

You: When we say "payment gateway," are we building the in-house orchestration service for something like an e-commerce checkout, integrating an external PSP like Stripe? Or are we reimplementing the card network?

Interviewer: The in-house orchestration service. Assume you integrate one or more PSPs.

You: Good. Two directions of money: pay-in (customer to us) and pay-out (us to sellers/refunds). Do we need both?

Interviewer: Focus on pay-in, but mention how pay-out differs.

You: What kind of volume are we designing for?

Interviewer: You tell me.

You: Fine, I will assume a large e-commerce platform: on the order of a million successful payments a day. I will size for that and design for correctness first, because at that volume raw throughput is easy but never double-charging is hard. Writing that down as an assumption.

Interviewer: Sounds right. Oh, and it should also do real-time fraud scoring with an ML model, and support 40 currencies, and crypto.

You: Each of those is a real subsystem. Fraud scoring is a call-out to a risk service in the auth path - I will leave a hook for it and treat the model itself as out of scope. Multi-currency mostly means storing an ISO currency code and amount in the smallest unit per row, plus FX at the ledger boundary - cheap to leave room for. Crypto is a whole different rail (no chargebacks, different settlement) - I will call that out and defer it so we can nail the core. Is that split okay?

Interviewer: Works.

The rule I am following: use their answer if they give one; if they punt, pick a sensible number, say it out loud, and write it as an assumption; if they pile on scope, name what each piece would cost and defer the ones that would eat the hour.

Functional requirements

• Accept a payment request for an order (amount, currency, payment method token) and drive it to a final state.
• Authorize and capture a card payment through a PSP.
• Support refunds (full and partial) and cancellations/voids.
• Maintain an accurate internal ledger of every money movement, as the source of truth for finance.
• Receive and process asynchronous outcome notifications (webhooks) from the PSP.
• Run reconciliation against the PSP's settlement reports.
• (Mention only) Pay-out flow to sellers; fraud-scoring hook; multi-currency.

Non-functional requirements

• Exactly-once money movement. A retry, a duplicate click, or a network timeout must never cause a second charge. This is the top-priority requirement, above everything else.
• Correctness and durability over latency. We would rather be a little slow than lose or duplicate a transaction. No financial write may be silently dropped.
• Strong consistency / ACID for the payment and ledger writes. Money is not a domain for "eventually consistent, probably fine."
• Auditability. Every state change must be reconstructable after the fact - who, what, when, how much.
• High availability on the auth path (checkout down = revenue lost), but never at the cost of correctness.
• Security / compliance - PCI DSS. We must not casually store raw card numbers (more on this below).

Back-of-the-envelope: why this is not a throughput problem

Quick sizing so we design for the right thing (if the estimation habit is new to you, I unpack it in back-of-the-envelope estimation):

Payments per day        = 1,000,000
Seconds per day         = ~86,400
Average write TPS        = 1,000,000 / 86,400  = ~12 transactions/sec
Peak (say 5x average)    = ~60 TPS

Twelve transactions a second. Sixty at peak. That is nothing - a single well-tuned PostgreSQL box laughs at 60 writes/sec. So say this explicitly in the interview: this is not a scale problem, it is a correctness problem. The hard part is not "how do we do 60 TPS," it is "how do we make sure not one of those 60 is a double charge, a lost record, or a silent mismatch with the bank."

Storage is also tiny by modern standards. Say each payment plus its ledger entries is ~2 KB:

Per day   = 1,000,000 x 2 KB   = ~2 GB/day
Per year  = 2 GB x 365          = ~730 GB/year

Under a terabyte a year of the hottest data. We keep it in a relational database and archive old rows to cold storage. No exotic scaling needed. Every design decision from here optimizes for never getting the money wrong, not for QPS.

Step 2: Build the architecture from those numbers

Now I reason each component into existence. I am not going to draw a finished diagram and back-fill it; I will grow it from the requirements.

We need somewhere to record payments, and it must be ACID. The one requirement that dominates is "exactly-once, strongly consistent money movement." Candidates: a relational database (PostgreSQL, MySQL) with real ACID transactions and unique constraints, or a NoSQL store (DynamoDB, Cassandra) for scale. But we just showed there is no scale pressure - 60 TPS - and NoSQL's weaker multi-row transactional guarantees are exactly what you do not want when a single logical payment writes several rows that must all commit or all fail. Pick a relational database (PostgreSQL). The UNIQUE constraint and multi-row transactions are load-bearing features here, not niceties. Real systems agree: Stripe runs on sharded MySQL, PayPal on Oracle moving to PostgreSQL.

A payment is not a boolean, it is a state machine. The worst thing a junior candidate does is model this as paid = true. A payment moves through explicit states, and only certain transitions are legal. Modeling it as a state machine is what makes illegal operations (capturing a failed payment, refunding something that never settled) impossible rather than merely discouraged:

Terminal states (Failed, Voided, Refunded, Settled) never move again. Every transition is written down. If a webhook tries to move a payment from Settled back to Authorized, we reject it - it is not a legal edge.

We need a stateless service in front, and to accept the request at ~60 TPS. A Payment Service (multiple stateless instances behind a load balancer / API gateway) owns the state machine and the database writes. Stateless so we can run several copies for availability; the gateway does auth, TLS termination, rate limiting (I cover how to build one in design a rate limiter), and - critically - enforces the idempotency key on every state-changing call.

The PSP call is slow and unreliable, so it cannot live on the synchronous request path forever. Talking to Stripe/Adyen can take hundreds of milliseconds to seconds, can time out, and can fail transiently. We do not want the customer's HTTP request holding a database transaction open while we wait on a bank. So we split it: the Payment Service records intent durably and hands the actual PSP call to asynchronous Payment Workers via a message queue (Kafka). Workers own retries with exponential backoff. This also isolates failure - if the PSP is degraded, requests pile up in the queue instead of collapsing the API.

But async + a database write is the classic dual-write trap. If we write the payment row and then publish to Kafka, the process can crash in between: DB says "payment created," Kafka never heard about it, the charge never happens. Or the reverse. The fix is the transactional outbox: in the same database transaction that writes the payment, we also insert a row into an outbox table. A separate relay reads the outbox and publishes to Kafka, marking rows sent. Because both writes are in one ACID transaction, they are atomic - we can never end up with a payment the queue never learned about. This is the standard way to make a DB write and an event publish agree.

The money-truth needs its own append-only structure: the ledger. The payment row tells you the state of one payment. The ledger tells you where money is, in a form an auditor and the finance team trust: double-entry bookkeeping. Every movement is recorded as balanced entries - a debit to one account and an equal credit to another - and rows are never edited in place, only appended. A capture of 500 might be: debit customer_cash 500, credit merchant_receivable 500. The invariant "all debits equal all credits" must always hold; if it does not, something is broken and you can detect it. Append-only means you can always reconstruct history. This is non-negotiable for a real payment system and a huge tell in interviews.

The outcome often arrives later and out-of-band, so we need a webhook handler. Because settlement and even final capture status can land asynchronously, the PSP notifies us via webhooks (HTTP callbacks to our endpoint). A Webhook Handler receives these, verifies their signature (so an attacker cannot forge "payment succeeded"), and drives the state machine forward. Webhooks are unreliable too - they can be delayed, duplicated, or dropped - so they are a source of truth we reconcile, not one we blindly trust.

Finally, because nothing above is fully trustworthy, we need reconciliation. A Reconciliation Job runs (nightly, plus real-time where possible) and compares our internal ledger against the PSP's settlement report. This is the safety net that catches everything the live path missed. More on it in the deep dive.

Put it together and the architecture is:

That is the whole system on one page. Every box earned its place from a requirement. Now the deep dive.

Step 3: Deep dive - never charge twice (exactly-once)

This is the heart of the question, and where you win or lose it. The core problem: the network between our system and the PSP is unreliable, so we will inevitably send the same charge more than once, and we must make sure the customer's card is only hit once.

Where do duplicates even come from? Everywhere:

• The customer double-clicks "Pay."
• The mobile app's request times out, so it retries.
• Our own worker retries a PSP call after a timeout - but the timeout was on the response, and the charge actually went through.
• Kafka redelivers a message (at-least-once delivery).

The naive answer is "check the DB before charging." That has a race: two concurrent retries both read "no charge yet" and both charge. We need something stronger.

Idempotency keys

An idempotent operation is one you can run many times and get the same result as running it once - like setting x = 5 versus x = x + 5. We make "charge this card" idempotent with an idempotency key: a unique token the client generates once per logical payment (a UUID) and sends on every attempt of that payment, including retries.

The server stores that key with a UNIQUE constraint. The database becomes the arbiter of "have I seen this before," and the database is very good at that - a unique-index insert is atomic even under concurrency. The race disappears: of two concurrent inserts with the same key, exactly one wins and the other gets a constraint violation.

Here is the tricky bit most people miss: what about the first request that already reached the PSP but whose response we lost (the timeout in the diagram)? We must not treat the retry as "already done, return success" until we actually know the PSP outcome. So the idempotency record carries a status: in_progress while the PSP call is outstanding, then completed with the stored response. A retry that finds in_progress waits or returns "still processing," and we also make the downstream safe: we send the same idempotency key to the PSP (Stripe accepts an Idempotency-Key header, PayPal a PayPal-Request-Id), so even if our worker really does call twice, the PSP itself deduplicates and charges once. Defense in depth: our DB key stops most duplicates, the PSP's key catches the rest.

In TypeScript, the enforcing insert is the whole trick - let the database reject the duplicate rather than checking-then-inserting:

type PaymentStatus = "in_progress" | "completed" | "failed";

interface IdempotencyRecord {
  key: string;            // client-supplied UUID
  requestHash: string;    // hash of the request body
  status: PaymentStatus;
  responseBody: string | null;
}

async function processPayment(
  key: string,
  req: PaymentRequest,
): Promise<PaymentResult> {
  const requestHash = sha256(canonicalize(req));

  // Atomic claim. If the row exists, ON CONFLICT DO NOTHING inserts nothing
  // and rowCount is 0 - meaning someone already owns this key.
  const inserted = await db.query(
    `INSERT INTO idempotency_keys (key, request_hash, status)
     VALUES ($1, $2, 'in_progress')
     ON CONFLICT (key) DO NOTHING`,
    [key, requestHash],
  );

  if (inserted.rowCount === 0) {
    const existing = await getIdempotencyRecord(key);
    // Same key, different body = client bug. Refuse rather than guess.
    if (existing.requestHash !== requestHash) {
      throw new ConflictError("Idempotency key reused with different payload");
    }
    if (existing.status === "in_progress") {
      throw new RetryLaterError("Original request still processing");
    }
    return JSON.parse(existing.responseBody as string); // replay stored result
  }

  // We won the race - we are the only one allowed to call the PSP.
  const result = await callPsp(req, { idempotencyKey: key });

  await db.query(
    `UPDATE idempotency_keys
       SET status = $1, response_body = $2
     WHERE key = $3`,
    [result.ok ? "completed" : "failed", JSON.stringify(result), key],
  );
  return result;
}

Notice the second guard: if the same key arrives with a different body, that is a client bug (they reused a key for a new payment) and we refuse it rather than silently charging or silently returning the wrong stored result. That detail is what separates a strong answer from a memorized one.

Exactly-once = at-least-once + at-most-once

The clean way to say this in the interview: true "exactly-once delivery" is impossible in a distributed system, but exactly-once effect is very achievable, and it is the sum of two guarantees you build separately:

• At-least-once comes from retries - the worker keeps retrying (with exponential backoff) until it gets a definitive answer, so no charge is ever silently lost.
• At-most-once comes from idempotency - the keys above ensure all those retries collapse into a single actual charge.

Retries push toward "maybe more than once," idempotency clamps it back to "no more than once," and together they land on exactly once. Retries that never succeed eventually go to a dead letter queue (DLQ) - a parking lot for messages that failed too many times - where an on-call engineer or an automated job inspects them instead of retrying forever.

The ledger write and the PSP call must not disagree

One more consistency knot: when the charge succeeds, we must both write the ledger entries and mark the payment captured. If we crash between them, the money moved but our books do not show it. We already have the tools: the ledger write and the state transition happen in one ACID transaction (same database), and the event that fans out to analytics/billing goes through the outbox in that same transaction. We never do a bare dual-write across systems. This is where a candidate who name-drops "saga" should be careful - a full distributed saga with compensations is the answer when steps span services that each own separate databases; within our own DB, a single transaction plus the outbox is simpler and stronger, and you should say so.

Step 4: Reconciliation - proving the books are right

Even with all of the above, reality drifts. A webhook gets dropped. A charge succeeds at the PSP but our final update crashes. A refund is issued at the bank out-of-band. The live path is best-effort; the thing that makes the system trustworthy is reconciliation: periodically, compare our internal ledger against the PSP's authoritative settlement file (the report of what actually settled) and resolve every discrepancy.

Three kinds of mismatch and what each means:

• In our ledger, not in theirs - we think we charged, the PSP has no record. Often a payment we marked captured optimistically that actually failed. We correct our ledger.
• In theirs, not in ours - the money moved but we missed the webhook. We create the missing ledger entries (this is why the ledger is append-only and keyed by the PSP reference - we can safely insert what we missed).
• Amount/fee mismatch - we recorded 500, they settled 497 after fees. We book the fee as its own ledger entry so debits still equal credits.

Small, explainable discrepancies get auto-healed by rules; anything unexpected is flagged for a human. Reconciliation is the single strongest signal in an interview that you have thought about payments as they really are, not as a happy-path diagram. Say the line: webhooks tell us what probably happened; reconciliation proves what actually happened.

A note on PCI DSS: don't touch the card number

You cannot design a payment system without addressing card-data security, and the modern answer is refreshingly simple: avoid handling raw card numbers (the PAN) at all. Storing them drags your entire infrastructure into the strictest scope of PCI DSS (the Payment Card Industry Data Security Standard, the card networks' security rulebook), which is expensive and risky.

Instead:

• The card number is captured by a PSP-hosted field or SDK on the client and sent straight to the PSP, never touching our servers.
• The PSP returns a token - an opaque reference like pm_card_visa_x1a2... that stands in for the card. This is tokenization: swapping sensitive data for a meaningless placeholder that is useless to a thief.
• Our system only ever stores and passes around that token. To charge, we send the PSP the token, not a card number.

If you truly must store card data yourself, it lives in an isolated vault with encryption, tightly scoped access, and heavy auditing - but for an interview, "we tokenize via the PSP and keep raw PANs out of scope" is the right, senior answer. Always add: everything is over HTTPS, and the auth endpoint is rate-limited to blunt card-testing and denial-of-service abuse.

Follow-up questions to expect

These are the probes that come after the core design. Short, reasoned answers:

"What if the Payment Service database goes down mid-transaction?" The transaction never commits, so we are in a clean state - either the payment exists fully or not at all; no half-writes, that is the point of ACID. On recovery, any request that timed out is safely retried by the client with the same idempotency key, which either finds the completed record or re-runs cleanly. The database itself runs with a replica and automatic failover, since it is the one component the whole system cannot do without.

"A webhook from the PSP arrives twice. Now what?" Webhook handling is idempotent by the same mechanism: each PSP event has a unique id, we record processed event ids with a unique constraint, and a duplicate is a no-op. And even if we missed it entirely, reconciliation catches it. We never rely on webhooks being exactly-once.

"How do you scale the ledger when it gets huge?" Reads and writes are still low TPS, so the pressure is storage and reporting, not throughput. We shard the ledger by merchant (or account) id so one merchant's history stays together, archive settled rows older than N months to cold storage, and serve finance/reporting from a read replica or a separate analytics store (CDC into a columnar DB) so heavy reports never touch the write path. General database-scaling tactics are in how to scale databases.

"A single hot merchant is taking a huge share of traffic." Sharding by merchant id concentrates that load on one shard. For a genuine whale, give them a dedicated shard, and make sure the ledger append path uses per-account row locking (or an append-only design) so concurrent writes to that account do not serialize into a bottleneck.

"The PSP is down. What does the customer see?" The auth path fails fast rather than hanging. Ideally we support multiple PSPs and fail over / route around the degraded one (a real reason to keep the PSP behind an adapter interface). Requests queue for retry with backoff, and only after exhausting retries do we surface a clear "payment could not be processed, you were not charged" - which is true, because idempotency guarantees no phantom charge slipped through.

"How is pay-out different from pay-in?" Pay-in pulls money from a customer; pay-out pushes money to a seller (or refunds a customer), often via a different provider (Tipalti, bank transfer/ACH). Same ledger discipline (double-entry, idempotency, reconciliation), but the failure modes flip: the risk is paying someone twice or paying the wrong amount, and payouts are usually batched and have their own settlement timing. You reuse the entire correctness core and swap the adapter.

What you just built, and where it breaks next

Recap in one line: a correctness-first orchestration layer - an idempotent API in front, a payment state machine on an ACID relational database, a double-entry ledger as the money source of truth, an outbox + queue + workers to talk to PSPs asynchronously and safely, webhooks for outcomes, and reconciliation as the final proof - all sitting on the fact that authorization and settlement are separate events.

The bottlenecks and next scale curves, honestly stated:

• The ledger is the long-term pressure point - not on TPS but on data volume and reporting load. Sharding by account and offloading analytics is the path.
• PSP dependence is your real availability ceiling. Multi-PSP routing and failover is the next big investment, and it is a project in itself.
• Multi-region and multi-currency were deferred but real: cross-region you must decide where the ledger's source of truth lives, because you cannot have two regions independently deciding a charge happened; multi-currency means FX at the ledger boundary and per-currency accounts.
• Fraud was a hook; a mature system grows a real-time scoring service in the auth path and a dispute/chargeback subsystem behind it.

But the spine never changes, and it is the thing to walk out of the interview having demonstrated: we will not move money we did not mean to, we will not move it twice, and we can always prove where every cent went. Idempotency makes retries safe, the state machine makes illegal transitions impossible, the double-entry ledger makes the books provable, and reconciliation makes our view of reality match the bank's. Get that story straight and the rest is detail.