Real-Time Backends

Every backend you have written so far has one shape. The client asks, the server answers, and the connection closes. Authentication, routing, services, repositories, caching, background jobs, object storage: all of it hangs off that one exchange, and none of it needs the server to ever start a conversation.

This book is about the moment that stops being enough. Two people have the same board open. One drags a task from To Do to In Progress. The other should see it move, live, without refreshing.

The lecture behind this book is Real-Time Backends (youtube.com/watch?v=wUQryt697cs, 52 minutes), and its running order is the right spine: it does not start with WebSockets. It starts with a loop that asks the server "anything new?" every three seconds, and then measures what that costs until the cost forces the next design.

The one question

That question generates every part of this book, because the inversion is not free and the bill arrives in six different currencies:

If you can say what each design bought and what it charged, you can answer almost any real-time question asked in an interview, including the ones about technologies this book does not name.

The five block types

The four highlighters

Colour is never decorative here. Each one means one thing, so a marked phrase is already half-read before you reach the words.

Figures follow the same discipline. Box drawing recedes to a pale wire so the shape reads first, NODE LABELS take full ink, and anything after a left arrow is an annotation in pencil rather than part of the machine.

What is in the lecture and what is not

Parts I to VI are the lecture, expanded, with the arithmetic filled in where it hand-waves and the production details added where it stops. Part VII is entirely outside it: authenticating a long-lived socket, the hijack that CORS does not protect you from, presence, collaborative editing, and what replaces WebSockets next. The lecture names presence and collaborative editing as things it is not covering; they are asked about constantly, so they are here.

Sections that overlap Notebook 06 (Scaling and Performance) on load balancing and statelessness, or Notebook 05 (Backend Security) on authorisation, say so and point at the section that goes deeper rather than repeating it badly.

A suggested route

1. Read Part I in one sitting, with a pen, and do the arithmetic yourself. The cost model in section 4 is the argument for everything that follows.
2. Read Parts II and III together. They are one comparison, not two topics.
3. Work Part IV with a terminal open. ulimit -n and sysctl net.ipv4.ip_local_port_range on whatever machine you have.
4. Parts V and VI are where interviews actually go. Drill them.

   BROWSER A                                          BROWSER B
   (board 412)                                        (board 412)
       │                                                  ▲
       │ 1. POST /tasks/9/move        7. frame: task.moved │
       ▼                                                  │
  ┌─────────────┐                                  ┌─────────────┐
  │ INSTANCE 1  │                                  │ INSTANCE 2  │
  │             │                                  │             │
  │ ws conns:   │                                  │ ws conns:   │
  │   A ────────┼──┐                          ┌────┼──────── B   │
  └──────┬──────┘  │                          │    └─────────────┘
         │         │  each instance holds     │
    2. commit      │  its own sockets and     │  6. look up local
         │         │  knows nothing about     │     subscribers
         ▼         │  the other's             │     for board 412
  ┌─────────────┐  │                          │
  │  DATABASE   │  └──────────────────────────┘
  │             │              ▲
  │ source of   │              │
  │ truth       │       ┌──────┴───────┐
  └─────────────┘       │   PUB / SUB  │
         │              │              │
    3. assign seq   4.  │ topic:       │  5. delivered to every
       number    publish│  board.412   │     subscribed instance
         └──────────────▶              │
                        └──────────────┘
                               │
                        ┌──────┴───────┐
                        │  EVENT LOG   │ ← only if you need at-least-once
                        │  seq 1..N    │   and a catch-up path
                        └──────────────┘

Four rules are encoded in that picture, and they are the whole architecture.

• A connection is state, and it lives in exactly one process. Instance 1 is the only machine in the world that can write to browser A. No load balancer setting changes that.
• The instances must talk to each other, because the connections cannot. That is the entire job of the pub/sub layer.
• Every event carries a sequence number. It is the only thing that makes a dropped connection recoverable, and it costs almost nothing to add on day one.
• The database is still the source of truth. The real-time path is a delivery optimisation on top of it, never a replacement for it. If the socket path is broken, a refresh must still show the correct board.

You have built a task management application. It has authentication, a database, services, repositories, caching, background jobs, and object storage for attachments. It is, by any reasonable measure, a complete backend.

Two users have board 412 open. Aas drags task 9 from To Do to In Progress. Leo, in another country, should see the card move.

Not on refresh. Not on navigation. Live.

That is the entire requirement, and it is ordinary. Nobody in the room thinks they are asking for something exotic. But it is the first feature you have ever been given that your backend has no mechanism to deliver, and the reason is worth stating precisely before reaching for a solution.

  every feature you have shipped so far
 
    client  ──── asks ────▶  server
    client  ◀─── answers ──  server
            (connection closes)
 
  this feature
 
    server  ──── ??? ─────▶  client
            (nothing the server can do starts this arrow)

Aas's move arrives at your server as an ordinary POST. That half is solved. The unsolved half is that your server now knows something Leo's browser wants, and has no way to say it.

It is worth being exact about the constraint, because the exactness is what makes the next four designs legible.

An HTTP exchange has a fixed order. The client opens a connection, writes a request line and headers, and only then may the server write anything at all. The server can be slow, it can stream a body, it can hold the connection open for minutes, but it cannot write first, and it cannot write twice.

  what the server is allowed to do          what it is not
  ────────────────────────────────          ──────────────
  respond slowly                            respond before being asked
  respond in chunks over time               respond again later, unprompted
  keep the connection open                  open a connection to the browser
  refuse to respond                         address a client it is not talking to

The last line on the right is the one people forget. Even if your server could open connections outbound, it does not know where the browser is: the browser is behind a home router, a carrier NAT, a corporate firewall, and has no stable address. The asymmetry is not politeness, it is addressability. The client can reach the server. The server cannot reach the client. The only channel that will ever exist is one the client opened.

Everything in this book is therefore a variation on one move: the client opens the channel, and the server uses it later. The designs differ only in how long "later" is allowed to be, and how many times the server may use it.

Strip away everything you know and the intuitive solution is the correct place to start: if the server cannot tell you, ask it.

  browser                                server
    │ GET /boards/412/changes?since=...   │
    ├────────────────────────────────────▶│  SELECT ... WHERE updated_at > ?
    │◀────────────────────────────────────┤  200 {"changed": false}
    │  wait 3s                            │
    ├────────────────────────────────────▶│
    │◀────────────────────────────────────┤  200 {"changed": false}
    │  wait 3s                            │
    ├────────────────────────────────────▶│
    │◀────────────────────────────────────┤  200 {"changed": true, ...}

Polling is a loop on the client that issues an ordinary request on a fixed interval and applies whatever came back. It uses nothing you do not already have: your existing routes, your existing auth middleware, your existing handlers. It survives every proxy, every corporate firewall and every ancient browser, because it is not a special case of anything.

For internal tooling with a few dozen users, polling every few seconds is a correct engineering decision and you should not apologise for it. It is the design with the fewest moving parts, and the failure mode of a missed poll is "the next poll picks it up", which is the best failure mode any design in this book has.

The reason to keep going is not that polling is wrong. It is that its cost is attached to the wrong number, and section 4 does that arithmetic.