System Design Notebook / 07

Real-Time Backends

What it costs to let the server speak first

Polling, long polling, SSE and WebSockets, priced honestly. The proxies that eat your stream, the six-connections-per-origin limit, and what any of it does to a server you have to keep running.

pages
90
sections
49
editions
Reading, Print, Tablet

Sign in to read it

Sign in and the whole notebook opens, all three editions with it. An account is what makes a download belong to someone.

Sign in with GitHub or Google

Start reading

How to use this book

Start reading

What is inside

Every section, and every one of them opens in the browser. Nothing here is hidden: if a section you need is not in this list, the notebook is not the right one and you should not spend a week on it.

  1. How to use this book
  2. The whole picture on one page
  3. 1The requirement: two people, one board
  4. 2The shape of every request you have written so far
  5. 3Polling, and why it is not stupid
  6. 4The arithmetic of polling
  7. 5Long polling, and the gap
  8. 6The two inversions
  9. 7A 200 that does not close
  10. 8The wire format: four fields and a blank line
  11. 9Reconnection you did not write
  12. 10Why nobody streams an LLM with EventSource
  13. 11The proxy that eats your stream
  14. 12Six connections per origin
  15. 13What SSE cannot do
  16. 14The handshake, byte by byte
  17. 15What 101 actually changed
  18. 16The frame
  19. 17Masking, and the proxy it was designed to protect
  20. 18Ping, pong, and the connection that is already dead
  21. 19Close codes, and the one you cannot send
  22. 20Choosing: polling, long polling, SSE, WebSocket
  23. 21Ceiling 1: file descriptors
  24. 22The thousand-descriptor limit is language-dependent
  25. 23Ceiling 2: the client's ports, not the server's
  26. 24Ceiling 3: memory, and where 10 KB goes
  27. 25The ceiling the benchmark did not measure
  28. 26Compression, and the budget it destroys
  29. 27What you just broke
  30. 28Sticky sessions are not the answer, and not for the reason you think
  31. 29Pub/sub: the instances talk to each other
  32. 30Redis pub/sub is at-most-once, and deploys are when it bites
  33. 31Ordering, and why sequence numbers come first
  34. 32The catch-up protocol
  35. 33Fan-out: events times subscribers
  36. 34Coalescing: the other lever
  37. 35The reconnect storm
  38. 36Deploys: draining a connection that never ends
  39. 37Authenticating a socket
  40. 38The token that never expires
  41. 39Cross-site WebSocket hijacking
  42. 40Presence: who is online
  43. 41Collaborative editing, in one page
  44. 42Where this goes next: WebTransport
  45. 43The delivery script
  46. 44Whiteboard drawing order
  47. 45Follow-up question bank
  48. Appendix A. Numbers to memorise
  49. Appendix B. Glossary
  50. Appendix C. Self-test

How to read it

Read with a pen. Every notebook opens with a question to answer before you start and asks you to redo the answer at the end, and the margin in the Print edition exists so you have somewhere to be wrong first. The Tablet edition is 16:9 with vector text, so note apps draw on it rather than treating it as a photograph.

more notebooks