Binary Semaphorelearn
sign in

System Design Notebook / 10

Caching

How wrong you are willing to be, and for how long

Where a cache goes and what it costs you when it is stale. The hit ratio arithmetic, the stampede that turns smooth load into spikes, and the invalidation nobody gets right the first time.

pages
77
sections
47
editions
Reading, Print, Tablet

Sign in to read it

Sign in and the whole notebook opens, all three editions with it. An account is what makes a download belong to someone.

Sign in with GitHub or Google

Start reading

How to use this book

Start reading

What is inside

Every section, and every one of them opens in the browser. Nothing here is hidden: if a section you need is not in this list, the notebook is not the right one and you should not spend a week on it.

  1. How to use this book
  2. The whole picture on one page
  3. 1The requirement: the query that runs a million times
  4. 2What a cache actually is
  5. 3The arithmetic of a hit ratio
  6. 4The cost you just took on
  7. 5What must never be cached
  8. 6The hierarchy, end to end
  9. 7Hardware caching, and the numbers worth memorising
  10. 8Why RAM is fast, and why sequential is faster still
  11. 9The caches you did not configure
  12. 10DNS, and caching at five levels
  13. 11CDN: the edge, the PoP, and the origin
  14. 12The application cache: Redis and Memcached
  15. 13Cache-aside, and the race inside it
  16. 14Read-through
  17. 15Write-through, and the claim that it is always fresh
  18. 16Write-behind
  19. 17Refresh-ahead and stale-while-revalidate
  20. 18Choosing
  21. 19Why invalidation is the hard part
  22. 20TTL is the honest answer
  23. 21Delete, do not update
  24. 22The ordering problem
  25. 23Versioned keys
  26. 24Cache key design, and the day you serve the wrong user's data
  27. 25The thundering herd
  28. 26Avalanche, and the one line that prevents it
  29. 27Penetration, and caching the miss
  30. 28Hot keys
  31. 29When the cache goes down
  32. 30Eviction: what to drop when you run out
  33. 31Cache-Control, and the directive everybody misreads
  34. 32max-age, s-maxage, and who obeys which
  35. 33ETag and conditional requests
  36. 34stale-while-revalidate
  37. 35Vary, and the cache key you did not know you had
  38. 36Single-threaded, and the command that kills production
  39. 37Persistence, and what durable means here
  40. 38Sessions and rate limits, and the bug in the counter
  41. 39Redis or Memcached
  42. 40The delivery script
  43. 41Whiteboard: adding a cache live
  44. 42Follow-up question bank
  45. Appendix A. Numbers worth memorising
  46. Appendix B. The caching checklist
  47. Appendix C. Glossary
  48. Appendix D. Self-test

How to read it

Read with a pen. Every notebook opens with a question to answer before you start and asks you to redo the answer at the end, and the margin in the Print edition exists so you have somewhere to be wrong first. The Tablet edition is 16:9 with vector text, so note apps draw on it rather than treating it as a photograph.

more notebooks