REST API Design

Every other notebook in this series is about making something work. This one is about making something understandable, which is harder, because the person who has to understand it is not in the room and never will be.

You will design an API once and then answer questions about it for two years. The questions arrive as Slack messages from a frontend engineer, as bug reports from a partner integration, as a support ticket from a customer whose retry loop created four hundred duplicate orders. Every one of those questions is a place where your interface failed to say something, and a human had to be paid to find it out instead.

The one question

The lecture behind this book is Complete REST API Design (youtube.com/watch?v=RG6q57DwV8Y, 2h04), and it names the problem exactly once, at 48:30. If you do not follow a standard, the only two ways anyone can learn your API are to read your source code or to try things until something works. Both are expensive. Both produce wrong assumptions that ship.

Everything in this book is one more thing they no longer have to ask you:

If you can say, for any design decision, what it tells the consumer and what it leaves them guessing, you can answer almost any API design question in an interview, including the ones about standards this book does not cover.

What this book is not

It is not about HTTP. Notebook 11 covers the protocol: what a header is, how a connection is established, what actually travels on the wire. This book assumes all of that and asks a different question about the same machinery: given that PATCH exists, when is choosing it the right design decision, and what does choosing it promise? Where the two touch, this book points across rather than repeating badly.

It is also not about your framework. There is no routing code here, no ORM, no controller. The lecture is emphatic about this and it is right: an API is designed before it is programmed, and the design outlives the language you first wrote it in.

The five block types

The four highlighters

Colour is never decorative here. Each one means one thing, so a marked phrase is already half-read before you reach the words.

What is in the lecture and what is not

Parts I, II, III, VI and VII are the lecture, expanded, with the specification put back where it hand-waves. Part IV is roughly a quarter lecture: it covers four status codes, and the other seven in that part are the ones interviews actually ask about. Part V corrects the lecture directly, because the pagination it teaches has a bug that appears the moment two people use the system at once.

Where a claim in the lecture is wrong, this book says so, in a trap block, with the specification reference. There are four of them.

A suggested route

1. Read Part I once and do not memorise it. It exists so that section 41 lands.
2. Parts II and III are reference. Read them with an API you have shipped open in another window, and mark every rule you broke.
3. Part IV is the highest interview yield in the book. Drill the codes until you can place any of the eleven without thinking.
4. Part V is the one to work with a pen.

One endpoint. Every decision in this book is visible in it.

  CLIENT                                                  SERVER
    │
    │   POST  /v1/organizations/8f2a/archive
    │   ────   ──  ─────────────  ────  ───────
    │    1      2        3          4      5
    │
    │   Content-Type: application/json               6
    │   Idempotency-Key: 7c1e-44b9                   7
    │   If-Match: "v8"                               8
    │
    │   { "reason": "merged into 3d10" }             9
    │ ─────────────────────────────────────────────────▶ │
    │                                                    │
    │ ◀───────────────────────────────────────────────── │
    │                                                    │
    │   202 Accepted                                10
    │   Location: /v1/jobs/44                       11
    │   ETag: "v9"                                  12
    │
    │   { "id": "8f2a", "status": "archiving" }     13
    │

Thirteen decisions, and each one is a question the consumer does not have to ask:

Four rules are encoded in that picture, and they are the whole book.

• The interface must carry the answer, or a human will carry the question. Every line above replaces a Slack message.
• Consistency beats correctness, and it is not close. A consumer who has integrated one of your endpoints has already guessed the rest. Being uniformly unusual costs them one lookup. Being occasionally standard costs them all of them.
• Defaults are part of the design. Sending nothing must work, and what comes back when they send nothing is a decision you make once for everybody.
• The method and the status code are the only two things every intermediary understands. Proxies, caches, retry libraries and load balancers read those and nothing else. Get them wrong and machinery you never wrote will do the wrong thing on your behalf.

You are building a project management platform. Organisations contain projects, projects contain tasks. Think of Jira or Linear with the ambition removed.

The backend is yours. The web client belongs to a frontend engineer two desks away, the mobile client belongs to a contractor in another timezone, and in six months a customer's integration team will connect their own systems to it without ever speaking to you. Those three groups will never read your source code. The interface is all they get.

Now consider what happens if you design it carelessly. Say you used POST where a PUT belonged, called one field description and another desc, returned 404 for an empty search result, and named one collection /organizations and another /project. Nothing here is a bug. Every endpoint works exactly as you intended.

The lecture puts the consequence plainly at 48:30. Someone integrating that API has only two ways to discover how it behaves:

  1. read your code            requires access, and the ability to read
                               your language, and time you did not budget
 
  2. call it and observe       requires guessing, and every guess that
                               works becomes a permanent assumption

Both are expensive, and the second one is worse than it looks, because an assumption that happens to work is indistinguishable from documentation until the day you change the behaviour it was resting on.

The standard is the alternative to the conversation. If you follow the same conventions everyone else follows, the consumer arrives already knowing most of your API. They know the collection is plural. They know GET will not change anything. They know 201 means something was created and 404 means it was not found. You spent none of your time and none of theirs establishing any of that.

That is the entire economic argument for standards, and it is worth stating in one line because interviews ask for it:

In 1990, Tim Berners-Lee started a project to share documents between researchers. Within about a year, that project had produced six things, and you are still using all six:

  1. the URI          a name for a thing, that anyone can write down
  2. HTTP             the rules for asking for it
  3. HTML             a format for what comes back
  4. the web server    the thing that answers
  5. the web browser   the thing that asks
  6. an HTML editor    built into the browser, and the one that did not last

That sixth one is worth a moment. The original browser could edit the page you were reading and write it back. The web was designed to be read and written symmetrically, and it took about thirty years and a great deal of JavaScript to get back to something resembling that.

Then it grew, and growth was the problem. The design assumed a research network. What arrived was the public, in numbers nobody had planned for. By 1993 the project was heading toward breakdown, not because any single piece was wrong, but because nothing in it had been designed for the number of participants it suddenly had.

Roy Fielding, then co-founder of the Apache HTTP server project, was working on exactly this problem. His question was not "how do we make the server faster". It was structural: what properties must it have so that new pieces, written by people who never coordinate, can be added without the whole thing falling over?

His answer was a set of constraints. Not features, constraints: rules about what components are not allowed to do, chosen so that useful properties fall out for free.

Two things came out of that work. Fielding and Berners-Lee co-authored the specification for HTTP/1.1, the first version of the protocol that was really engineered rather than invented. Then, in 2000, Fielding wrote his doctoral dissertation, which did something unusual: rather than proposing a new architecture, it described and named the one the web had already become.

He called it Representational State Transfer, or REST.