Road to a book lexicon (Part 1)

The article where I don't define a lexicon, and instead, complain about ISBNs


Books are an important piece of cultural material, and, the subject of many conversations. In trying to design a lexicon for expressing them, we've hit a fundamental problem in computing: How do you represent things which exist in "the real world", AKA not just in tidy databases with clean URLs ready to be distributed. You might ask, but isn't there such a thing as an ISBN? If we've numbered all books surely they are stored somewhere? Right? Right?!

Meme of a guy saying "I've numbered all books since 1970" a girl asks "Cool, so you have all of them in a database, right?" and he just stairs blankly while she realizes that he has not done so

Unfortunately, no. The ISBN standard only specifies the format of the digits themselves, allocating ranges of numbers by country and publisher. There is no central database behind the numbers. Registration is scattered across national agencies and publisher feeds, and none of them are required to tell anyone what book a number belongs to. An ISBN proves a publisher paid for a number. It doesn’t tell you which book is which. To make a hard problem harder, every "edition" of a book is given a new identifier so:

An ISBN cannot be used as a generic identifier for a work or title. A different ISBN is assigned to each separate edition and variation of a publication, but not to a simple reprinting of an existing item. For example, an e-book, a paperback and a hardcover edition of the same book must each have a different ISBN, but an unchanged reprint of the hardcover edition keeps the same ISBN.

- Wikipedia (emphasis mine)

This situation has led platforms that want to talk about books to have to build up their own dataset (or more likely bootstrap off of another one or some amalgamation of several datasets if they dare). The datasets don’t share a schema (one calls it title and authors, another name and writer), and worse, they don’t share identifiers.

When you and I talk about having read a book, we don't really care that you read the first edition hardcover, and that I read a paperback; we agree that we read the same book. But, by ISBN alone we cannot actually know if we did read the same book, since if I don't already have your ISBN in my database, I don't know if you read my book or an entirely different one. Most services get around this by building some sort of mapping or grouping system to say: "These 6 different ISBN numbers are actually all the same book". This leads us to the distinction of an Edition (i.e. a single ISBN) and a Work (some grouping of identifiers that point to a particular book). A Work more closely matches what most of us would think of as a book, and an Edition is the sort of pedantry that only a publisher would ever care about.

Editions are facts, Works are opinions

An Edition is roughly tied to an ISBN (or ASIN, SBN, etc.), whereas Works are arbitrary groupings of those ISBNs depending on utility. For example, the hardcover, the paperback, the e-book, and the audiobook of Dune each have their own identifier, but every reader would say they’re the same book. Those are four separate Editions; the one book they all point at is the Work. And services disagree about where the Work ends: does the French translation belong to it? The abridged version? The annotated edition? Every platform answers differently, and none of them are wrong.

So if every service draws its own Work boundaries differently, a shared lexicon can’t be built on Works. There’s nothing to agree on. Editions are the only level where an identifier exists that nobody in the room made up: an ISBN can be checked by anyone, while a Work ID is always somebody’s opinion with a database behind it.

An interoperable book lexicon should then be primarily (not completely!) based on Editions, not Works.

So, what does this look like in practice?

Now, I won't be getting into what fields a book record should have just yet; but, I can offer some part of the solution upfront and the reasoning for its shape:

type Edition = {
  identifiers: Array<Identifier>; // global or platform name-spaced
  // other fields TBD for another article
};

type Identifier = {
  type: string;  // "isbn", "asin", or domain like "bookhive.buzz"
  value: string; // the identifier itself, in canonical form
  url?: string;  // optional: where to see it in the system that minted it
};

We've established that services have been forced to mint their own identifiers to create Works in their respective systems. So, we've chosen to link an Edition to many Identifiers, allowing for both "real-world" identifiers like ISBNs and service-specific identifiers to live in the same structure.

For global identifiers (like ISBN, ASIN, etc.), type is a well-known token (e.g. isbn, asin, etc.). For platforms, type is a domain which naturally segments platforms into namespaces. You can compare identifiers by an exact string match between both type and value. That's the entire matching algorithm, reducing the need for any sort of fuzzy logic, or inspecting the shape of an identifier. Two Editions refer to the same thing when their identifiers share entries. There are a couple of extra rules needed for this to work though, given that something like an ISBN can often be seen as hyphenated or not. For example, 0-7653-4832-1 and 978-0-7653-4832-6 are the same number in two encodings, so the canonical form for isbn is the hyphen-less ISBN-13, which every ISBN-10 converts to losslessly. This guarantees that two services will write the same ISBN for the same Edition. The optional url allows for further disambiguation by allowing platforms to link to their canonical location on their website which may not map 1:1 with their identifier (e.g. /dune vs. /1234).

We've secretly brought back Works again

With this construction, we get Works back for free. When two Editions both carry {type: "bookhive.buzz", value: "bk_123"}, that shared identifier is BookHive’s claim that they’re the same book. A Work is now more honestly represented: not as its own entity, but as a grouping of Editions. Platforms can disagree (a review site and a translation site will draw different boundaries around the same Editions) and nobody is wrong, because nobody’s grouping is canonical.

For that reason, we likely won’t define a shared lexicon for a Work. A platform that wants richer Work data than a bare grouping can publish its own shape under its own NSID, and platforms have room to converge on those shapes later. It’s enough work to define the base layer as it is!

I know that all of this is very riveting! And, I'm excited too! But, I'll have to leave you on a cliff hanger here until Part 2, where we will discuss the shape of lexicon itself, now that we've sorted out how to identify and talk about an Edition vs. a Work.


4
2