Checking whether a citation is real
An ISBN is the part of a citation you can test, because it is arithmetic rather than description.
Language models produce citations that look correct and are not. The author is plausible, the title sounds like a real book, the publisher exists, and the ISBN has thirteen digits starting with 978. Courts have sanctioned lawyers over it, and journal editors now check.
An ISBN is the part you can test, because it is arithmetic rather than description.
Start with the check digit
The last digit of an ISBN is a checksum over the other twelve. It is not chosen; it is computed. A model writing a citation is reproducing the shape of an ISBN — the 978, a plausible group, the right length — and nothing in the surrounding sentence tells it what the final digit has to be.
So an invented ISBN passes the check by luck, roughly once in ten. Nine times in ten it fails, and a failure is decisive: no real book has ever carried that number, because the number is not well-formed.
You can test it without a lookup and without sending anything anywhere — paste it into the converter.
A hyphenation error is not a check-digit error. Hyphens carry no information
and are stripped before the sum, so 978-0-226-45811-3 and
9780226458113 are the same number.
A valid check digit is weaker evidence than it looks
Three things produce a well-formed ISBN, and only one of them is a correct citation.
- The book is real and the ISBN is right.
- The ISBN was copied from a real book and attached to a different, invented title. Models do this: an identifier seen often in training gets reused in a sentence about something else.
- The number is arithmetically valid but was never issued to anything.
The check digit separates malformed from well-formed. It cannot separate these three. For that you have to look the number up and compare what comes back.
Compare the record, not just the existence
Look the ISBN up and read the record against the citation you were given. Four fields decide it:
| Field | What a mismatch usually means |
|---|---|
| Title | A different book entirely. The citation is wrong, whatever else matches. |
| Author | Often a real book with an author who did not write it. |
| Publisher | Frequently a different edition rather than a different book. |
| Year | Same — usually an edition or a reprint, worth checking before rejecting. |
Title and author mismatches are disqualifying. Publisher and year mismatches are usually the edition problem below.
The failure that survives every check
A citation can pass all of the above and still be wrong in a way that matters: the book is right, the ISBN is right, and it is the wrong edition.
Editions are repaginated. A quotation attributed to page 112 of a second edition is not on page 112 of the fourth. Hardback, paperback and ebook each carry their own ISBN, and an ebook often has no fixed pages at all. If a page number matters to your argument, the edition has to match the copy you are quoting from — verifying the ISBN does not do that for you.
When you cannot verify it
A lookup that finds nothing is not proof of invention. Cataloguing is uneven: small presses, older books, and books published outside the English-language trade are all less likely to be reachable, and a book can be perfectly real and absent from any given catalogue. Treat a failed lookup as unresolved rather than as a verdict.
What to do instead: search the title and author directly, check the publisher's own catalogue, or look it up in a library catalogue. If the book cannot be found by any of those and the ISBN fails its check digit, you are almost certainly looking at something that was generated rather than cited.
Ask for the ISBN in the first place
Most of the work above disappears if you ask differently. A model asked for a citation returns prose — author, title, publisher, year — and every field of it is description. There is nothing in it you can test without going and finding the book.
Ask for the ISBN as well and you get one field that is arithmetic, and that changes what you can do with the answer. It takes seconds to falsify, it does not depend on the model being honest about anything else, and a wrong one is wrong in a way you can demonstrate rather than suspect. It is worth asking for even when you do not intend to cite the ISBN — neither MLA nor APA prints it — because its only job here is to be checkable.
If a model declines, or produces a title with no identifier attached, that is itself informative. A book it has actually seen usually comes with one.
A short procedure
- Strip the hyphens and check the digit. If it fails, stop — the citation is not salvageable as written.
- Look the ISBN up. If nothing comes back, treat it as unresolved, not as false.
- Compare title and author. A mismatch on either is disqualifying.
- If you are quoting a page, confirm the edition matches the copy in front of you.
Steps one and two take a few seconds each: paste the number into the converter, then into the lookup. Steps three and four are the ones people skip, and they are where the wrong citations survive.
See also: how to cite a book, and building a works cited page.