AI Watermarking

Anthropic is introducing watermarking to Claude’s outputs.

On Twitter/X there are people who are mad about it, happy about it, indifferent about it, and in very many cases technologically illiterate about it. We would like to do better. I’m not going to belabor the technical details of how Anthropic does it, among other reasons because Anthropic has not told us how they do it. But essentially every method works in a conceptually similar way.

Watermarking

First, let’s take a look at what an AI output looks like. The below quote is straight from Claude Fable 5 in response to me asking it “Generate a paragraph in plain language describing how to distinguish between a monkey and an ape.” (If you don’t already know how to do this, see the canonical reference.)

The easiest way to tell a monkey from an ape is to look for a tail: nearly all monkeys have one, while apes—gorillas, chimpanzees, orangutans, gibbons, and bonobos—do not. Body shape offers another clue. Apes tend to be larger and more barrel-chested, with broad shoulders and long arms suited to swinging beneath branches, whereas monkeys are usually smaller and slimmer, built for running along the tops of branches on all fours. Their postures differ too: apes can sit and stand more upright, while monkeys typically hold their bodies horizontally. Faces and behavior also help—apes generally have flatter faces, larger brains relative to their size, and more complex social behaviors, including tool use in some species. So if you see a tailless, broad-shouldered primate knuckle-walking or swinging by its arms, you are looking at an ape; a smaller, tailed animal scampering along a branch is almost certainly a monkey.

That probably doesn’t scream AI to you. It doesn’t scream AI to me. It doesn’t have “It’s not X; it’s Y.” It doesn’t have a bunch of lists-of-three. It doesn’t describe anything as load-bearing. It’s not a bunch of bullet points. Not everything is described as “quiet.”

It’s not full of pauses and line breaks.

Punchy.

Decisive.

Pauses.

It does have a lot of em-dashes and one semicolon, but human writers use em-dashes and semicolons too. So you might be suspicious, but not certain. You know the kinds of things to look for, at least. Essentially, you’re looking for specific quirks of AI writing. And you know roughly what to look for because you’ve seen a million examples of human writing and a million examples of AI writing.

You know what’s good at distinguishing between things that it has millions of examples of? AI. The good people at Pangram have developed just such an AI, and while you may have heard that “AI detectors” are impossible or snake oil, this just isn’t true. They’re not that hard to make, and Pangram is probably the best-known and best-vetted example. If you doubt it — and you should doubt claims that you haven’t vetted — do the following experiments. Seriously, do them. They’re free and quick. Go to Pangram and plug in:

• Everything between the paragraph starting “That probably doesn’t scream AI…” and this line. I wrote it by hand. Does Pangram label it human? Note that this is a real challenge! I deliberately named a bunch of AI quirks, so Pangram will have its work cut out for it.
• The quoted paragraph above that Claude wrote. It’s not actually all that AI-quirky, so it’s a challenge too.
• A paragraph you wrote by hand.
• A paragraph you got Claude or ChatGPT to write for you.

I predict it’ll go four-for-four.

In essence the text is watermarked by its own style, and that style goes beyond just the obvious quirks and into subtle correlations between words. AI watermarking of the kind Anthropic will be doing is just a matter of deliberately injecting very subtle nuances in the probability of generating any given token.¹ Given enough text, these subtle correlations are totally invisible to the reader but screamingly obvious to AI.

Questions

What if it’s just a tiny bit of text?

These techniques don’t work on tiny bits of text. If I ask AI to pick a random word, and I ask a human (in this case, my son Luke) to pick a random word, can you tell which is which?

• “Summer”
• “Petrichor”

Well, in this case actually you probably can. But you get the idea. To reliably distinguish between human and AI writing you need enough of it to start getting a good count of the subtle correlations and nuances.

Why is Anthropic doing this?

The EU is mandating it. Anthropic could probably choose to comply only within the EU, but my understanding is that they’re generally supportive of watermarking and will be implementing it worldwide.

Are other companies doing it?

Yes, all the major AIs that generate images and videos already do this for that kind of media. Companies like OpenAI (who makes ChatGPT) and Google (who makes Gemini) will be doing this too, for the same reasons as Anthropic.

What if I edit the AI’s output? What if the AI edits my output?

It depends. If you take an AI output and tweak one tiny thing, it’ll register as AI. If you take a human output and have AI tweak one tiny thing, it’ll register as human. The borderline is fuzzy and difficult to predict in advance. My personal recommendation is simply to never let AI touch your writing directly. If you want an AI proofreader, give it your text and tell it to make suggestions, and take those suggestions with a grain of salt. But if you just chuck your writing at ChatGPT and say “make this better” the result is going to get detected as AI because it is AI.

Is it good to have AI watermarking?

My instinct is to expect that if the EU mandates something, it’s a bad idea. In this case though, I’m ambivalent. It simply doesn’t change the status quo very much. AI writing was already detectable in practice; now it will be detectable de jure. As a user or non-user of AI, your experience will not really be any different with or without the official watermarking.

Is it possible to create an AI that can beat an AI detector?

Yes, but there’s a problem for people who want to do this. Once a group like Pangram gets access to the detector-beating model, they’ll train on its output and then it won’t beat the AI detector anymore. So you have to build or fine-tune a model yourself, and you have to not let the general public have access to it. Most 8th graders who just want an AI to write their English essays aren’t going to do this.

Do AI detectors ever falsely classify human writing as AI?

I wouldn’t rely on them in life-and-death situations, but generally if Pangram says 100% AI, there’s very little chance it’ll be wrong.

Can a lazy student take an AI output, run it through Pangram, and keep tweaking wording until it passes?

Theoretically yes, practically it requires rewriting the whole thing.

Can a detector tell if a person used AI ideas in their writing, but they wrote all the actual words themselves?

No. It’s purely about whether the AI actually generated the words.

Matthew Springer
2026-08-11