Right. Now please explain what “signal” is preserved if I take a piece of AI-generated text and run it through OCR to produce a copy, or better still, run it through text-to-speech and then back to speech-to-text. What is the persistent signal in the material that survives this transformation? Other than, of course, the overall high-level linguistic pattern as the only possible mechanism, which was my whole point.
OCR or TTS/ASR pipelines don’t change the words that are used any more than copying and pasting the text.
The signal in all cases is the selection of words used and not used. If you bias the word sampler slightly, you can detect that bias later.
Bias not in an independent, statistical sense like there are X many “thes” and Y many “ands”, but in a causal sense. The LLM knows the probability it will predict “jumps” after “the quick brown fox”. If it algorithmically chooses “hops” instead it’s a bias. For one word it might be random noise, but if you do this for every word in an algorithmic way, then the detector can estimate the probability that it detects a bias signal instead of random noise.
(Again, using words instead of tokens.)
Which is exactly what I said many posts earlier:
This is the part I disagreed with:
There is another way that is possible.
You can do much more than detect high-level patterns if you control both the generator and the detector.
I see this as a different approach where no additional signal is embedded in the generated text. This approach analyzes text to see if it “sounds like” an LLM output. If it’s looking for writing style, it’s a subjective task.
I thought of another way to explain the watermarking as I understand it.
Every time an LLM is going to generate a word:
- it calculates the probability of every word in its vocab
- takes the top K words
- tosses any of those words that don’t meet a probability threshold
- expands or compacts the probability based on temperature
- normalizes the remaining probabilities to 1
- then randomly selects the word using the weighted probabilities. This is why you’re likely to get “fox” after “the quick brown” almost every time
A detector can take a block of text, preform the steps above, and estimate how likely it would be to create that same sequence of random samples.
A generator could apply a slight bias to the sampler to make the detection more robust.
The causal aspect is handled by the LLM only. The detector is just measuring how well a set of random choices fit their probabilities.
So it becomes a question of how well does this output fit this discrete series of probability distributions?
I signed a NDA and I am a little over 6 years out of date, but I worked for one of the world’s leading companies in watermark technology for a little over 10 years. I am not a researcher, so I am not a true expert on this field but I spend 10 years working with multiple PDH researchers where I implemented\tested\gather data for them in implementing their watermarks in both audio and visual media.
You can pretty easily do this. You can even do this is such a way that the human eye isn’t going to notice the watermark but only a machine would and you can do it in such a way that you can’t copy the text without also coping the watermark.
Those methods could be defeated if you generated with AI and then just typed it out yourself instead of coping\paste. So I don’t think this will be effective in stopping people from cheating in school for example.
Audio and visual media are wildly different because you can just do steganography in pixel-space or on a spectrogram of the audio. It’s impossible for a human to tell if a pixel 50 pixels in a million are not 254 instead of 255. Changing a character is much more obvious, and the watermarking would have many many fewer characters to work with.
There is still a fair amount of freedom to encode a watermark when writing code, such as in choosing variable and function names.
What about copying and pasting as plain text in notepad. Then copying that into your Word document or wherever. That should defeat any watermark that relies on fonts & characters. If it’s based on word usage patterns, then just typing it in yourself verbatim would not defeat it.
As Snarky alluded to, the methods I was thinking of are more in the realm of steganography than word usage. If they are using word usage that wouldn’t technically be a watermark.
Edit. Nevermind. I did some quick research. They are not using a steganography method. They will be using a word choice method. I did a quick search on Claud to get an idea of what they are doing and this were the results. (Sorry I couldn’t resist using Claud to tell me about what Claude is up to.)
- partitions its vocabulary into green‑list and red‑list tokens using a keyed hash of the context
- boosts the probability of green‑list tokens during generation
- produces text that looks normal but statistically contains more green tokens than chance
- can be detected by running the same keyed partition and performing a z‑test on token choices.
A quick a dirty translation of that in layterms would be
- As the AI writes, it repeatedly generates a secret number based on the text so far.
- That secret number is used to split all possible next words into “green” (preferred) and “red” (discouraged).
- The AI subtly favors green words. Over a long passage, the text ends up with more green words than random chance would allow.
- A detector with the same secret key can measure this imbalance and confirm the text was AI‑generated.
Sign of the times, @CaveMike has made 8 replies to this thread explaining how the watermarking will work, its limitations, how it can be detected, etc, but your instinct was to go to Claude instead…
It seems to me that @wolfpup and @CaveMike are have slightly different definitions of what counts as a watermark, and no one has yet defined it in this thread.
Perhaps something like “Trap Syntax” (similar to Trap Streets that cartographers use) would be a better term than watermark. It more accurately describes what is being done.
Well it’s all proprietary so we don’t really know how it works. But I think wolfpup is thinking more like an image watermark, which is just an overlay that you can crop out. And so like Dorjan says, maybe it’s not the right term.
This would be like, sometimes I say things are groovy and sometimes I say things are radical. If I use the letter “S” an odd number of times in the preceding 1000 characters, I’ll use groovy, and if it’s even I’ll use radical. I can use this basic logic to determine if I didn’t write something. Enough of these little rules and I can determine with near certainty whether or not I wrote something. And the rules apply whether the words are typed, scanned, spoken, or scratched in sand.
To be honest, using “watermark” certainly tripped me up as I wouldn’t consider what they are doing a watermark. Thinking about it more, what they are doing could broadly fall under the technical meaning of the word even if it doesn’t fall under the common usage of the word. (New technology has a tendency to do that.)
Nail hit on head, or something.
I am not sure it is helpful the way you seem to think it should be. I wonder why are they introducing this watermark? Not for you, I fear. It is for themselves. AI slop is so prevalent this days in the internet AI feeds on that AI is corrupting itself. This watermark is probably there just to try to avoid this self-poisoning. They want to avoid using it while still imposing it on everybody else, hoping to make a lot of money in the process.
I hope they fail.
“Invisible watermark” is what the OP’s article called Anthropic’s technique to comply with EU Artificial Intelligence Act. Specifically “Article 50(2), Transparency obligations for providers and deployers of certain AI systems”.
It’s not an ideal term and I’m not sure the EU law uses that term.
Yeah and @steronz’s example of groovy and radical is clearer example of the concept than any of mine.
I think this is the paper Claude was describing. It has some good graphically examples: A Watermark for Large Language Models
Note the paper uses the term watermark.
As @steronz said though, any vendor’s technique will be propriety.
It’s for compliance with EU Artificial Intelligence Act. It might be helpful for C2PA; I haven’t looked.