Recent news story: Anthropic announces that very soon, all its models will embed a watermark in all output they generate to mark it as AI-generated. This applies to all categories of output, including text. And apparently, the watermark wouldn’t be just metadata in a file; it’s meant to stay with the text as it is copied and pasted elsewhere, so it must be in the text itself.
Question: How would you do that? The first idea that came to my mind would be an acrostic, where the first letters of each word spell out a message. This has a long literary tradition. No doubt Anthropic would come up with something much more sophisticated than that, but I suppose the general idea would be the same? That the structure and arrangements of characters in the text encodes a message hidden underneath the plain message spelt out in the text?
If that’s the general idea, then I suppose the watermark could easily be removed by editing the text? Or is there a way to introduce some kind of redundancy in this letter arrangement to ensure that it is preserved even if the text is moderately edited?
And apparently this watermark will be introduced in the output of all Anthropic models, including Claude Code. But this introduces a massive conflict of goals here - when you write code, you’re not free to arrange words and letters any way you like (and need to encode your watermark), because that will affect the functionality of the code you produce.
Quite possibly I’m oversimplifying this, and Anthropic has much more intricate and subtle techniques at its disposal than I am thinking of, but I can’t quite imagine how this is supposed to work.
I’m trying to find this in something official, but I see a lot of places saying that code, amongst a few other things, is exempt from this.
Marking of AI Content (Article 50(2)). Providers of AI systems (expressly including general purpose AI systems) that generate synthetic audio, image, video or text content must ensure that: (i) outputs are marked in a machine readable format that is detectable as AI-generated or manipulated; and (ii) their technical solutions are effective, interoperable, robust and reliable taking into account various factors, in each case, excluding AI systems that perform an assistive function for standard editing or do not substantially alter the input data, and AI systems used to detect, prevent, investigate or prosecute criminal offences. The Guidelines note that Article 50 applies to AI systems generally, not only general purpose AI (as noted in the AI Act itself), so narrow, single-purpose tools such as voice cloners are potentially caught. However, the Guidelines also confirm that certain AI systems and outputs fall outside Article 50(2), such as systems that merely reproduce or rearrange existing content (e.g. music playlists or recommender systems). The Guidelines also identify specific categories that fall outside of scope: (a) content that falls outside Article 50(2) altogether on the basis that there is no risk of deception, manipulation or to ensure integrity and trust in the information, and that is not perceptible to people or not intended to be interpreted, verified or acted upon by natural persons, including short outputs (e.g. single words, image captions and data labels), source code, machine-to-machine communications, and interim outputs in closed loop environments in industrial and product development workflows; and (b) what amounts to an assistive function for standard editing or does not substantially alter the input, such as grammar and spell-checking, format conversion and minor cropping.
My understanding is that it has to do with frequencies of certain letters and letter combinations. If the letter “z” is expected to appear every 500 characters on average but Claude output has it show up every 350, that’s a tell. Do that across several different markers and you have a pretty reliable watermark.
But they can’t actually tell users what those watermarks are or it does become trivial to hunt for them and smooth them away.