Google is making SynthID Text, its technology that lets developers watermark and detect text generated by generative AI models, generally available.

SynthID Text can be downloaded from the AI platform Hugging Face and Google's updated Responsible GenAI Toolkit.

"Today, we're open sourcing our SynthID Text watermarking tool," the company wrote in a post on X. "Available freely to developers and businesses, it will help them identify their AI-generated content."

So how does it work?

Given a prompt like "What's your favorite fruit?," text-generating models predict which "token" most likely follows another -- one token at a time. Tokens are the building blocks a generative model uses to process information. They can be a single character, word, or part of a phrase.

The model assigns each possible token a score, which is the percentage chance it's included in outputted text. SynthID Text inserts additional data in this token distribution by "modulating the likelihood of tokens being generated," Google says.

"The final pattern of scores for both the model's word choices combined with the adjusted probability scores are considered the watermark," the company wrote in a blog post. "This pattern of scores is compared with the expected pattern of scores for watermarked and unwatermarked text, helping SynthID detect if an AI tool generated the text or if it might come from other sources."

Google claims that SynthID Text, which has been integrated with its Gemini models since this spring, doesn't compromise the quality, accuracy, or speed of text generation, and works even on text that's been cropped, paraphrased, or modified.

But the company also admits that its watermarking technology has limitations.

For example, SynthID Text doesn't perform as well with short text or text that's been rewritten or translated from another language, and with responses to factual questions. "On responses to factual prompts, there are fewer opportunities to adjust the token distribution without affecting the factual accuracy," explains the company. "This includes prompts like 'What is the capital of France?' or queries where little or no variation is expected like 'recite a William Wordsworth poem.'"

This article originally appeared on TechCrunch at https://techcrunch.com/2024/10/23/google-releases-tech-to-watermark-ai-generated-text/