AI watermarking and detection technologies, while seemingly designed to enhance content authenticity, could paradoxically lead to a false sense of security, according to Nadav Ziv, writing for Time. Ziv argues that these tools may leave users overconfident in the legitimacy of content identified as genuine, a scenario with historical precedents.
Ziv points to the evolution of online information verification. In the early days of the internet, visual cues like polished design and error-free pages often signaled credible sources, while poorly designed websites were frequently associated with scams. Information experts advised users to scrutinize such features.
However, this advice became outdated as readily available tools like website builders and image editing software democratized the creation of sophisticated-looking content. This ease of use allowed malicious actors to produce convincing fake websites, rendering traditional visual inspection unreliable. A 2022 study led by Ziv found that 96% of top U.S. colleges and universities provided outdated guidance on evaluating online information, continuing to emphasize visual cues long after they ceased to be reliable indicators of authenticity.
Ziv suggests that the development of AI watermarks and detectors follows a similar pattern. Any discernible signal, such as a watermark or a pattern recognized by a detector, is likely to be identified and then circumvented by sophisticated actors. This could lead to a situation where users become overly reliant on these technological markers, potentially making them more vulnerable to deception when these markers are eventually bypassed.