Amazon Joins Tech Giants in Scanning Books for AI Training
The company is reportedly purchasing and destroying books to gather data for its artificial intelligence models.
Amazon has reportedly joined a growing number of technology companies, including Anthropic and Meta, in acquiring vast quantities of books to scan their content for artificial intelligence training. This practice involves destroying the books after digitization to create datasets for AI models.
The primary goal, according to a report by 404 Media, is to train AI on high-quality written material rather than relying solely on internet text, which is often of lower quality. Automated machines are used to remove the spines of the books, a process that streamlines scanning and reduces costs.
404 Media reported that it uncovered Amazon's involvement by placing a tracking device in a rare book it suspected an AI firm might purchase. The device reportedly led investigators to an Amazon warehouse in Las Vegas. Employees at the facility described their work as primarily scanning books, with some tasked with cutting the bindings off the volumes.
Book sellers have observed significant increases in sales over the past year, which they believe are driven by AI companies seeking new training data. These bulk purchases are often characterized by a wide and seemingly random selection of books, with buyers appearing indifferent to price, unlike traditional academic or library acquisitions.
Books published before 2022, particularly those predating the public release of OpenAI's ChatGPT, are considered valuable for AI training because they are less likely to contain AI-generated text. This is important to avoid a phenomenon known as "model collapse," where training on AI-generated text can degrade the performance of AI models.
Anthropic has also been engaged in a similar project, code-named "Project Panama," aimed at destructively scanning books. Internal documents revealed that the company sought to keep this initiative confidential, considering the acquisition of books essential for competing in the AI landscape. The company had previously explored sourcing books from used bookstores and libraries.