A recent investigation has uncovered a concerning trend on the platform in which hundreds of accounts use artificial intelligence to mass-produce content that is raking in billions of views. Content ranges from bizarre animal videos to anti-immigrant messaging and sexualized imagery. These accounts seem specifically created to game the TikTok recommendation algorithm by flooding the service with content in hopes that something will go viral. One of the more aggressive accounts posted upwards of 70 times a day, while others kept suspiciously regular posting schedules-clear indications of automation rather than any human creator.
AI Forensics, a Paris-based nonprofit, found 354 accounts dedicated to AI-generated content, collectively posting 43,000 pieces and amassing an astonishing 4.5 billion views over just one month.
The scale of the operation raises serious questions about content authenticity and platform moderation.
The Surge of Deceptive AI Content
Most of these accounts were launched earlier this year, indicating a coordinated effort to establish a presence on the platform. TikTok for itself acknowledged last month that there are at least 1.3 billion AI-generated posts on its platform, though this is just a fraction of the 100 million pieces of content uploaded every day.
Of the most prolific accounts, half focused on content depicting the female body. The report noted these AI-generated women are “always stereotypically attractive, with sexualized attire or cleavage.” More disturbingly, some content appeared to sexualize girls who looked underage.
Other accounts churned out fake news broadcasts carrying anti-immigrant narratives, complete with appropriated branding of legitimate news organizations like Sky News and ABC. This type of content easily deceives viewers who don’t realize they’re watching AI-generated fabrications, not real news reports.
What’s most worrisome, however, is that half of the AI-generated content surveyed was unlabeled, and less than 2% had an official TikTok AI label. This lack of transparency greatly increases the material’s potential to deceive viewers who may not recognize what they see isn’t real.
These accounts routinely skirt the platform’s moderation systems for months on end, even as they post content that clearly runs afoul of its terms of service. But it appeared that some enforcement action occurred in response, as dozens of accounts identified in this study have since been deleted.
The most-viewed accounts posted what is called “slop”: nonsense AI-generated content, bizarre and designed only to clutter social media feeds. Think animals doing Olympic dives, or talking babies. While researchers said some of the content may be considered entertaining or cute, it becomes a larger issue of synthetic content crowding out human creativity.
TikTok Clashes with Report on Surging AI-Generated Content
TikTok pushed back against the findings in the report, saying the claims were “unsubstantiated” and that it was being singled out for something that affected several social media sites. A company representative said it is removing dangerous AI-generated content, blocking bot accounts, and continuing to invest in labeling technologies.
The service recently added an option to decrease the number of AI-generated postings in the feeds, but AI Forensics doubted the viability of the solution considering the general failure to correctly label the material created by AI.
The investigation shows how deeply AI-generated content has become intertwined with social media platforms and the broader ecosystem of creating viral content. Some accounts are cashing in on the trend, peddling health supplements via sham influencers or hawking tools to create one’s own viral AI content.
AI Forensics said TikTok should consider more radical solutions, such as creating a separate AI-only section of the app to segregate synthetic content from human-created posts. The researchers argue that it is high time for platforms to move beyond weak or optional labeling systems to mandatory visible identification of AI-generated material.
While the line between real human content and AI-generated materials continues to blur, it gets increasingly hard for users to discern what’s real and what’s synthetic. This is not a problem that will disappear anytime soon, with billions of views flowing into AI-created content every month-possibly changing how we use social media in general.
