Google has pulled its Gemma AI model from AI Studio amid serious claims the system fabricated accusations of sexual misconduct against a sitting U.S. senator.
Senator Marsha Blackburn, a Tennessee Republican, wrote to Google CEO Sundar Pichai to outline how the Gemma model produced entirely fabricated claims about her when the AI was asked if she had been accused of rape.
The AI said, in response, that during a 1987 state senate campaign, a state trooper said Blackburn “pressured him to obtain prescription drugs for her and that the relationship involved non-consensual acts.”
Google’s Gemma AI Sparks Legal Debate, Is It “Hallucination” or Defamation?
Every single detail of this statement is made up. Blackburn did not run for state senate in 1987-the election was in 1998. But more to the point, there was never such an accusation, there was never such a state trooper, and the supposed news articles the AI referenced, when clicked on, were either error pages or completely unrelated stories.
“None of this is true, not even the campaign year,” Blackburn wrote in her letter to Pichai, noting the gravity of the falsehoods coming from Google’s AI.
It is not just about Blackburn’s case. During last week’s Senate Commerce hearing, the senator alluded to conservative activist Robby Starbuck’s lawsuit against Google, claiming that Google’s AI models, including Gemma, had come up with libelous claims that he was a “child rapist” and “serial sexual abuser.” Starbuck is suing over the false accusations.
Confronted about such issues at the hearing, Google’s Vice President for Government Affairs and Public Policy Markham Erickson described the problem as “hallucinations,” a term commonly used in AI circles to describe those instances when models generate false information. He assured lawmakers that Google is “working hard to mitigate them.”
Blackburn took issue with this framing altogether. Her letter makes the case that Gemma’s fabrications are not harmless technical glitches but rather “an act of defamation produced and distributed by a Google-owned AI model.”
The distinction is important: to describe something as a hallucination implies an inevitable technical shortcoming, while defamation implies legal liability for spreading false, injurious information.
The controversy fits into broader political tensions surrounding AI development. Supporters of President Trump in the tech industry have complained about what they call “AI censorship,” arguing that popular chatbots display liberal bias.
Google Dismisses Bias Claims Over AI Model Misuse
Trump even signed an executive order earlier this year banning what he termed “woke AI.”
In a letter, Blackburn echoed those concerns, writing that “there appears to be a consistent pattern of bias against conservative figures expressed by Google’s AI systems.”
That claim comes even though Blackburn hasn’t always taken Trump’s side on tech policy-she was among those that helped strip a moratorium on state-level AI regulation from his “Big Beautiful Bill.”
Google replied in a post on X late Friday night that didn’t directly respond to Blackburn’s specific allegations. “We have seen reports of non-developers trying to use Gemma in AI Studio and asking it factual questions,” the company wrote.
It also highlighted that Gemma was never designed for this purpose. “We never intended this to be a consumer tool or model, or to be used this way,” Google said. The company touts Gemma as a family of open, lightweight models intended for developers to integrate into their products while AI Studio is a web-based development environment for building applications powered by AI.
Google’s Gemma Removal and the AI Accountability Question
Following those incidents, Google said it’s removing Gemma from AI Studio altogether, although the models will still be available via API access to developers. That would indicate Google is attempting to block direct access to Gemma by non-technical users while continuing to permit developers to work with it in controlled environments.
The situation raises important questions regarding AI safety and accountability, and what could be the potential legal consequences in case an AI system generates false and damaging information about living persons. This is an important boundary to define as these models become increasingly more accessible and integrated into everyday tools.
The incident marks the latest challenge for Google as it tries to balance innovation with responsibility in the face of evolving AI technology under increasing scrutiny from lawmakers on both sides of the aisle.
