A Landmark Case in Texas on GenAI Accuracy

This story didn’t get much attention when it broke late last year, but it marks a significant moment in the regulation of generative AI in healthcare—and in legal requirements for the accuracy of GenAI systems.

The Texas Attorney General reached a settlement with Pieces Technologies, a Dallas-based company that develops AI tools for the medical field. The state launched an investigation after concerns surfaced that the company exaggerated the accuracy of its products—specifically claiming a “critical/severe hallucination rate” of less than 0.001%.

Texas officials argued those statements were misleading and potentially violated the state’s Deceptive Trade Practices Act.

As part of a five-year settlement agreement, Pieces Technologies must now:

• Be fully transparent about how it calculates any performance metrics in advertising, or have its claims independently verified by a third-party auditor.
  
  
• Clearly disclose to customers any known or foreseeable limitations and risks tied to its AI technology.
  
  

This marks one of the first public enforcement actions in the U.S. involving generative AI in the healthcare sector, and one of the first specifically focused on the accuracy and performance of GenAI systems.

You can read the full announcement here: Texas Attorney General’s Office

To learn more about how Luminos helps manage the risks of GenAI systems, click “Get Started” above or reach out to contact@luminos.ai.

‍