American magazine faces criticism for several factual errors in its AI-generated article: Report

A sports magazine has come under fire for serious errors in its articles generated by Artificial Intelligence (AI). The Men's Journal, which is published by The Arena Group, came out with its first AI-generated article — "What All Men Should Know About Low Testosterone," which had a byline of "Men’s Fitness Editors." According to a report by Futurism on Thursday (February 9), the article shared a cornucopia of medical claims, nutrition and lifestyle advice. The article also suggested a specific medical treatment in the form of testosterone replacement therapy.

Like much content generated by AI, the story was written with the confident authority of an actual expert. It also had academic citations and a disclosure that it was reviewed and fact-checked by the editorial team.

However, the errors were discovered on a closer inspection. Bradley Anawalt, the chief of medicine at the University of Washington Medical Center told Futurism that the story had persistent factual mistakes and mischaracterisations of medical science which provide readers with a profoundly warped understanding of health issues. Anawalt said the story had many inaccuracies and falsehoods.

"It lacks many of the nuances that are crucial to understand normal male health," he added. Anawalt also pointed out 18 specific errors he found in the story,some about basic medical topics while others claimed sweeping links between diet, testosterone levels, and psychological symptoms.

"There is just enough proximity to the scientific evidence and literature to have the ring of truth," the chief of medicine at the UW Medical Center told Futurism, while adding,"But there are many false and misleading notes."

Last week, The Arena Group announced its magazines would start to publish AI-generated articles and chief executive officer (CEO) Ross Levinsohn assured that this practice would not result in a decline in quality. But this claim is now debatable.

The Futurism report said that the Men's Journal had been through difficult times. In 2020, the magazine’s owner at the time had laid off its entire editorial staff. The Arena Group only bought the magazine in December last year, and its staff page shows only five employees, two of whom are part-time.

After Futurism reached out to the group and the magazine about the issues, several changes started to appear in the testosterone article, with various errors pointed out by Bradley Anawalt corrected.

A message also appeared at the bottom of the story which said, "The original version of this story described testosterone replacement therapy as using ‘synthetic hormones’ and stated poor nutrition as one of the most common causes of low T, which are inaccurate."

However, there were additional changes that were not mentioned in the above note including a claim about the health risks of milk that Anawalt had flagged. After the story was rewritten, a spokesperson from the group sent a statement to Futurism which read, "Along with many media companies, we are exploring various AI software that can be additive to our editorial teams' workflow."

"An initial pilot collating previously published archival content demonstrated the potential of AI coupled with editors and writers. These early experiments are a work in progress. Based on these learnings and ongoing monitoring, we will continue to refine our use of these tools as part of our workflow, which has been and will always be anchored in editorial oversight," the statement added.

On being asked if The Arena Group was responsible to publish health advice that needed to be almost completely rewritten and whether it was confident accuracy of the rest of the bot’s stories and also if it had plans to continue publishing AI content, the statement said, "Confident, yes, but no immediate plans."

WATCH WION LIVE HERE