Does ChatGPT enhance equity for global health publications? Copyediting by ChatGPT compared to Grammarly and a human editor

Abstract

English language copyediting poses significant barriers to global health authors in academic publishing. Editing is too expensive for most researchers in low-income countries, and large language models (LLMs) like ChatGPT may offer a cost-effective alternative. The technology, however, has been criticized for its biases and inaccuracies. In a preliminary, in-depth case comparison, we compared the number and quality of corrections made by U-M GPT, a secure, University of Michigan-hosted generative AI tool, to those from Grammarly and a human editor to text from two draft papers written by Ugandan sexual and reproductive health researchers. Overall, U-M GPT made about three times as many corrections compared to the human editor and about ten times more than Grammarly. U-M GPT was the least discriminating in terms of quality: only 61% (51/83) of its corrections were judged as improvements. Despite this, U-M GPT has advantages, such as a broad scope of correction types, fast turnaround, and no cost. Its disadvantages, which reflect shortcomings of LLMs more broadly, include the need for prompt engineering skill, careful review of corrections, and high environmental costs due to energy consumption. Additional concerns involve data privacy and content moderation policies that restrict discussions on topics deemed as sensitive; these included words related to sexual and reproductive health. Although LLMs could improve equity, efficiency, and productivity, several important issues should be considered when using the technology. Larger follow-up investigations are needed to confirm our findings. Authors using LLMs should consult journal guidelines and disclose their use.