Comparative analysis of text readability and writing styles in AI-generated vs. Human-written academic abstracts

Abstract

Research article abstracts are vital in scientific publications for readers to assess a study’s significance. The increasing use of AI tools, such as Kimi, ChatGPT and DeepSeek, to generate abstracts raises concerns about their readability and writing styles compared to human-written ones. The study aims to compare the differences in text readability and writing styles between human-written against AI-generated abstracts. A total of 150 abstracts of high-impact journal articles in the field of linguistics and computer science, 75 from each discipline, and another 150 AI-generated abstracts from the same corpus of articles served as the source texts for analysis. The Readability Scoring System, a computational tool, yielded readability and writing style metrics, while expert evaluation was performed to assess the quality of AI-generated academic abstracts. The quantitative data generated were analysed using SPSS 27 with non-parametric statistical methods. Key findings revealed: (1) AI-generated abstracts exhibited significantly lower readability across eight metrics, indicating greater complexity and lower readability; (2) Discipline-specific analysis showed five differing metrics in linguistics and eight in computer science; (3) Interdisciplinary comparisons revealed non-significant differences across nine readability metrics, highlighting AI’s potential to mimic natural writing. However, it still faces challenges in generating lexically diverse content. These results underscored the current limitations of AI in generating readable and human-like abstracts, especially in technical fields.