Evaluating online health information on PIFP: A cross-platform analysis of content quality and readability

Abstract

Abstract Persistent Idiopathic Facial Pain (PIFP), also known as atypical facial pain, is a poorly understood condition, often leading patients to seek information online. The accuracy, readability, and usability of such information are critical for informed decision-making. This study aimed to assess and compare the readability, quality, and usability of online content about PIFP retrieved from both conventional web searches and generative AI platforms. A cross-sectional comparative analysis was conducted on January 26th 2025 using three search. Google, ChatGPT, and Gemini. Google was searched using the terms “Atypical Facial Pain” and “Persistent Idiopathic Facial Pain,” and the first 100 results for each term were screened. ChatGPT and Gemini were prompted to generate the ten most frequently asked questions about PIFP. Selected content was evaluated using the Patient Education Materials Assessment Tool (PEMAT) for understandability and actionability, the Journal of American Medical Association (JAMA) benchmarks for content quality, and two readability measures: Flesch Reading Ease and the Simple Measure of Gobbledygook (SMOG) formulae. A total of 43 websites were included. Only 44.1% of websites met at least one JAMA benchmark, while 55.9% met none. AI-generated content failed to meet any JAMA criteria. However, AI content was significantly more understandable (mean score: 83.31%) than traditional websites (mean: 64.96%, p < 0.001). In contrast, websites had higher actionability scores (25.25%) compared to AI-generated content (11%, p < 0.001). All sources were rated as “difficult” to read. Online information regarding PIFP is often difficult to read and generally lacks both quality indicators and actionable guidance. While AI-generated content is more understandable, it lacks practical advice. These findings underscore the need for improved online patient education materials that are both high in quality and easy to understand, especially for complex pain conditions like PIFP. Health professionals and digital content developers should collaborate to ensure that online resources meet standards for clarity, reliability, and usefulness.