A computer vision-based approach for automatically extracting data from bar chart raster images to facilitate meta-analysis of biomedical literature

Abstract

Although bar charts are widely used in scientific publications, their rasterized format within Portable Document Format (PDF) files complicates automated data extraction, hindering large-scale evidence synthesis and meta-analysis. To address this, we developed and evaluated an automated pipeline for extracting quantitative data from bar charts embedded in the biomedical literature. The four-stage pipeline comprises (1) image extraction and panel segmentation, (2) optical character recognition (OCR)-based text detection, (3) image disassembly to identify chart components, and (4) data reconstruction using numeric parsing and axis-based interpolation. The system combines edge detection, morphological operations, and convolutional neural network (CNN)-based figure classification using a transfer-learned Inception v3 model. Performance was validated on randomized controlled trials in age-related macular degeneration, with manually annotated values from a semi-automated labeling tool as the reference standard, and agreement was assessed using Bland–Altman analysis. Across 28 bar charts from ten publications, the pipeline correctly recognized 92.9% (95% confidence interval [CI], 77.4–98.0) of figure types, 96.0% (95% CI, 94.2–97.3) of text blocks, and 79.1% (95% CI, 74.7–83.0) of bars. For numerical reconstruction, 81.2% (95% CI, 76.3–85.2) of bar values fell within ±5% of the reference standard, 63.0% (95% CI, 57.3–68.3) within ±2%, and 48.6% (95% CI, 43.0–54.3) within ±1%. Bland–Altman analysis showed a small negative bias of −0.18 (95% CI, −0.34 to −0.02), with 94.9% of differences within the limits of agreement. Most outliers arose from OCR digit misclassification or ambiguous bar boundaries. This proof-of-concept study demonstrates the feasibility of automated data extraction from bar charts using a hybrid approach that combines image-processing heuristics with CNN-based classification. Although currently limited to bar charts and a single clinical domain, the pipeline represents a step toward scalable, end-to-end systems for automated evidence extraction to support meta-analyses across the biomedical literature.