Integration of Multimodal Generative AI System for Improved Image Captioning and Document Classification: An Analysis

Abstract

Purpose: This paper investigates the integration of multimodal generative artificial intelligence (GenAI) frameworks for enhancing image captioning and document classification, focusingparticularly on applications in healthcare. De-sign/Methodology/Approach: A systematic analysis of state-of-the-art architectures Generative Adversarial Networks (GANs), Variational Autoencoders (VAEs), and Transformer-based models is conducted. We evaluate their capacity to process heterogeneous data (text and images), with experiments carried out on benchmark datasets including Reuters-21578, 20 Newsgroups, MS-COCO, Flickr30k, FUNSD, CORD, and healthcare datasets such as chest X-rays. Findings: Experimental results indicate that CNN and Hierarchical Attention Networks achieve high document classification accu-racy (>92%), while transformer-based captioning systems outperform traditional models with BLEU (35%), METEOR (28%), CIDEr (105%), and SPICE (22%). Instruction-tuned Vision-Language Models (VLMs) such as InstructBLIP and LLa-VA-1.5 further enhance zero-shot captioning and multimodal reasoning. In healthcare applications, multimodal integration improves diagnostic imaging interpretation, clinical note classification, and automated report generation. Practical Implica-tions: The framework demonstrates potential in domains requiring context-rich outputs, including medical imaging, pa-tient data analysis, and cross-domain information retrieval. Originality/Value: Unlike previous studies, this work offers a consolidated evaluation of multimodal generative AI through comparative experimentation, integration of recent models (OCR-free document AI, RAG, SAM-based grounding), and ethical considerations, establishing a roadmap for future re-search.