MeVGAN: GAN-based plugin model for video generation with applications in colonoscopy

Abstract

The generation of videos is crucial, particularly in the medical field, where a significant amount of data is presented in this format. However, due to the extensive memory requirements, creating high-resolution videos poses a substantial challenge for generative models. In this paper, we introduce the Memory Efficient Video GAN (MeVGAN)–a Generative Adversarial Network (GAN) that incorporates a plugin-type architecture. This system utilizes a pre-trained 2D-image GAN, to which we attach a straightforward neural network designed to develop specific trajectories within the noise space. These trajectories, when processed through the GAN, produce realistic videos. We deploy MeVGAN specifically for creating colonoscopy videos, a critical procedure in the medical field, notably helpful for screening and treating colorectal cancer. We show that MeVGAN can produce good quality synthetic colonoscopy videos, which can be potentially used in virtual simulators.