A remote Digital Twin interaction system based on virtual mobile infrastructure

Abstract

Abstract To address the challenge of achieving smooth real-time presentation and interaction of factory Digital Twin (DT) systems on mobile terminals, this paper proposes a Virtual Mobile Infrastructure (VMI) based Digital Twin Remote Interaction System (VDTRIS). It improves the performance of DT applications in real-time presentation and interaction on mobile devices by making the following two key improvements to the conventional VMI. First, the transmission path has been optimized by eliminating unnecessary links (e.g. OpenGL ES transmission). Concurrently, Android Docker has the capacity to access GPU resources directly by developing a mobile GPU driver. Second, integrating a dedicated Video Processing Unit (VPU) to specifically handle the RGBA coding of GPU-rendered results into H.264 format directly, thereby reducing data transmission in RGBA format and offloading CPU coding tasks that consume substantial CPU resources. These improvements collectively reduce transmission latency and CPU consumption, leading to more efficient real-time presentation and interaction on mobile terminals. Experimental results show that VDTRIS performs better than existing state-of-the-art schemes, particularly on mobile terminals with insufficient hardware configurations. By adopting VDTRIS, factory DT applications can effectively overcome constraints like geographical restrictions, insufficient computing resources of mobile terminals, and limited funds. Meanwhile, VDTRIS can be easily extended to other digital twin application fields and resource-intensive applications.