Action-factorized Rainbow deep Q-network with token Transformer for computation offloading in edge computing-enabled Internet of Ships

Abstract

Edge computing (EC) in the Internet of Ships (IoS) reduces the latency and energy burdens of cloud-centric architectures, but fully realizing its benefits requires effective computation offloading strategies. Designing such strategies in dynamic maritime environments remains challenging due to the high-dimensional, combinatorial decision space, strict system constraints, and rapidly varying maritime wireless channels. This study proposes the action-factorized Rainbow deep Q-network (DQN) with token Transformer, a deep reinforcement learning (DRL) algorithm for discovering effective computation offloading strategies in EC-enabled IoS (EC-IoS). The core innovation of the algorithm lies in a novel action factorization mechanism coupled with our custom token Transformer-based state and action encoders, which effectively handle the complex decision space. Built upon Rainbow DQN and further accelerated with a parallel training architecture, the algorithm improves learning efficiency and stability. Experimental results illustrate that the computation offloading strategies learned by our algorithm significantly outperform multiple baselines on the weighted latency–energy objective. More importantly, these strategies achieve a zero rate of invalid actions, satisfy all system constraints, and ensure practical feasibility. Overall, the study demonstrates that the algorithm provides a robust method for computation offloading, effectively balancing latency and energy consumption in EC-IoS, thereby supporting maritime digitalization and automation.