Decentralized queue control with delay shifting in edge-IoT using reinforcement learning

Abstract

Abstract The article presents an adaptive approach to modelling and managing the service process of requests at peripheral nodes of edge-IoT systems. This approach is highly relevant in light of increasing demands for energy efficiency, responsiveness, and self-regulation under unstable traffic conditions. A stochastic G/G/1 model with a parameterised time shift is proposed, accounting for the temporary unavailability of the device prior to request processing. Analytical expressions for key QoS indicators (delay, variability, loss, energy consumption) as functions of the shift parameter are derived, and a multi-factor reward function is constructed. A DQN-based reinforcement learning agent architecture is implemented to dynamically control the shift parameter in a decentralised manner based on the local real-time queue state. Experimental results using real-world datasets demonstrated a reduction in average delay by 17–26%, decreased fluctuations in service time, and improved queue recovery stability after peak loads compared to current state-of-the-art models. The proposed solution is traffic-type agnostic and scalable across edge architectures of varying complexity. The results are suitable for deployment in sensor networks, 5G/6G edge scenarios, and systems with dynamic QoS and energy management.