Adaptive spatial-temporal information processing based on in-memory attention-inspired devices

Abstract

Abstract Spatial-temporal information perception is widely used for motion processing in dynamic scenes, but present technology requires relatively huge hardware resource consumption. The attention mechanism helps the human brain extract required information from tremendous data at a low cost. Here, we propose an attention-inspired artificial intelligence architecture based on hetero-dimensional modulations between zero-dimensional contact and two-dimensional electrostatic interfaces. An adaptive spatial-temporal information processing primitive is successfully implemented based on in-memory analog computing. Experiments of attention adjustments responding to different situations validate the adaptation capability to environmental changes. A demonstration of 5×5-unit data stream processing is conducted, and intensities of spatial and temporal information are varied with attention distribution from 0% to 100%. The attention-inspired device is applied to autonomous driving edge intelligence scenarios, showing high adaptability to traffic scene variations. The proposed architecture exhibits a tens-fold latency reduction, hundreds-fold area improvement, and thousands-fold energy saving compared to the conventional transistor-based circuit.