Analog Tensor Processing With Carbon Nanotube In‐Memory Matrix Multiplications for Edge Computer Vision Acceleration

Abstract

ABSTRACT Computer vision requires intense tensor operations, primarily matrix multiplications, imposing substantial computation demands. Using hardware such as GPU and ASICs for acceleration offers a viable solution. Their application at edge, however, can be constrained by complexity in the computing architecture and incompatibility with analog systems. Here, we prototype an analog tensor core based on carbon nanotube charge‐trapping nonvolatile memory for edge computer vision acceleration. The memory, exhibiting ∼100 linear, symmetric analog weights with fast programming (500 ns) and data processing (1 Mbit/s), enables in‐memory matrix multiplications, facilitating compact tensor core design and operation of vectors and scalars in visual tasks. As demonstrations, the analog tensor core proves three‐dimensional (3D) spatial transformation with an error of <2.81% and edge detection with a signal‐to‐noise ratio of >22 dB, underpinning its potential for accelerating computer vision tasks in, for example, autonomous driving, VR/AR, robot navigation, and industrial automation. Proof‐of‐concept simulation achieves viewfield distortion correction and edge detection of street view fisheye captures by parallel single‐layer tensor processing using our analog tensor core in large scales.