SpikeMicroNet: neuromorphic visual sensing for energy-efficient optical microrobot pose and depth estimation under microscopy

Abstract

Abstract Optical microrobots actuated by optical tweezers (OT) are an emerging tool for cell-level manipulation, micro-assembly, and targeted biomedical interventions, but their closed-loop control depends on perception subsystems that must run in real time within tight power budgets. Existing perception pipelines on optical-microscopy data rely on dense artificial neural networks that consume several to tens of millijoules per inference, which is incompatible with embedded controllers driving the optical hardware. This paper proposes SpikeMicroNet , a directly trained, defocus-aware spiking neural network for single-frame optical microrobot perception. The proposed model integrates a Phase-Coded Defocus Encoder (PCDE) that converts a static microscopy frame into a temporally structured spike train, an Adaptive-Threshold Leaky Integrate-and-Fire (AT-LIF) neuron that maintains stable firing rates across heterogeneous microrobot geometries, and a Defocus-Aware Spiking Self-Attention (DASSA) block whose attention map is conditioned on the temporal phase of the encoder. The performance of the proposed method is benchmarked on the OpTical MicroRobot dataset against seven ANN and SNN baselines under a subject independent evaluation protocol. SpikeMicroNet attains $$95.1\%$$ top-1 accuracy on pose classification and a depth mean absolute error of $$1.78\,\mu$$ m while reducing the estimated model-side compute energy by up to $$28.4\times$$ relative to the strongest dense vision transformer baseline. To the best of our knowledge, this is the first spiking neural network designed and benchmarked for optical microrobot perception.