高能效视觉SLAM硬件加速器设计

齐修远; 刘野; 郝爽; 周军

doi:10.20193/j.ices2097-4191.2024.0039

PDF(16357 KB)

集成电路与嵌入式系统 ›› 2024, Vol. 24 ›› Issue (11) : 51-59. DOI: 10.20193/j.ices2097-4191.2024.0039

智能机器人高能效专用芯片研究专栏

高能效视觉SLAM硬件加速器设计

作者信息 +

An energy-efficient hardware accelerator for visual SLAM

Author information +

文章历史 +

摘要

随着计算机视觉技术的不断迭代和发展,以计算机视觉技术为核心的智能应用和设备逐渐在人们的日常生活和工作中扮演越来越重要的角色。其中,基于视觉的同步定位与建图技术(Simultaneous Localization and Mapping,SLAM)在机器人、无人机、自动驾驶等领域中被广泛应用,上述领域需要视觉SLAM技术为其提供精准的定位信息,以实现其精确建图和自主导航功能。然而,由于视觉SLAM算法本身的特性,计算量极大,数据依赖性极高,导致其在传统的硬件平台(CPU或GPU)上运行时,难以满足前述边缘端应用场景对实时性和低功耗的需求,成为限制视觉SLAM技术被广泛应用的关键因素。为了解决这一问题,本文基于算法与硬件协同设计的优化策略,针对ORB特征提取和匹配算法提出了一种面向视觉SLAM的高能效专用加速器,通过多种硬件设计技术提高计算性能和能效,包括基于数据依赖关系解耦的多层次并行计算技术、基于多尺寸存储桶的数据存储技术和像素级对称-轻量化描述子生成和方向计算策略。提出的视觉SLAM加速器在Xilinx ZCU104上进行了测试和验证。与ORB-SLAM2的算法精度对比,本加速器的精度在5%以内,帧率提升至108 fps,与同期其他硬件加速器相比,查找表使用降低了32.7%,FF使用降低了41.17%,同时帧率提升了1.4倍和0.74倍。

Abstract

With the continuous iteration and development of computer vision technology, intelligent applications and devices centered on computer vision are increasingly playing a crucial role in daily life and work. Among these, visual Simultaneous Localization and Mapping (SLAM) technology finds extensive applications in fields such as robotics, drones, and autonomous driving. These fields critically rely on visual SLAM to provide accurate localization information for precise mapping and autonomous navigation. However, due to the inherent characteristics of visual SLAM algorithms, which involve high computational complexity and significant data dependency, traditional hardware platforms (CPU or GPU) struggle to meet the real-time and low-power requirements of edge applications. This limitation has become a key obstacle to the widespread adoption of visual SLAM. To address this issue, this paper proposes a high-efficiency domain-specific accelerator for ORB feature extraction in SLAM, designed through a co-optimization strategy of algorithms and hardware. Various hardware design techniques are employed to enhance computational performance and energy efficiency, include multi-level parallel computing based on decoupling data dependencies, data storage technology based on multi-size buckets, and pixel-level symmetric lightweight descriptor generation and direction calculation strategies. The proposed visual SLAM accelerator was tested and verified on the Xilinx ZCU104. Compared to the algorithm accuracy of ORB-SLAM2, the accuracy of this accelerator is within 5%, and the frame rate has increased to 108 fps. When compared to other hardware accelerators of the same period, the lookup table usage is reduced by 32.7%, the flip-flop (FF) usage is reduced by 41.17%, while the frame rate is increased by 1.4x and 0.74x.

导出引用

齐修远, 刘野, 郝爽, 等. 高能效视觉SLAM硬件加速器设计[J]. 集成电路与嵌入式系统. 2024, 24(11): 51-59 https://doi.org/10.20193/j.ices2097-4191.2024.0039

QI Xiuyuan, LIU Ye, HAO Shuang, et al. An energy-efficient hardware accelerator for visual SLAM[J]. Integrated Circuits and Embedded Systems. 2024, 24(11): 51-59 https://doi.org/10.20193/j.ices2097-4191.2024.0039

中图分类号： TN47 (大规模集成电路、超大规模集成电路)

参考文献

列表( 原文顺序 | 文献年度倒序 | 文中引用次数倒序 ) 可视化分析

[1]	丁荣涛. 基于FPGA的ORB图像特征提取算法研究[D]. 北京: 北京邮电大学, 2019. DING R T. Research on ORD image feature extraction algorithm based on FPGA[D]. Beijing: Beijing University of Posts and Telecommunications, 2019 (in Chinese). 本文引用 [1]

[2]	高翔, 张涛, 刘毅, 等. 视觉SLAM十四讲从理论到实践[M]. 北京: 电子工业出版社, 2019. GAO X, ZHANG T, LIU Y, et al. Fourteen lectures on visual slam from theory to practice[M]. Beijing: Electronic Industry Press, 2019 (in Chinese). 本文引用 [1]

[3]	Michael Grupp. Python package for the evaluation of odometry and SLAM[EB/OL].(2017-06) [2024-07].https://github.com/MichaelGrupp/evo. https://github.com/MichaelGrupp/evo 本文引用 [2]

[4]	MATAS J, CHUM O, URBAN M, et al. Robust wide-baseline stereo from maximally stable extremal regions[J]. Image and vision computing, 2004, 22(10):761-767. 本文引用 [1]

[5]	LOWE D G. Distinctive image features from scale-invariant keypoints[J]. International journal of computer vision, 2004, 60(2):91-110. 本文引用 [1]

[6]	CALONDER M, LEPETIT V, STRECHA C, et al. Brief: Binary robust independent elementary features[C]// European conference on computer vision.Springer,Berlin,Heidelberg,2010:778-792. 本文引用 [4]

[7]	DETONE D, MALISIEWICZ T, RABINOVICH A. Superpoint:Self-supervised interest point detection and description[C]// Proceedings of the IEEE conference on computer vision and pattern recognition workshops,2018:224-236. 本文引用 [1]

[8]	SIMO-SERRA E, TRULLS E, FERRAZ L, et al. Discriminative learning of deep convolutional feature point descriptors[J]. Proceedings of the IEEE international conference on computer vision,2015:118-126. 本文引用 [1]

[9]	倪奇. 基于FPGA的双目特征提取与匹配系统[D]. 哈尔滨: 哈尔滨工业大学, 2019. NI Q. Binocular feature extraction and matching system based on FPGA[D]. Harbin: Harbin Institute of Technology, 2019 (in Chinese). 本文引用 [1]

[10]	LIU R, YANG J, CHEN Y, et al. eSLAM:An Energy-Efficient Accelerator for Real-Time ORB-SLAM on FPGA Platform[C]// 2018 IEEE International Conference on Systems, Man, and Cybernetics SMC.IEEE 56th ACM/IEEE Design Automation Conference (DAC),2019: 1-6. 本文引用 [1]

[11]	WANG Z, ZHANG Z, CHEN H. A Streaming Feature Extraction Accelerator using DPCM Image Compression Technique for SLAM Applications[C]// 2018 IEEE International Conference on Systems, Man, and Cybernetics SMC.IEEE IEEE 14th International Conference on ASIC (ASICON).IEEE,2021: 1-4.

[12]	SULEIMAN A, ZHANG Z, CARLONE L, et al. Navion:A 2-mW Fully Integrated Real-Time Visual-Inertial Odometry Accelerator for Autonomous Navigation of Nano Drones[J]. IEEE Journal of Solid-State Circuits, 2019.doi:10.1109/JSSC.2018.2886342.

[13]	WANG J C, WANG X W, CHARLES ECKERT, et al. 14.2 a compute SRAM with bit-serial integer/floating-point operations for programmable in-memory vector acceleration[C]// 2019 IEEE International Solid-State Circuits Conference-(ISSCC).IEEE, 2019.

[14]	KARAMI E, PRASAD S, SHEHATA M. Image matching using SIFT,SURF, BRIEF and ORB: performance comparison for distorted images[J]. arXiv preprint arXiv:1710.02726,2017.

[15]	BALNTAS V, LENC K, VEDALDI A, et al. HPatches:A benchmark and evaluation of handcrafted and learned local descriptors[C]// Proceedings of the IEEE conference on computer vision and pattern recognition,2017:5173-5182.