Review of AI chips design for robotics

GAO Jinyang; FAN Zhendong; BAO Minjie; WANG Ke; LI Ruifeng; KANG Peng

doi:10.20193/j.ices2097-4191.2024.0031

PDF(17520 KB)

Integrated Circuits and Embedded Systems ›› 2024, Vol. 24 ›› Issue (11) : 60-77. DOI: 10.20193/j.ices2097-4191.2024.0031

Special Topic of Energy-efficient Dedicated Chips for Intelligent Robots

Review of AI chips design for robotics

GAO Jinyang ¹ ,
FAN Zhendong ¹ ,
BAO Minjie ¹ ,
WANG Ke ¹^,^* ,
LI Ruifeng ¹ ,
KANG Peng ²

Author information +

History +

Abstract

The combination of Robots and artificial intelligence will lead the transformation of new intelligent technologies. Furthermore, as a crucial component of artificial intelligence, neural networks demonstrate immense potential in robotic perception. However, the increasing complexity of AI algorithms and the prominent energy efficiency bottleneck of general-purpose processors such as CPUs pose significant challenges. Traditional processing chips fail to effectively accommodate the inference computing tasks of large-scale neural networks. In recent years, robotic AI chips, with high computing performance and low power consumption, have emerged as an ideal choice for deploying of neural networks in robot systems due to their, attracting widespread attention. This article focusing on robotic applications, studies the current status of AI algorithms, reviews the latest advances in AI chip design technology, proposes technical difficulties and feasible technical routes, and discusses the technical trends and challenges in the design of robotic AI chips.

Key words

robots / neural network / artificial intelligence chips / software and hardware co-design / hardware accelerator

Cite this article

EndNote

Ris (Procite)

Bibtex

Download Citations

GAO Jinyang , FAN Zhendong , BAO Minjie , et al . Review of AI chips design for robotics[J]. Integrated Circuits and Embedded Systems. 2024, 24(11): 60-77 https://doi.org/10.20193/j.ices2097-4191.2024.0031

References

List( Publishing order | Descend order by publishing year | Descend order by cited within ) Chart analysis

[1]	LECUN Y, BENGIO Y, HINTON G. Deep learning[J]. Nature, 2015, 521(7553):436-444. Cited in this article [1]

[2]	LIU Q, YIN J, LEUNG V C M, et al. Applying a new localized generalization error model to design neural networks trained with extreme learning machine[J]. Neural Computing and Applications, 2016, 27(1):59-66. Cited in this article [1]

[3]	MU C, WANG D. Neural-network-based adaptive guaranteed cost control of nonlinear dynamical systems with matched uncertainties[J]. Neurocomputing, 2017,245:46-54. Cited in this article [1]

[4]	LIN Z, MA D, MENG J, et al. Relative ordering learning in spiking neural network for pattern recognition[J]. Neurocomputing, 2018,275:94-106. Cited in this article [1]

[5]	LECUN Y, BOTTOU L, BENGIO Y, et al. Gradient-based learning applied to document recognition[J]. Proceedings of the IEEE, 1998, 86(11):2278-2324. Cited in this article [1]

[6]	MCCULLOCH W S, PITTS W. A logical calculus of the ideas immanent in nervous activity[J]. The bulletin of mathematical biophysics, 1943(5):115-133. Cited in this article [1]

[7]	ROSENBLATT F. The perceptron:a probabilistic model for information storage and organization in the brain[J]. Psychological review, 1958, 65(6):386. https://doi.org/10.1037/h0042519 https://www.ncbi.nlm.nih.gov/pubmed/13602029 Cited in this article [1]

[8]	RUMELHART D E, HINTON G E, WILLIAMS R J. Learning representations by back-propagating errors[J]. Nature, 1986, 323(6088):533-536. Cited in this article [1]

[9]	KRIZHEVSKY A, SUTSKEVER I, HINTON G E. Imagenet classification with deep convolutional neural networks[J]. Advances in neural information processing systems, 2012,25. Cited in this article [1]

[10]	GOODFELLOW I, POUGET-ABADIE J, MIRZA M, et al. Generative adversarial net[J]. Advances in neural information processing systems, 2014,27. Cited in this article [1]

[11]	VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[J]. Advances in neural information processing systems, 2017,30. Cited in this article [1]

[12]	DEVLIN J, CHANG M W, LEE K, et al. Bert:Pre-training of deep bidirectional transformers for language understanding[J]. arXiv preprint arXiv:1810.04805,2018. Cited in this article [1]

[13]	BROWN T, MANN B, RYDER N, et al. Language models are few-shot learners[J]. Advances in neural information processing systems, 2020,33:1877-1901. Cited in this article [1]

[14]	HE K, ZHANG X, REN S, et al. Delving deep into rectifiers:Surpassing human-level performance on imagenet classification[C]// Proceedings of the IEEE international conference on computer vision.2015:1026-1034. Cited in this article [1]

[15]	GIRSHICK R. Fast r-cnn[C]// Proceedings of the IEEE international conference on computer vision. 2015:1440-1448. Cited in this article [1]

[16]	REN S, HE K, GIRSHICK R, et al. Faster r-cnn: Towards real-time object detection with region proposal networks[J]. Advances in neural information processing systems, 2015,28. Cited in this article [1]

[17]	LIU W, ANGUELOV D, ERHAN D, et al. Ssd: Single shot multibox detector[C]// Computer Vision-ECCV 2016:14th European Conference, Amsterdam,The Netherlands, October 11-14, 2016,Proceedings,Part I 14.Springer International Publishing,2016:21-37. Cited in this article [1]

[18]	REDMON J, DIVVALA S, GIRSHICK R, et al. You only look once:Unified,real-time object detection[C]// Proceedings of the IEEE confe rence on computer vision and pattern recognition.2016:779-788. Cited in this article [1]

[19]	DETONE D, MALISIEWICZ T, RABINOVICH A. Superpoint:Self-supervised interest point detection and description[C]// Proceedings of the IEEE conference on computer vision and pattern recognition workshops. 2018: 224-236. Cited in this article [1]

[20]	SIMO-SERRA E, TRULLS E, FERRAZ L, et al. Discriminative learning of deep convolutional feature point descriptors[C]// Proceedings of the IEEE international conference on computer vision.2015:118-126. Cited in this article [1]

[21]	RADENOVIĆ F, TOLIAS G, CHUM O. Fine-tuning CNN image retrieval with no human annotation[J]. IEEE transactions on pattern analysis and machine intelligence,2018,41(7):1655-1668 Cited in this article [1]

[22]	REDMON J, FARHADI A. YOLO9000:better,faster,stronger[C]// Proceedings of the IEEE conference on computer vision and pattern recognition.2017:7263-7271. Cited in this article [1]

[23]	GUO Y, CHEN G, ZHAO P, et al. Gen-lanenet:A generalized and scalable approach for 3d lane detection[C]// Computer Vision-ECCV 2020: 16th European Conference, Glasgow,UK,August 23-28, 2020,Proceedings,Part XXI 16.Springer International Publishing,2020:666-681. Cited in this article [1]

[24]	ZHOU Y, WAN G, HOU S, et al. Da4ad: End-to-end deep attention-based visual localization for autonomous driving[C]// Computer Vision-ECCV 2020:16th European Conference, Glasgow,UK,August 23-28, 2020,Proceedings,Part XXVIII 16.Springer International Publishing,2020:271-289. Cited in this article [1]

[25]	PAN J, SUN H, XU K, et al. Lane-attention:Predicting vehicles’ moving trajectories by learning their attention over lanes[C]// 2018 IEEE International Conference on Systems, Man, and Cybernetics SMC.IEEE,2020:7949-7956. Cited in this article [1]

[26]	WANG J, CHI W, LI C, et al. Neural RRT*: Learning-based optimal path planning[J]. IEEE Transactions on Automation Science and Engineering, 2020, 17(4):1748-1758. Cited in this article [1]

[27]	LIU D, CHEN T, LIU S, et al. Pudiannao:A polyvalent machine learning accelerator[J]. ACM SIGARCH Computer Architecture News, 2015, 43(1):369-381. Cited in this article [4]

[28]	CHEN Y H, KRISHNA T, EMER J S, et al. Eyeriss:An energy-efficient reconfigurable accelerator for deep convolutional neural networks[J]. IEEE journal of solid-state circuits, 2016, 52(1):127-138. Cited in this article [3]

[29]	CHO H. Risa:A reinforced systolic array for depthwise convolutions and embedded tensor reshaping[J]. ACM Transactions on Embedded Computing Systems (TECS), 2021, 20(5s):1-20. Cited in this article [2]

[30]	SAMAJDAR A, PELLAUER M, KRISHNA T. Self-adaptive reconfigurable arrays (sara): Using ml to assist scaling gemm acceleration[J]. arXiv preprint arXiv:2101.04799,2021. Cited in this article [2]

[31]	TONG H, HAN K, HAN S, et al. Design of a Generic Dynamically Reconfigurable Convolutional Neural Network Accelerator with Optimal Balance[J]. Electronics, 2024, 13(4):761. Cited in this article [2]

[32]	GOKHALE V, JIN J, DUNDAR A, et al. A 240 g-ops/s mobile coprocessor for deep neural networks[C]// Proceedings of the IEEE conference on computer vision and pattern recognition workshops.2014:682-687. Cited in this article [3]

[33]

GENC

, KIM

, AMID

, et al. Gemmini:Enabling systematic deep-learning architecture evaluation via full-stack integration[C]// 2018 IEEE International Conference on Systems, Man, and Cybernetics SMC.IEEE 58th ACM/IEEE Design Automation Conference (DAC).IEEE,2021: 769-774.

Cited in this article [4]

[34]	NGUYEN X Q, PHAM-QUOC C. An FPGA-based convolution IP core for deep neural networks acceleration[J]. REV Journal onElectronics and Communications, 2022, 12(1-2). Cited in this article [5]

[35]	ZHANG H, LI Z, YANG H, et al. A High-efficient and Configurable Hardware Accelerator for Convolutional Neural Network[C]// 2018 IEEE International Conference on Systems, Man, and Cybernetics SMC.IEEE IEEE 14th International Conference on ASIC (ASICON).IEEE,2021: 1-4. Cited in this article [2]

[36]	PEEMEN M, SETIO A A A, MESMAN B, et al. Memory-centric accelerator design for convolutional neural networks[C]// 2018 IEEE International Conference on Systems, Man, and Cybernetics SMC.IEEE IEEE 31st international conference on computer design (ICCD).IEEE,2013: 13-19. Cited in this article [2]

[37]	DINELLI G, MEONI G, RAPUANO E, et al. MEM-OPT:A scheduling and data re-use system to optimize on-chip memory usage for CNNs on-board FPGAs[J]. IEEE Journal on Emerging and Selected Topics in Circuits and Systems, 2020, 10(3):335-347. Cited in this article [2]

[38]	CHEN T, DU Z, SUN N, et al. Diannao:A small-footprint high-throughput accelerator for ubiquitous machine learning[J]. ACM SIGARCH Computer Architecture News, 2014, 42(1):269-284. Cited in this article [4]

[39]	CHEN Y, LUO T, LIU S, et al. Dadiannao: A machine-learning supercomputer[C]// 2018 IEEE International Conference on Systems, Man, and Cybernetics SMC.IEEE 47th Annual IEEE/ACM International Symposium on Microarchitecture.IEEE,2014: 609-622. Cited in this article [1]

[40]	DU Z, FASTHUBER R, CHEN T, et al. ShiDianNao: Shifting vision processing closer to the sensor[C]// Proceedings of the 42nd annual international symposium on computer archit ecture.2015:92-104. Cited in this article [3]

[41]	SAMAJDAR A, JOSEPH J M, ZHU Y, et al. A systematic methodology for characterizing scalability of DNN accelerators using scalesim[C]// 2018 IEEE International Conference on Systems, Man, and Cybernetics SMC.IEEE,2020:58-68. Cited in this article [1]

[42]

SANKARADAS

, JAKKULA

, CADAMBI

, et al. A massively parallel coprocessor for convolutional neural networks[C]// 2018 IEEE International Conference on Systems, Man, and Cybernetics SMC.IEEE 20th IEEE International Conference on Application-specific Systems, Architectures and Processors.IEEE,2009: 53-60.

Cited in this article [1]

[43]	SRIRAM V, COX D, TSOI K H, et al. Towards an embedded biologically-inspired machine vision processor[C]// 2018 IEEE International Conference on Systems, Man, and Cybernetics SMC.IEEE,2010:273-278. Cited in this article [1]

[44]	ZHOU S, WU Y, NI Z, et al. Dorefa-net:Training low bitwidth convolutional neural networks with low bitwidth gradients[J]. arXiv preprint arXiv: 606.06160,2016. Cited in this article [2]

[45]	CHANG S E, LI Y, SUN M, et al. Mix and match:A novel FPGA-centric deep neural network quantization framework[C]// 2018 IEEE International Conference on Systems, Man, and Cybernetics SMC.IEEE,2021:208-220. Cited in this article [1]

[46]	COURBARIAUX M, BENGIO Y, DAVID J P. Binaryconnect:Training deep neural networks with binary weights during propagations[J]. Advances in neural information processing systems, 2015,28. Cited in this article [1]

[47]	RASTEGARI M, ORDONEZ V, REDMON J, et al. Xnor-net:Imagenet classification using binary convolutional neural networks[C]// European conference on computer vision. Cham: Springer International Publishing,2016:525-542.

[48]	LI F, LIU B, WANG X, et al. Ternary weight networks[J]. arXiv preprint arXiv:1605.04711,2016. Cited in this article [1]

[49]	HAN S, MAO H, DALLY W J. Deep compression: Compressing deep neural networks with pruning, trained quantization and huffman coding[J]. arXiv preprint arXiv:1510.00149,2015. Cited in this article [1]

[50]	ZHOU A, YAO A, GUO Y, et al. Incremental network quantization:Towards lossless cnns with low-precision weights[J]. arXiv preprint arXiv:1702.03044,2017. Cited in this article [1]

[51]	COURBARIAUX M, HUBARA I, SOUDRY D, et al. Binarized neural networks:Training deep neural networks with weights and activations constr ained to +1 or-1[J]. arXiv preprint arXiv:1602.02830,2016. Cited in this article [1]

[52]	HUBARA I, COURBARIAUX M, SOUDRY D, et al. Quantized neural networks:Training neural networks with low precision weights and activations[J]. Journal of Machine Learning Research, 2018, 18(187):1-30. Cited in this article [1]

[53]	LEE E H, MIYASHITA D, CHAI E, et al. Lognet: Energy-efficient neural networks using logarithmic computation[C]// 2018 IEEE International Conference on Systems, Man, and Cybernetics SMC.IEEE,2017:5900-5904. Cited in this article [1]

[54]	S HAN, J POOL, J TRAN, et al. Learning both weights and connections for efficient neural networks[C]// Proc. NIPS,2015:1135-1143. Cited in this article [2]

[55]	YANG T J, CHEN Y H, SZE V. Designing energy-efficient convolutional neural networks using energy-aware pruning[C]// Proc. CVPR, 2017. Cited in this article [1]

[56]

FARMAHINI-FARAHANI

, AHN

J H

, MORROW

, et al. NDA:Near-DRAM acceleration architecture leveraging commodity DRAM devices and standard memory modules[C]// 2018 IEEE International Conference on Systems, Man, and Cybernetics SMC.IEEE IEEE 21st International Symposium on High Performance Computer Architecture (HPCA).IEEE,2015: 283-295.

Cited in this article [1]

[57]	PUGSLEY S H, JESTES J, BALASUBRAMONIAN R, et al. Comparing implementations of near-data computing with in-memory mapreduce workloads[J]. IEEE Micro, 2014, 34(4):44-52. Cited in this article [1]

[58]	CHI P, LI S, XU C, et al. Prime:A novel processing-in-memory architecture for neural network computation in RERAM-based main memory[J]. ACM SIGARCH Computer Architecture News, 2016, 44(3):27-39. Cited in this article [1]

[59]	KIM D, KUNG J, CHAI S, et al. Neurocube:A programmable digital neuromorphic architecture with high-density 3D memory[J]. ACM SIGARCH Computer Architecture News, 2016, 44(3):380-392. Cited in this article [3]

[60]	AGA S, JAYASENA N, IGNATOWSKI M. Co-ML:a case for Collaborative ML acceleration using near-data processing[C]// Proceedings of the International Symposium on Memory Systems.2019:506-517. Cited in this article [1]

[61]	SHAFIEE A, NAG A, MURALIMANOHAR N, et al. ISAAC:A convolutional neural network accelerator with in-situ analog arithmetic in crossbars[J]. ACM SIGARCH Computer Architecture News, 2016, 44(3):14-26. Cited in this article [3]

[62]	SONG L, QIAN X, LI H, et al. Pipelayer:A pipelined reram-based accelerator for deep learning[C]// 2018 IEEE International Conference on Systems, Man, and Cybernetics SMC.IEEE,2017:541-552. Cited in this article [3]

[63]	SHIN D, LEE J, LEE J, et al. An energy-efficient deep learning processor with heterogeneous multi-core architecture for convolutional neural networks and recurrent neural networks[C]// 2018 IEEE International Conference on Systems, Man, and Cybernetics SMC.IEEE,2017:1-2. Cited in this article [2]