基于强化学习的阈值电压分配漏功耗优化方法研究

张展华; 王家豪; 丁文杰; 曹鹏

doi:10.20193/j.ices2097-4191.2024.02.007

PDF(1075 KB)

集成电路与嵌入式系统 ›› 2024, Vol. 24 ›› Issue (2) : 57-63. DOI: 10.20193/j.ices2097-4191.2024.02.007

研究论文

基于强化学习的阈值电压分配漏功耗优化方法研究

作者信息 +

Research on reinforcement learning based leakage power optimization framework (RL-LPO)

Author information +

文章历史 +

摘要

随着先进工艺的演进,泄漏功耗在集成电路总功耗中的占比不断增大,已逐渐成为制约电路功耗降低的重要因素之一。在已有的漏功耗优化方法中,基于阈值电压分配的方法具有指数关系的功耗优化效果,并且对已进行的布局布线不产生影响,因而被广泛采用。然而,在商用签核工具中,为了保持伪线性复杂度而限制了底层算法所做的全局搜索,使得难以获得最优结果。本文提出一种基于图神经网络和强化学习的联合优化框架RL-LPO,实现高效的门单元阈值电压分配。在RL-LPO中,采用图神经网络GraphSAGE编码电路的时序和物理信息对目标单元及其局部邻域信息进行聚合;采用深度确定性策略梯度(Deep Deterministic Policy Gradient,DDPG)强化学习算法,在奖励函数的指导下,考虑漏功耗和时序变化进行阈值电压的分配。本文提出的门单元阈值电压分配框架RL-LPO在28 nm工艺下由IWLS2005和Opencores基准电路进行验证,与商用签核工具相比,在不增加时序违例的前提下,RL-LPO降低了至少2.1%的额外漏功耗,并实现了至少4.2倍的加速。

Abstract

With the evolution of advanced technology,the proportion of leakage power consumption in the total power consumption of integrated circuits continues to increase,which has gradually become one of the important factors restricting the reduction of circuit power consumption.Among the existing leakage power optimization methods,the method based on threshold voltage allocation has an exponential power optimization effect and has no influence on the layout and routing,so it is widely adopted.However,in commercial signoff tools,in order to maintain pseudolinear complexity,the global search made by the underlying algorithm is limited,which makes it difficult to obtain optimal results.In this paper,a joint optimization framework RL-LPO based on graph neural network and reinforcement learning is proposed to achieve efficient gate unit threshold voltage distribution.In RL-LPO,the timing and physical information of the graph neural network GraphSAGE encoding circuit are used to aggregate the target unit and its local neighborhood information.Using the Deep Deterministic Policy Gradient (DDPG) reinforcement learning algorithm,the threshold voltage allocation is carried out considering the leakage power consumption and timing variation under the guidance of the reward function.The gate unit threshold voltage distribution framework RL-LPO proposed in this paper is verified by IWLS2005 and Opencores reference circuits under the 28 nm process,and compared with commercial signoff tools,RL-LPO reduces the additional leakage power consumption by at least 2.1% and achieves at least 4.2 times acceleration without adding timing violations.

导出引用

张展华, 王家豪, 丁文杰, 等. 基于强化学习的阈值电压分配漏功耗优化方法研究[J]. 集成电路与嵌入式系统. 2024, 24(2): 57-63 https://doi.org/10.20193/j.ices2097-4191.2024.02.007

ZHANG Zhanhua, WANG Jiahao, DING Wenjie, et al. Research on reinforcement learning based leakage power optimization framework (RL-LPO)[J]. Integrated Circuits and Embedded Systems. 2024, 24(2): 57-63 https://doi.org/10.20193/j.ices2097-4191.2024.02.007

中图分类号： TN47 (大规模集成电路、超大规模集成电路)

参考文献

列表( 原文顺序 | 文献年度倒序 | 文中引用次数倒序 ) 可视化分析

[1]	A T El-THAKEB, et al. Performance evaluation of FinFET based SRAM under statistical VT variability[C]//in Proc. Int. Conf. on Microelectronics, Dec. 2014:88-91. 本文引用 [1]

[2]

PATANJALI

, M

PATNAIK

, S

POTLURI

, et al. Mltimer: Leakage power minimization in digital circuits using machine learning and adaptive lazy timing analysis. Journal of Low Power Electronics, 2018, 14(2):285-301.

https://doi.org/10.1166/jolpe.2018.1549

http://www.ingentaconnect.com/content/10.1166/jolpe.2018.1549

本文引用 [1]

[3]	LU Y CH, et al. Rl-sizer:Vlsi gate sizing for timing optimization using deep reinforcement learning[C]//2021 58th ACM/IEEE Design Automation Conference (DAC). IEEE, 2021. 本文引用 [3]

[4]	NING WING. Strongly NP-hard discrete gate-sizing problems[J]. IEEE transactions on computer-aided design of integrated circuits and systems, 1994, 13(8):1045-1051. https://doi.org/10.1109/43.298040 http://ieeexplore.ieee.org/document/298040/ 本文引用 [1]

[5]	LIU YIFANG, JIANG HU. A new algorithm for simultaneous gate sizing and threshold voltage assignment[C]// Proceedings of the 2009 international symposium on Physical design, 2009. 本文引用 [1]

[6]	ROY S, LIU D, UM J, et al. OSFA: A new paradigm of gate-sizing for power/performance optimizations under multIPle operating conditions[C]// Design Automation Conference.ACM, 2015. 本文引用 [1]

[7]	KEUTZER D G C K. Linear programming for sizing, Vth and Vdd assignment[J]. Proc.International Symp.on Low Power Electronics & Design, 2005:149-154. 本文引用 [1]

[8]	S BAO. Optimizing leakage power using machine learning[D].CS Department,Stanford University, 2010. 本文引用 [1]

[9]	LU YICHEN, et al. A fast learning-driven signoff power optimization framework[C]// Proceedings of the 39th International Conference on Computer-Aided Design, 2020. 本文引用 [11]

[10]	A MIRHOSEINI, A GOLDIE, M YAZGAN, et al. Chip placement with deep reinforcement learning.arXiv preprint arXiv:2004.10746,2020. 本文引用 [1]

[11]	LIAO H, ZHANG W, DONG X, et al. A Deep Reinforcement Learning Approach for Global Routing[J]. Journal of Mechanical Design, 2019, 142(6):1-17. 本文引用 [1]

[12]	G PASANDI, S NAZARIAN, M PEDRAM. Approximate logic synthesis:A reinforcement learning-based technology mapping approach[C]// In 20th International Symposium on Quality Electronic Design (ISQED),IEEE, 2019:26-32. 本文引用 [1]

[13]	W HAMILTON, Z YING, J LESKOVEC. Inductive representation learning on large graphs. In Advances in neural information processing systems, pages 1024-1034, 2017. 本文引用 [3]

[14]	HASSELT H V, GUEZ A, SILVER D. Deep Reinforcement Learning with Double Q-learning[J]. Computer ence, 2015.DOI:10.48550/arXiv.1509.06461. 本文引用 [2]