PDF(35936 KB)
An irregular memory access prefetching framework based on enhanced reinforcement learning
Yang Yuze, Huang Zhengwei, Wang Yongwen
Integrated Circuits and Embedded Systems ›› 2026, Vol. 26 ›› Issue (8) : 1-18.
PDF(35936 KB)
PDF(35936 KB)
An irregular memory access prefetching framework based on enhanced reinforcement learning
Irregular data access patterns in high-performance computing and intelligent computing often render traditional data prefetching techniques ineffective. Existing models that rely on fixed rules or offline learning based on specific program contexts also struggle to adapt to dynamically changing memory access patterns during runtime. While the Pythia reinforcement learning (RL) prefetching framework demonstrates adaptability through online learning, it still requires manual tuning under extreme irregular workloads, limiting its generalization in practical applications. This paper proposes IEP (Irregular Enhanced Pythia), a context-aware reinforcement learning prefetching framework to enhance the prediction capability for irregular memory access patterns. The framework introduces two key innovations. The first is an irregular feature enhancement module. It incorporates address bit masks and access sequence distance as state features to capture hidden spatiotemporal patterns in memory allocator behavior, thereby improving the representation of irregular memory accesses. The second is a hierarchical reward strategy module, which employs a dynamic reward mechanism combining confidence awareness and bandwidth sensitivity. This mechanism finely guides the agent’s learning process, accelerating policy optimization and improving final performance. Experiments were conducted using the ChampSim simulator, to test various irregular workloads. Results show that compared to the Pythia framework, the proposed solution achieves a maximum improvement of 2.27% in average prefetching accuracy and 2.90% in average single-core IPC for typical irregular workloads such as Ligra and PARSEC, while maintaining stable performance in multi-core environments.
irregular memory access / hardware prefetching / reinforcement learning / ChampSim / state features / dynamic reward mechanism
| [1] |
|
| [2] |
|
| [3] |
|
| [4] |
宗鹏陈, 曲劭儒, 赵文哲, 等. 基于开源处理器的间接访问数据预取器设计[J]. 集成电路与嵌入式系统, 2026(1):47-53.DOI:10.20193/j.ices2097-4191.2025.0082.
|
| [5] |
王锦涵, 李俊, 路冬冬, 等. 基于双倍步长数据流的硬件预取机制[J]. 计算机工程, 2019, 45(6):115-118,126.
硬件数据预取技术可以有效提升处理器的访存性能,但传统流预取策略存在预取不及时的问题。为此,提出一种双倍步长流预取策略,并设计对应的预取部件结构。预取部件自动检测数据流的固定步长并将该步长扩大为原有的2倍,以计算预取地址。实验结果表明,加入该预取部件后,运行SPEC2006测试集的整数应用与浮点应用时,处理器性能最高可分别提升45%与57%,针对Cache Miss率较高的应用,该预取部件可以有效隐藏访存延时。
|
| [6] |
|
| [7] |
|
| [8] |
|
| [9] |
|
| [10] |
|
| [11] |
|
| [12] |
王玉庆, 杨秋松, 李明树. 基于指令流混合模式学习的缓存预取算法[J]. 电子学报, 2023, 51(2):342-354.
近期缓存预取算法的研究热点是使用基于模式识别的预测技术,例如Lookahead,推算访存请求的地址.此类算法一方面很难学习访存行为中的依赖缓存失效,另一方面不能精确控制预取请求发送和写回的时机.为了解决上述问题,本文提出了一种基于分支预测技术和混合模式学习的缓存预取(Instruction Flow Based Hybrid Prediction,IFBHP)算法.使用分支预测技术识别程序未来指令流中的访存指令流,通过多种地址关联模式的学习逐一计算访存指令流中每条指令的地址,写入访存地址队列.使用阈值评估未来指令流进入处理器主流水线的时刻,精确控制指令流所对应的预取请求的发送和写回.实验表明,本文算法相比STeMS(Spatio-Temporal Memory Streaming)算法、ISB++(Irregular Stream Buffer++)算法、SANGAM算法、IPCP(Instruction Pointer Classifier based spatial Prefetching)算法一级数据的读操作缓存失效次数分别平均减少31.58%,28.85%,17.85%,11.48%;本文算法相比STeMS算法、ISB++算法、SANGAM算法、IPCP算法一级数据的写操作缓存失效次数分别平均减少31.58%,28.85%,17.85%,11.48%.
|
| [13] |
|
| [14] |
|
| [15] |
|
| [16] |
|
| [17] |
|
| [18] |
|
| [19] |
|
| [20] |
|
| [21] |
|
| [22] |
|
| [23] |
SPEC CPU 2006[EB/OL]. [2025-10-12]. https://www.spec.org/cpu2006/.
|
| [24] |
SPEC CPU 2017[EB/OL]. [2025-10-12]. https://www.spec.org/cpu2017/.
|
| [25] |
PARSEC[EB/OL].[2025-10-12]. http://parsec.cs.princeton.edu/.
|
| [26] |
|
| [27] |
|
| [28] |
|
/
| 〈 |
|
〉 |