A 0.45-mm² 3.49-TOPS/W Cryogenic Deep Reinforcement Learning Module for End-to-End Integrated Circuits Control

This work presents a fully unrolled on-chip deep reinforcement learning (DRL) module with a deep Q-network (DQN) and its system integration for integrated circuits control and functionality augmentation tasks, including voltage regulation of a cryogenic single-input triple-output dc&#x2013;dc converter and recovery of RF fingerprints (RFFs) using a reconfigurable power amplifier (PA) under temperature variations. The complete DRL module features 6-bit fixed-point model parameters, 116 kB of memory, and 128 processing elements. It is equipped with on-chip training capabilities, fully unrolled on a 0.45-<inline-formula> <tex-math notation="LaTeX">${\mathrm { mm}}^{2}$ </tex-math></inline-formula> core area using 28-nm technology. The design achieves an efficiency of 0.12 nJ per action and a control latency of <inline-formula> <tex-math notation="LaTeX">$4.925~\mu $ </tex-math></inline-formula>s, with a maximum operational efficiency of 3.49 TOPS/W. Temperature effects on the chip are thoroughly demonstrated across a wide temperature range from 358 K (<inline-formula> <tex-math notation="LaTeX">$85~^{\circ }$ </tex-math></inline-formula>C) to 4.2 K (&#x2013;<inline-formula> <tex-math notation="LaTeX">$269~^{\circ }$ </tex-math></inline-formula>C).