Turbocharging the energy efficiency of AI processors

July 2024 | Highlights Community

Follow CDE

PDF Download

Image above was created with the assistance of DALL·E 2

Breakthroughs in chip design techniques offer three crucial benefits for AI devices: reduced power consumption, extended battery life and the ability to support intense computational workloads.

From the alarm on a smartphone that starts the day, to a home security system keeping a family safe, to the electronic control unit controlling the critical sub-systems of a vehicle, and the desktop computer driving office productivity, many facets of everyday life are interconnected by one crucial component: microchips, the brain cells of the digital age.

Despite the ubiquity of silicon chips, there is no ‘one-chip-fits-all’ solution. Consider smaller, low-power devices, such as those making up the expanding Internet-of-Things (IoT) universe. Energy-efficient chips are a mainstay for these applications. A smartwatch that can’t last a day without charging is a smartwatch won’t find a market. Sensors that fail to operate continuously would be a catalyst for catastrophe in modern factory settings.

This has placed energy efficiency at the front of mind for artificial intelligence (AI) chip manufacturers. At the College of Design and Engineering, National University of Singapore, Professor Massimo Alioto and his research team from the Department of Electrical and Computer Engineering have innovated a new class of silicon-based systems that gives the energy efficiency of AI-connected devices a significant boost. Developed in collaboration with industry partners Soitec and NXP Semiconductors, this technology has the potential to revolutionise the design of advanced semiconductor components in fully depleted silicon-on-insulator (FD-SOI).

Professor Massimo Alioto led a team to enhance the energy efficiency of chips that power AI devices.

“This new technology enables AI at the edge — where AI models and algorithms can be deployed directly on small devices such as sensors or IoT gadgets used in myriad applications, from smart buildings and Industry 4.0 to wearables and smart logistics,” says Prof Alioto, who is also the Director of the FD-fAbrICS (FD-SOI Always-on Intelligent & Connected Systems) joint lab, where the invention transitioned from concept to reality. “This localisation means real-time data processing and analysis with much-reduced burden on network and cloud infrastructure to support the exponential growth of intelligent and connected devices.”

Energy efficiency is top of mind

IoT devices are increasingly infused with AI and machine learning capabilities to bring intelligence and autonomy to various systems and processes, such as autonomous driving, industrial smart manufacturing, medical equipment and home automation. Most of these devices are small, power- and cost-constrained microcontroller-based systems. On top of that, network bandwidth and consumer expectations around data privacy and user experience continue to drive more on-device processing — where data is processed directly on IoT devices rather than in the cloud.

“IoT devices often operate on a very limited power budget — requiring low average power to perform regular tasks such as physical signal monitoring in sensors,” says Prof Alioto. “Concurrently, they also need high peak performance to process occasional signal events using computationally intensive AI algorithms.”

The new tech developed by the NUS researchers kills three birds with one chip. Not only does it support serious number-crunching in such applications, but it also helps smart wearables save battery juice — by tenfold — and slashes the power consumption associated with wireless communications with the cloud by half, thanks to the key contribution of Associate Professor Heng Chun Huat’s group on energy-efficient radios within the FD-fAbrICS joint lab.

"Our research outcomes amplify such power reductions by an order of magnitude through fundamental advances in circuits and architectures."

Unlike their conventional counterparts, chips based on FD-SOI technology offer superior power efficiency as current leakage is less of an issue. As a result, they consume less power and operate at lower voltages — making them a cornerstone for power-sensitive applications such as wearables and various IoT devices.

“Additionally, our research outcomes amplify such power reductions by an order of magnitude through fundamental advances in circuits and architectures. At the same time, such advances coexist with the traditional flexibility of software-programmable processors, filling the traditional energy-flexibility gap between dedicated accelerators and multi-core systems,” adds Prof Alioto.

"Our research outcomes amplify such power reductions by an order of magnitude through fundamental advances in circuits and architectures."

Lowering the barriers to entry

Prof Alioto aims to advance the FD-SOI systems and technologies from the laboratory to industry by lowering design barriers. A workshop organised by Prof Alioto and the FD-SOI & IoT Industry Consortium* on 3 May 2024 provided a platform for those from the research community and industry to talk all things FD-SOI — from AI models and wireless communications to digital circuits and system architectures — facilitating knowledge exchange and opening new doors for further collaboration with industry.

“This innovation has the potential to accelerate the time to market for the key players in Singapore’s semiconductor ecosystem,” adds Prof Alioto. “Through the FD-SOI & IoT Industry Consortium, we intend to drive the adoption and large-scale deployment of our design technologies, giving the nation an edge in the competitive AI and semiconductor arenas.”

At NUS FD-fAbrICS, researchers are now expanding the scope of their work to develop new classes of connected silicon systems that could potentially support larger AI model sizes for AI applications.

* The FD-SOI & IoT Industry Consortium was established in 2023 to extend the impact of the NUS FD-fAbrICS joint lab on Singapore’s semiconductor ecosystem, with Soitec and NXP Semiconductors as founding members.