Towards implementing neural networks on edge IoT devices

However, bringing AI capabilities to IoT edge devices presents a significant challenge. Artificial neural networks (ANNs) -- one of the most important AI technologies -- require substantial computational resources. Meanwhile, IoT edge devices are inherently small, with limited power, processing speed, and circuit space. Developing ANNs that can efficiently learn, deploy, and operate on edge devices is a major hurdle.

In response, Professor Takayuki Kawahara and Mr. Yuya Fujiwara from the Tokyo University of Science, are working hard towards finding elegant solutions to this challenge. In their latest study published in IEEE Access on October 08, 2024, they introduced a novel training algorithm for a special type of ANN called binarized neural network (BNN), as well as an innovative implementation of this algorithm in a cutting-edge computing-in-memory (CiM) architecture suitable for IoT devices.

"BNNs are ANNs that employ weights and activation values of only -1 and +1, and they can minimize the computing resources required by the network by reducing the smallest unit of information to just one bit," explains Kawahara, "However, although weights and activation values can be stored in a single bit during inference, weights and gradients are real numbers during learning, and most calculations performed during learning are real number calculations as well. For this reason, it has been difficult to provide learning capabilities to BNNs on the IoT edge side."

To overcome this, the researchers developed a new training algorithm called ternarized gradient BNN (TGBNN), featuring three key innovations. First, it employs ternary gradients during training, while keeping weights and activations binary. Second, they enhanced the Straight Through Estimator (STE), improving the control of gradient backpropagation to ensure efficient learning. Third, they adopted a probabilistic approach for updating parameters by leveraging the behavior of MRAM cells.

Afterwards, the research team implemented this novel TGBNN algorithm in a CiM architecture -- a modern design paradigm where calculations are performed directly in memory, rather than in a dedicated processor, to save circuit space and power. To realize this, they developed a completely new XNOR logic gate as the building block for a Magnetic Random Access Memory (MRAM) array. This gate uses a magnetic tunnel junction to store information in its magnetization state.

To change the stored value of an individual MRAM cell, the researchers leveraged two different mechanisms. The first was spin-orbit torque -- the force that occurs when an electron spin current is injected into a material. The second was voltage-controlled magnetic anisotropy, which refers to the manipulation of the energy barrier that exists between different magnetic states in a material. Thanks to these methods, the size of the product-of-sum calculation circuit was reduced to half of that of conventional units.

The team tested the performance of their proposed MRAM-based CiM system for BNNs using the MNIST handwriting dataset, which contains images of individual handwritten digits that ANNs have to recognize. "The results showed that our ternarized gradient BNN achieved an accuracy of over 88% using Error-Correcting Output Codes (ECOC)-based learning, while matching the accuracy of regular BNNs with the same structure and achieving faster convergence during training," notes Kawahara. "We believe our design will enable efficient BNNs on edge devices, preserving their ability to learn and adapt."

This breakthrough could pave the way to powerful IoT devices capable of leveraging AI to a greater extent. This has notable implications for many rapidly developing fields. For example, wearable health monitoring devices could become more efficient, smaller, and reliable without requiring cloud connectivity at all times to function. Similarly, smart houses would be able to perform more complex tasks and operate in a more responsive way. Across these and all other possible use cases, the proposed design could also reduce energy consumption, thus contributing to sustainability goals.