Memristors- the key to reduce the carbon footprints left due to the usage of AI

In a new study published in Nature Communications, engineers at University College London found that accuracy might be greatly improved by getting memristors to figure together in several sub-groups of neural networks and averaging their calculations, meaning that flaws in each of the networks might be cancelled out. In the study, Dr Adnan Mehonic, PhD student Dovydas Joksas (both UCL Electronic & Electrical Engineering), and colleagues from the United Kingdom and the US tested the new approach in several differing types of memristors and located that it improved the accuracy of all of them, no matter material or particular memristor technology. It also worked for several different problems which will affect memristors' accuracy.

The system, which uses memristors to make artificial neural networks, is a minimum of 1,000 times more energy-efficient than conventional transistor-based AI hardware but has so far been more susceptible to error. Existing AI is extremely energy-intensive - training one AI model can generate 284 tonnes of CO2, equivalent to the lifetime emissions of 5 cars. Replacing the transistors that structure all digital devices with memristors, a completely unique device first built in 2008, could reduce this to a fraction of a tonne of CO2 - like emissions generated in an afternoon's drive.

Since memristors are supposed to be a lot more energy-efficient than existing computing systems, they will potentially pack huge amounts of computing power into hand-held devices, removing the necessity to be connected to the web. This is especially important as over-reliance on the web is predicted to become problematic in the future thanks to ever-increasing data demands and therefore the difficulties of accelerating data transmission capacity past a particular point.

“A memristor is a non-linear two-terminal electrical component relating electric charge and magnetic flux linkage. It comprises a circuit, of multiple conventional components, which mimics key properties of the ideal memristor component and is also commonly referred to as a memristor.”  [Credit: Wikipedia]

“A memristor is a non-linear two-terminal electrical component relating electric charge and magnetic flux linkage. It comprises a circuit, of multiple conventional components, which mimics key properties of the ideal memristor component and is also commonly referred to as a memristor.” [Credit: Wikipedia]

Artificial neural networks are notoriously power- and time-consuming when implemented on conventional von Neumann computing systems. Consequently, recent years have seen an emergence of research in machine learning hardware that strives to bring memory and computing closer together. a preferred approach is to understand artificial neural networks in hardware by implementing their synaptic weights using memristive devices. However, various device- and system-level non-idealities usually prevent these physical implementations from achieving high inference accuracy. The researchers lead by some scientists from University College London proposed applying a well-known concept in computer science—committee machines—in the context of memristor-based neural networks. Using simulations and experimental data from three different types of memristive devices, they show that committee machines employing ensemble averaging can successfully increase inference accuracy in physically implemented neural networks that suffer from faulty devices, device-to-device variability, random telegraph noise and line resistance. Importantly, they also demonstrate that accuracy can often be improved even without increasing the overall number of memristors.

Memristors, described as "resistors with a memory" for their basic property of remembering the quantity of electrical charge that flowed through them even after being turned off, were considered revolutionary once they were first built over a decade ago, a "missing link" in electronics to supplement the resistor, capacitor and inductor. they need since been manufactured commercially in memory devices, but the research team say they might be wont to develop AI systems within the subsequent three years. Researchers found that their approach increased the accuracy of the neural networks for typical AI tasks to a comparable level to software tools run on conventional digital hardware.

Dr Mehonic, director of the study, said: "We hoped that there could be more generic approaches that improve not the device-level, but the system-level behaviour and that we believe we found one. Our approach shows that, when it involves memristors, several heads are better than one. Arranging the neural network into several smaller networks instead of one big network led to greater accuracy overall." Dovydas Joksas further explained: "We borrowed a well-liked technique from computing and applied it within the context of memristors. And it worked! Using preliminary simulations, we found that even simple averaging could significantly increase the accuracy of memristive neural networks."

Professor Tony Kenyon (UCL Electronic & Electrical Engineering), a co-author on the study, added: "We believe now's the time for memristors, on which we've been working for several years, to require a number one role during a more energy-sustainable era of IoT devices and edge computing."

Thumbnail Credit: news.mit.edu

Reference: D. Joksas, P. Freitas, Z. Chai, W. H. Ng, M. Buckwell, C. Li, W. D. Zhang, Q. Xia, A. J. Kenyon, A. Mehonic. “Committee machines -- a universal method to deal with non-idealities in memristor-based neural networks.” Nature Communications, 2020; 11 (1) DOI: 10.1038/s41467-020-18098-0