MIT Researchers Propose ‘Digital Optical Neural Network (DONN)’: An Energy-Efficient Optical Accelerator for Deep-Learning Inference

724
Source: https://www.nature.com/articles/s41598-021-82543-3.pdf

MIT researchers have developed a novel optical DNN (artificial deep neural network) accelerator that uses light to transmit activation and weight data. They called it DONN (digital optical neural network). With only a few percentage points accuracy cost, this system can achieve a transmission energy advantage up 1000x over traditional electronic devices.

The research was published ‘Freely scalable and reconfigurable optical hardware for deep learning’ in Nature’s Scientific Reports. DONN tackles the problem of power consumption in neural networks by replacing electric currents with optical signals. DONN’s constant energy usage has enabled it to scale up for larger deep learning models while keeping costs low and performance high – a single 8-bit MAC operation only requires 3 femtojoules (fJ) compared to more than 1,000 fJ needed on an electronic chip.

Large deep-learning models require correspondingly large compute and memory resources for both training and inference; often, the model also involves accelerator hardware such as GPUs or TPUs to perform computations promptly. These hardware resources consume lots of energy while running, so chip designers are constantly looking at new ways to make these machines more efficient by keeping memory close to computation elements.

Optical data transmission is a cheaper and more efficient option for long-distance data transmissions. While optical computing, especially for digital signal processing, has been an active area of research for decades, the development of photonic integrated circuits is spurring intense interest in its application to deep learning. Much work focuses on implementing linear algebra functions with optics used by neural networks that can be applied using light instead of electricity.

https://www.nature.com/articles/s41598-021-82543-3.pdf

In this new approach, neural networks no longer need analog computation. This allows for less noise and, therefore, more accurate outputs. DONN can avoid the restrictive physical limitations of digital computing and instead uses optical pathways that “fan-out” neural network activation values and weights. The researchers found that although this new tool does not have the noise sources of an analog method, there are still sources for bit error in optical transmission. The team then conducted experiments to determine the errors’ rate and how they affected the accuracy of an MNIST image classification task. They determined that some types of errors can be mitigated with correction schemes while others only affect accuracy by less than three percentage points.

Paper: https://www.nature.com/articles/s41598-021-82543-3.pdf

Github: https://github.com/alexsludds/Digital-Optical-Neural-Network-Code

Reference Source : https://www.infoq.com/news/2021/08/mit-optical-deep-learning/