This technology is a neuromorphic hardware implementation method that represents and decomposes the synaptic weights of a trained artificial neural network as tensors, compresses the weights into a sum-of-products form using low-rank approximation, and retrains the restructured neural network. The approximated factors are stored in neuromorphic circuits for use in learning and inference.
Large-scale artificial neural networks require storing millions to billions of synaptic weights, leading to significant energy consumption and bottlenecks due to data movement between memory and processing units in von Neumann architectures. Even when implemented with neuromorphic circuits, each n x m weight matrix requires nm constant storage spaces and nm multiplications, causing a substantial increase in memory resource usage and circuit power consumption.
This technology recursively decomposes weight tensors using a divide-and-conquer singular value decomposition and approximates them to a rank lower than the original, thereby reducing both the number of weights to store and the number of multiplications required. By repeating retraining after approximation, accuracy loss is mitigated, and the network can be converted into a time-delay neural network with reduced synapses for implementation in cell-based circuits. It can be applied to AI accelerator chips and neural processors for edge devices, allowing pre-trained deep neural networks to be deployed with smaller chip area and lower memory requirements.
This invention was developed with support from the Ministry of Education's research on general-purpose embedded neuromorphic processors for the Internet of Things.
N/A
