What we do
The Brain-inspired AI for Music project is aimed at learning, understanding, composing music by Brain-inspired Artificial Intelligence. The scientific contribution is to create biological realistic brain-inspired spiking neural network models for music learning and composition, with inspirations of how the human brain enables us to learn, enjoy, and create musical artistic work.
Memory and Learning of Melodies
We create a spiking neural network model of music memory based on brain mechanisms is proposed and used for melodies learning and memory. Four neural clusters, namely, a goal cluster, pacemaker cluster, and spatial and temporal subnetworks, are involved in encoding, storage, and retrieval sequences. Minicolumns with different preferences encode identical contents and time lengths of sequences. The connection architecture, which considers not only ordered information but also sequential context, can store a large number of sequences. An STDP learning rule is adopted to update connection weights during the memorizing process. The model can memorize a large number of musical melodies. Because of the associative property of the network, both goal-based and contextual retrieval give highly accurate results. The melody can be retrieved at different speeds by tuning the frequency of the pacemaker population, and this process makes the model work like human behaviors.
Stylistic Composition of Melodies
We introduce a brain-inspired spiking neural network to learn and create the musical melodies with different styles. Based on related brain mechanisms, we build two subsystem, the knowledge and memory subsystems, to achieve our goals. A hierarchical structure is utilized to learn and store the basic information of a musical piece. The genre, composer and title cluster encode and memorize the corresponding information of a piece. Besides, interneurons are involved in this system to perform the composition task. Sequential memory system encodes and stores the ordered musical notes. All the neurons are simulated by the Izhikevich model, both regular and fast spiking patterns are used in our model. During the learning process, synapses between neurons are updated by the STDP learning rule. Genre-based and composer-based melody composition can be achieved depending on the different circuits, different neural clusters are activated in these tasks. Our model can generate melodies with different styles of genres and composers. Some of them sound nice and have strong characteristics.
Melodies automatically composed by Brain-inspired Spiking Neural Networks
Here we release our first try on stylistic composition of melodies. Currently, the release contains compositions of main melody with different styles. Stay tuned for many more to come.
Project Team
- Principal Investigator: Yi Zeng
- Core Contributors: Qian Liang, Zizhe Ruan
Project Publications