From "Experiments in Musical Intelligence" (EMI or Emmy) by David Cope. He writes: "I began Experiments in Musical Intelligence in 1981 as the result of a composer's block. My initial idea involved creating a computer program which would have a sense of my overall musical style and the ability to track the ideas of a current work such that at any given point I could request a next note, next measure, next ten measures, and so on. My hope was that this new music would not just be interesting but relevant to my style and to my current work. Having very little information about my style, however, I began creating computer programs which composed complete works in the styles of various classical composers, about which I felt I knew something more concrete."

This file was downloaded from http://artsites.ucsc.edu/faculty/cope/mp3page.htm which linked to ftp://arts.ucsc.edu/pub/cope/chopin.mp3

Mazurka (After Chopin) by David Cope (1994?)

"These compositions were made by an LSTM recurrent neural network. After learning by example from the training set (next-step prediction), the network generated these musical examples. When the network composed, it worked with no guidance whatsoever. That is, the network had to reproduce both the chords and the melodies. For the simple training set used here, the chord structure was fixed. This meant that the network did not need to generalize chords. Even in this simple case, however, neither a feed-forward network nor very likely a traditional recurrent neural network (RNN) can learn these chords. See the technical report for more.

This work marks, I believe, a first step towards a neural network music composer that can learn and use global musical structure. Previous attempts at this task showed that RNNs can capture local structure in music but fail to capture the long-term structure that defines a musical form. These compositions show that LSTM can capture and reproduce long-term musical structure and can generate new (sometimes somewhat pleasing) examples of a form. In short, though the chord structure was fixed and the training set somewhat dull, these results are promising enough to warrant further research.

[...]

This fourth example shows an example of a network trained longer than those used for examples 1 through 3. Here, due to longer training time the network does a better job of reproducing the training set. It still improvises freely but with less departure from the target melodies."

Explanation from http://www.iro.umontreal.ca/~eckdoug/blues/

This version is trimmed to the first 90 seconds. The original is 5:47 http://www.iro.umontreal.ca/~eckdoug/blues/lstm_0224_2200.mp3

Photo from http://prometheus.med.utah.edu/~bwjones/2016/07/scifoo-2016/

LSTM Blues Improvisation by Doug Eck (2002)

"We present Performance RNN, an LSTM-based recurrent neural network designed to model polyphonic music with expressive timing and dynamics. Here’s an example generated by the model: [...] Note that this isn’t a performance of an existing piece; the model is also choosing the notes to play, “composing” a performance directly. The performances generated by the model lack the overall coherence that one might expect from a piano composition; in musical jargon, it might sound like the model is “noodling”— playing without a long-term structure. However, to our ears, the local characteristics of the performance (i.e. the phrasing within a one or two second time window) are quite expressive."

From the Magenta team at Google Brain. Original post here https://magenta.tensorflow.org/performance-rnn

This file was originally at https://magenta.tensorflow.org/assets/performance_rnn/good.mp3

Performance RNN by Magenta (2017)

"For a while now, I’ve been floating around vague ideas about writing a program to compose music. My original idea was based on a fractal decomposition of time and some sort of repetition mechanism, but after reading more about neural networks, I decided that they would be a better fit. So a few weeks ago, I got to work designing my network. And after training for a while, I am happy to report remarkable success!"

Description and graphic from http://www.hexahedria.com/2015/08/03/composing-music-with-recurrent-neural-networks/

This version is trimmed to the first 90 seconds. The original audio is 4:02. http://www.hexahedria.com/files/nnet_music_2.mp3

Composing Music With RNNs by Daniel Johnson (2015)

"Since WaveNets can be used to model any audio signal, we thought it would also be fun to try to generate music. Unlike the TTS experiments, we didn’t condition the networks on an input sequence telling it what to play (such as a musical score); instead, we simply let it generate whatever it wanted to. When we trained it on a dataset of classical piano music, it produced fascinating samples like the ones below"

Description and graphic from https://deepmind.com/blog/wavenet-generative-model-raw-audio/

Samples are combined from 6 URLs in order:
https://storage.googleapis.com/deepmind-media/pixie/making-music/sample_1.wav
https://storage.googleapis.com/deepmind-media/pixie/making-music/sample_2.wav
https://storage.googleapis.com/deepmind-media/pixie/making-music/sample_3.wav
https://storage.googleapis.com/deepmind-media/pixie/making-music/sample_4.wav
https://storage.googleapis.com/deepmind-media/pixie/making-music/sample_5.wav
https://storage.googleapis.com/deepmind-media/pixie/making-music/sample_6.wav

WaveNet Music Samples (2016)

"In algorithmic music composition, a simple technique involves selecting notes sequentially according to a transition table that specifies the probability of the next note as a function of the previous context. I describe an extension of this transition table approach using a recurrent autopredictive connectionist network called CONCERT. CONCERT is trained on a set of pieces with the aim of extracting stylistic regularities. CONCERT can then be used to compose new pieces. A central ingredient of CONCERT is the incorporation of psychologically-grounded representations of pitch, duration, and harmonic structure. CONCERT was tested on sets of examples artificially generated according to simple rules and was shown to learn the underlying structure, even where other approaches failed. In larger experiments, CONCERT was trained on sets of J. S. Bach pieces and traditional European folk melodies and was then allowed to compose novel melodies. Although the compositions are occasionally pleasant, and are preferred over compositions generated by a third-order transition table, the compositions suffer from a lack of global coherence. To overcome this limitation, several methods are explored to permit CONCERT to induce structure at both fine and coarse scales. In experiments with a training set of waltzes, these methods yielded limited success, but the overall results cast doubt on the promise of note-by-note prediction for composition."

Description from http://www.cs.colorado.edu/~mozer/Research/Selected%20Publications/music.html

Original audio is http://www.cs.colorado.edu/~mozer/Research/Selected%20Publications/audio/Bach.mp3

After Bach by Michael C. Mozer (1994)

From the album "Coditany of Timeness" https://dadabots.bandcamp.com/album/coditany-of-timeness

"This album is part of a submission to NIPS 2017 Workshop for Machine Learning, Creativity and Design: "Generating Black Metal and Math Rock". 

This album was generated with a recurrent neural network* trained on raw audio from the album "Diotima" by Krallice. All titles were generated by a Markov chain. The album cover was created with neural style transfer. "

Memoryearion by DADABOTS (2017)

From "An Improved Relative Self-Attention Mechanism for Transformer with Application to Music Generation, Online Supplement" here https://storage.googleapis.com/music-transformer/index.html

From the paper "Music Transformer" (2018) by Huang et al. https://arxiv.org/abs/1809.04281v2

Music Transformer by Huang et al. (2018)

From "An Improved Relative Self-Attention Mechanism for Transformer with Application to Music Generation, Online Supplement" here https://storage.googleapis.com/music-transformer/index.html

From the paper "Music Transformer" (2018) by Huang et al. https://arxiv.org/abs/1809.04281v2

Described as "New Sample: Relatively Jazz".

I collect sounds and music made with machine learning. All content here is sourced, and shared for educational purposes.

Composing Music With RNNs by Daniel Johnson (2015)

Machine Learning Music

Related tracks