Protein Counterpoint Sonification

Molecular structures are translated into a multi-instrument score, focused on the nuclocapsid spike in SARS-CoV-2 interacting with an antibody molecule. The complex piece, resembling the entanglement of three protein chains, is analyzed and a series of musical fragments are extracted (parts I, II, III, IV and V). The parts build on each other and are organized in increasing complexity, I being the simplest and V being the most complex version.

N Spike Antibody Fragment V

The construction of biological matter, and life itself, features striking similarities with musical creativity. In my research, I have explored these relationships with a combination of biology, chemistry, as well as mathematical and music theoretical tools. 

Listen here how dinoflagellate (a single-celled organism with two flagella, occurring in large numbers in marine plankton and also found in fresh water)  induce hierarchical vibrations translated to musical form.

Superfluous Dreams (Violin Adagio) - Part V

Superfluous Dreams (Violin Adagio) - Part VIIII

Swamp Variations: Emergence, Evolution, Division, Spring, Retreat 

Part of a series of 5 parts: This is Evolution.

Ambient version (reduced orchestration, focus on piano and flute, string instruments)

Swamp Variations - Part II - Evolution (((AMBIENT SPACE VERSION )))

Sounds of the escherichia virus T4,  a species of bacteriophages that infect Escherichia coli bacteria. What you hear is the sonified expression of the 3D structure of the protein needle at the core of this virus, classified in the protein data bank with ID 4osd (triple-stranded beta-helix comprising the central spike of bacteriophage T4. 

See http://dx.doi.org/10.2210/pdb4OSD/pdb for more information.  

The music represents the spike-shaped protein that functions to disrupt the membrane of the target cell during phage infection. The protein structure has first been reported in [1].

The protein structure is translated into an orchestral setting (koto [a Japanese string instrument], cello, double bass, viola, flute, and a range of other orchestral instruments). The various instruments is used to express the primary, secondary and tertiary structure of the protein. The piece is composed of a A-B-A' structure, where A and A' reflect the beginning and end coding of the protein, the B part represents the coding of the primary backbone sequence and structure. 

The track image shows a picture of this protein, reflecting a highly organized, repetitive structure with continuous slight variations, which is also reflected musically. The extensive length of 40+ minutes is due to the long protein chains included in this protein assembly. 

The piece reflects a classical period model of the hierarchical mapping to describe the 3D folded structure of a protein assembly (consisting of 9 protein chains) in musical space. The musical mapping begins from the primary amino acid sequence and encodes multiple hierarchical structural levels, and is complemented with musical experimentation. 

We use the concept proposed by Yu, Buehler et al. (ACS Nano) [2] to map amino acid sequences to music, and augment the mapping by adding more data. The 3D folded structure of a protein is coded by inserting strummed chords and interwoven, cascaded melodies and rhythms that code for the geometric arrangement of amino acids, the protein's tertiary structure. Rhythm and note volume code for secondary structure.

References:

[1] Original research paper describing the gp5β-BC2 fragment: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4576200/

[2] https://doi.org/10.1021/acsnano.9b02180

[3] Track image shows the phage, a virus that infects and replicates within bacteria and archaea (source: Wikipedia, by: Adenosine, CC BY-SA 3.0, https://commons.wikimedia.org/w/index.php?curid=8643737, depicting the structure of a typical myovirus bacteriophage)

Viral koto ensemble of protein 4osd with cello, violin, flute and bass

Protein resonances inspire a minimalist evolution of ideas across the resonances of piano and solo strings. The structure evolves slowly as the motion of jellyfish, a natural hydrogel, in water. Ethereal resonance.

The series of short pieces is based upon this musical reflection of infection, inverting the original molecular attachment: https://soundcloud.com/user-275864738/coronavirus-spike-protein-bound-to-ace2

Piano And Cello (VI)

Ethereal Resonance: Protein Inspiration - VI (PIano and Cello)

This piece is a sonification of the spike protein of the pathogen of COVID-19, 2019-nCoV (protein data bank identifier 6VSB [1]). What you hear is a multi-layered sonification approach featuring both the vibrational spectrum of the entire protein (expressed in sound and rhythmic elements), as well as the sequence and folding of amino acids that compose the virus spike structure [2]. 

References
[1] Wrapp et al., "Cryo-EM structure of the 2019-nCoV spike in the prefusion conformation," Science, 2020, DOI: 10.1126/science.abb2507
[2] Buehler et al., "A Self-Consistent Sonification Method to Translate Amino Acid Sequences into Musical Compositions and Application in Protein Design Using Artificial Intelligence," ACS Nano, 2019, DOI: https://pubs.acs.org/doi/10.1021/acsnano.9b02180

Sonification of a virus spike protein

Sonification of the Coronavirus Spike Protein (Amino Acid Scale)

While we cannot see small nanoscopic objects like proteins or other molecules that make up virtually all living matter including our cells, tissues, as well as pathogens such as viruses, our computational algorithm allows us to make its material manifestation audible.  This piece is a musical representation of the amino acid sequence and structure of the spike protein of the pathogen of COVID-19, 2019-nCoV (protein data bank identifier 6VSB [1]). 

MUSICAL DECEIT

A virus’ genome hijacks the host cell’s protein manufacturing machinery, and forces it to replicate the viral genome and produce viral proteins to make new viruses from it.  This musical art teaches us something about the fine line between beauty of life and death as an opposite pole.  As you listen to the protein you will find that the intricate design results in incredibly interesting and actually pleasing, relaxing sounds. This doesn’t really convey the deadly impacts this particularly protein is having on the world. This aspect of the music shows the deceiving nature of the virus, how it hijacks our body to replicate, and hurt us along the way. So, the music is a metaphor for this nature of the virus to deceive the host and exploit it for its own multiplication. 

HOW IT IS DONE: 

What you hear is a multi-layered algorithmic composition featuring both the vibrational spectrum of the entire protein (expressed in sound and rhythmic elements), the sequence and folding of amino acids that compose the virus spike structure, as well as interwoven melodies - forming counterpoint music - reflecting the complex hierarchical intersecting geometry of the protein [2].

Scientific references:

[1] Wrapp et al., "Cryo-EM structure of the 2019-nCoV spike in the prefusion conformation," Science, 2020, DOI: 10.1126/science.abb2507

[2] Buehler et al., "A Self-Consistent Sonification Method to Translate Amino Acid Sequences into Musical Compositions and Application in Protein Design Using Artificial Intelligence," ACS Nano, 2019, DOI: pubs.acs.org/doi/10.1021/acsnano.9b02180

Viral counterpoint of the coronavirus spike protein (2019-nCoV)

Viral Counterpoint of the Coronavirus Spike Protein (2019-nCoV)

This is a sonification of a protein extracted from honeybees, and folded using a deep learning method described in [1].

BACKGROUND:

The development of rational techniques to discover new mechanically relevant proteins for use in variety of applications ranging from mechanics, agriculture to biotechnology remains an outstanding nanomechanical design problem. The key barrier is to design a sequence to fold into a predictable structure to achieve a certain material function. Focused on alpha-helical proteins (as found in skin, hair, and many other mechanically relevant protein materials), we report a Multi-scale Neighborhood-based Neural Network (MNNN) model to learn how a specific amino acid sequence folds into a protein structure. 

The algorithm predicts the protein structure without using a template or co-evolutional information at a maximum error of 2.1 Å. We find that the prediction accuracy is higher than other models and the prediction consumes less than six orders of magnitude time than ab initio folding methods. We demonstrate that MNNN can predict the structure of an unknown protein that agrees with experiments, and our model hence shows a great advantage in the rational design of de novo proteins.

Reference: Z. Qin, B. Marelli, M.J. Buehler, et al., "Artificial intelligence method to design and fold alpha-helical structural proteins from the primary amino acid sequence," Extreme Mechanics Letters, Vol. 36, p. 100652, 2020 https://doi.org/10.1016/j.eml.2020.100652

Honeybee Silk

This piece explores melodic and rhythmic content from a variety of protein structures, including the binding event of COVID-19 to human cells. The progression models pre-pandemic humanity and the virus alone ("growth"), then the virus interacting with human cell - a struggle modeling repeated infection of new cells ("infection") - and finally back to virus alone overlaid with the regular rhythm of human life ("release").  As expressed musically, life after the virus is different than before , since we all changed due to the pandemic. How, we do not know yet. 

LYRICS:

Growth, Infection and Release.
Growth represents prepandemic humanity and the virus alone.
Infection represents the period during with the virus is interacting with human cells.
There is a struggle between infection of new cells and human defenses.
Finally, we return to the regular rhythm of human life, the release.
Life after the virus is different than before. We all changed due to the pandemic.
How we have changed, we do not know yet.

MORE DETAILS:

The rhythm is derived directly from the vibrational spectrum of the virus spike protein attached to the human ACE2 receptor, scaled to the frequencies that human ears perceive as rhythm. 

The construction of this piece involves the vibrational frequencies of virus proteins alone, melding into overlays of the virus with human proteins attached, peaking and then decaying as the virus releases its grip on humanity.  The piece ends with the steady motions of human existence, as we enter release.

The entire composition is written in the amino acid scale, making molecular vibrations audible in a range of experiences from rhythms, to harmony, to melodies. For sources of sonic content, we explore a variety of protein structures - virus spike protein alone, spike protein attached to human cell, human cell alone, and various states of the human cell alone. These provide the melodic foundation on which the progression is based. 

The track image is taken from [1], a recent study to make protein vibrations visible via water waves.

[1] M. Buehler, Liquified protein vibrations, classification and cross-paradigm de novo image generation using deep neural networks, https://arxiv.org/abs/2004.07603, 2020

Growth, Infection and Release

Matter is sound and sound is matter. Since the mapping of amino acids to sound is unique, it can be reversed – out of sound, comes a unique sequence of amino acids. Here we experiment with an exercise to make sound into material, whereby we compose new music derived from protein sonifications of different virus species. This offers a new avenue for interspecies communication, using sound as a sort of universal language of life. 

This is a modified version of this piece: https://soundcloud.com/user-275864738/viral-protein-jam, expressed here as an ambient, lengthy musical experiment that offers variations of the protein design theme. 

Through this musical experiment we travel into the strange sounds, rhythms, and melodies of the inner workings of proteins - not just one, but a musical integration, a jamming, of different ones.  

The music was created by generating a large amount of "songs" from spike proteins found in various coronavirus species, then teaching an artificial neural network - AI - how these different proteins sound like. And then, we use AI to generate new music, and translate the music back to proteins. This allowed us to come up with new protein designs that nature has not yet invented.

The sound reflects the musical realization of this new protein design. It is a journey into a new world where proteins across species interact and recombine, each contributing patterns, ideas, from their respective language of life.

BROADER IMPACTS

Through music, we can learn to speak the language of life. This may revolutionize our ability to understand life, treat diseases, and develop new medical technologies and biomaterials, or even drugs.

Music is everywhere. Matter is sound and sound is matter.

TRACK IMAGE

The track image was generated using the method described in [1].

[1] M. Buehler, "Liquified protein vibrations, classification and cross-paradigm de novo image generation using deep neural networks," arxiv.org/abs/2004.07603

Ambient Viral Jam (Protein Inceptionism)

Swamp Variations: Emergence, Evolution, Division, Spring, Retreat 

Part of a series of 5 parts: This is Division.

Swamp Variations - Part III - Division

Protein resonances inspire a minimalist evolution of ideas across the resonances of piano and solo strings. The structure evolves slowly as the motion of jellyfish, a natural hydrogel, in water. Ethereal resonance.

Piano And Cello (VIIII)

Ethereal Resonance: Protein Inspiration - VIIII (PIano and Cello)

A musical experiment with proteins, coyote, bells, guitar, and a rooster. This is an audible journey through a variety of natural sources of sound, invoking various forms of life, materials, as well as electronic synthesis methods.

Typically, sound is made from vibrating a material, like a guitar string, and music is made by arranging sounds in hierarchical patterns. Here we explore a variety of modes of sound generation - from string vibrations (guitar), to protein vibrations (molecular music), to animals (coyote and rooster), to bells, and a few other unusual sources. In one sweep, our ears pick up all of the features a complex hierarchical system: pitch, timbre, volume, melody, rhythm, and chords.

Music, as a universal language, connects people across the world. More so, it allows us to explore the physical features of matter - both living systems and inanimate - in their ability to generate sound through energetic excitation. Hence, it is a model of our shared humanity. And we can cross scales and species, as done in this piece, where we visualize distinct organisms like animals, materials, and humans interacting in a sort of virtual jam, in musical space.

Energy Equilibrium: Proteins, Coyote, Bells, Guitar, and a Rooster (((Ambient Version)))

Ethereal Resonance: Protein Inspiration - VII(PIano and Cello)

Ethereal Resonance: Protein Inspiration - I (PIano and Cello)

Markus J. Buehler is the McAfee Professor of Engineering at MIT, and a composer of experimental, classical and electronic music, with an interest in sonification. Using an approach termed "materiomusic", his artistic work explores the creation of new forms of musical expression - such as those derived from biological materials and living systems - as a means to better understand the underlying science and mathematics. One of his goals is to use musical and sound design as a novel and abstract way to model, optimize and create new forms of matter from the bottom up, and to assess cross-system design relationships. He is also interested in research to explore relationships between classical music, mathematics, and the physical and biological sciences, an in the mapping of models of consciousness across systems. In recent work he has developed a new framework to compose music based on proteins – the basic molecules of all life, as well as other physical phenomena such as fracturing, to explore similarities and differences across species, scales and between philosophical and physical models. 

Contact: mbuehler@MIT.EDU

Ambient Viral Jam (Protein Inceptionism)

Markus J. Buehler

Related tracks