Protein Counterpoint Sonification

Sounds of the escherichia virus T4,  a species of bacteriophages that infect Escherichia coli bacteria. What you hear is the sonified expression of the 3D structure of the protein needle at the core of this virus, classified in the protein data bank with ID 4osd (triple-stranded beta-helix comprising the central spike of bacteriophage T4. 

See http://dx.doi.org/10.2210/pdb4OSD/pdb for more information.  

The music represents the spike-shaped protein that functions to disrupt the membrane of the target cell during phage infection. The protein structure has first been reported in [1].

The protein structure is translated into an orchestral setting (koto [a Japanese string instrument], cello, double bass, viola, flute, and a range of other orchestral instruments). The various instruments is used to express the primary, secondary and tertiary structure of the protein. The piece is composed of a A-B-A' structure, where A and A' reflect the beginning and end coding of the protein, the B part represents the coding of the primary backbone sequence and structure. 

The track image shows a picture of this protein, reflecting a highly organized, repetitive structure with continuous slight variations, which is also reflected musically. The extensive length of 40+ minutes is due to the long protein chains included in this protein assembly. 

The piece reflects a classical period model of the hierarchical mapping to describe the 3D folded structure of a protein assembly (consisting of 9 protein chains) in musical space. The musical mapping begins from the primary amino acid sequence and encodes multiple hierarchical structural levels, and is complemented with musical experimentation. 

We use the concept proposed by Yu, Buehler et al. (ACS Nano) [2] to map amino acid sequences to music, and augment the mapping by adding more data. The 3D folded structure of a protein is coded by inserting strummed chords and interwoven, cascaded melodies and rhythms that code for the geometric arrangement of amino acids, the protein's tertiary structure. Rhythm and note volume code for secondary structure.

References:

[1] Original research paper describing the gp5β-BC2 fragment: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4576200/

[2] https://doi.org/10.1021/acsnano.9b02180

[3] Track image shows the phage, a virus that infects and replicates within bacteria and archaea (source: Wikipedia, by: Adenosine, CC BY-SA 3.0, https://commons.wikimedia.org/w/index.php?curid=8643737, depicting the structure of a typical myovirus bacteriophage)

Viral koto ensemble of protein 4osd with cello, violin, flute and bass

This piece is a sonification of the spike protein of the pathogen of COVID-19, 2019-nCoV (protein data bank identifier 6VSB [1]). What you hear is a multi-layered sonification approach featuring both the vibrational spectrum of the entire protein (expressed in sound and rhythmic elements), as well as the sequence and folding of amino acids that compose the virus spike structure [2]. 

References
[1] Wrapp et al., "Cryo-EM structure of the 2019-nCoV spike in the prefusion conformation," Science, 2020, DOI: 10.1126/science.abb2507
[2] Buehler et al., "A Self-Consistent Sonification Method to Translate Amino Acid Sequences into Musical Compositions and Application in Protein Design Using Artificial Intelligence," ACS Nano, 2019, DOI: https://pubs.acs.org/doi/10.1021/acsnano.9b02180

Sonification of a virus spike protein

Sonification of the Coronavirus Spike Protein (Amino Acid Scale)

While we cannot see small nanoscopic objects like proteins or other molecules that make up virtually all living matter including our cells, tissues, as well as pathogens such as viruses, our computational algorithm allows us to make its material manifestation audible.  This piece is a musical representation of the amino acid sequence and structure of the spike protein of the pathogen of COVID-19, 2019-nCoV (protein data bank identifier 6VSB [1]). 

MUSICAL DECEIT

A virus’ genome hijacks the host cell’s protein manufacturing machinery, and forces it to replicate the viral genome and produce viral proteins to make new viruses from it.  This musical art teaches us something about the fine line between beauty of life and death as an opposite pole.  As you listen to the protein you will find that the intricate design results in incredibly interesting and actually pleasing, relaxing sounds. This doesn’t really convey the deadly impacts this particularly protein is having on the world. This aspect of the music shows the deceiving nature of the virus, how it hijacks our body to replicate, and hurt us along the way. So, the music is a metaphor for this nature of the virus to deceive the host and exploit it for its own multiplication. 

HOW IT IS DONE: 

What you hear is a multi-layered algorithmic composition featuring both the vibrational spectrum of the entire protein (expressed in sound and rhythmic elements), the sequence and folding of amino acids that compose the virus spike structure, as well as interwoven melodies - forming counterpoint music - reflecting the complex hierarchical intersecting geometry of the protein [2].

Scientific references:

[1] Wrapp et al., "Cryo-EM structure of the 2019-nCoV spike in the prefusion conformation," Science, 2020, DOI: 10.1126/science.abb2507

[2] Buehler et al., "A Self-Consistent Sonification Method to Translate Amino Acid Sequences into Musical Compositions and Application in Protein Design Using Artificial Intelligence," ACS Nano, 2019, DOI: pubs.acs.org/doi/10.1021/acsnano.9b02180

Viral counterpoint of the coronavirus spike protein (2019-nCoV)

Viral Counterpoint of the Coronavirus Spike Protein (2019-nCoV)

Musical representation of de novo protein designed using artificial intelligence, and folded and equilibrated using molecular dynamics, as reported in [1]. The musical composition renders the 3D structure of this alpha-helix rich protein in an audible space, presenting complex melodic and rhythmic patterns that reflect the hierarchical design of this protein. This is a de novo protein, in other words, a protein that does not exist in nature, but which was designed through our AI algorithm that generated new music, then translated back into a protein amino acid sequence. 

[1] C.H. Yu, M.J. Buehler, "Sonification based de novo protein design using artificial intelligence, structure prediction and analysis using molecular modeling," APL Bioengineering, DOI: 10.1063/1.5133026, 2020.

Concerto of a De Novo Protein Designed using Artificial Intelligence

Collagen constitutes one-third of the human proteome, providing mechanical stability, elasticity, and strength to organisms and is the prime construction material in biology. Collagen is also the dominating material in the extracellular matrix and its stiffness controls cell differentiation, growth, and pathology. 

As a molecule, it forms a rope-like structure with three interwoven amino acid chains, resembling a powerful string, or triple helix. As one of the most abundant proteins, it plays a foundational role in all of biology. Read more about collagen in the references below [1-3].  

Creating its unique triple helical shape is a particular sequence pattern of amino acids, which is reflected in the regular, periodic realization of the musical score presented here. The protein sonified in this composition is a short collagen segment with Protein Data Bank ID 1cag, with identical chains, each of which has this sequence:

PPGPPGPPGP PGPPAPPGPP GPPGPPGPPG 

[1] https://en.wikipedia.org/wiki/Collagen 

[2] A. Gautieri, M.J. Buehler, et al., Nano Letters, 2012, https://pubs.acs.org/doi/10.1021/nl103943u

[3] M.J. Buehler, PNAS, 2006,https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1567872/

Collagen Etude

This is a sonification of a protein extracted from honeybees, and folded using a deep learning method described in [1].

BACKGROUND:

The development of rational techniques to discover new mechanically relevant proteins for use in variety of applications ranging from mechanics, agriculture to biotechnology remains an outstanding nanomechanical design problem. The key barrier is to design a sequence to fold into a predictable structure to achieve a certain material function. Focused on alpha-helical proteins (as found in skin, hair, and many other mechanically relevant protein materials), we report a Multi-scale Neighborhood-based Neural Network (MNNN) model to learn how a specific amino acid sequence folds into a protein structure. 

The algorithm predicts the protein structure without using a template or co-evolutional information at a maximum error of 2.1 Å. We find that the prediction accuracy is higher than other models and the prediction consumes less than six orders of magnitude time than ab initio folding methods. We demonstrate that MNNN can predict the structure of an unknown protein that agrees with experiments, and our model hence shows a great advantage in the rational design of de novo proteins.

Reference: Z. Qin, B. Marelli, M.J. Buehler, et al., "Artificial intelligence method to design and fold alpha-helical structural proteins from the primary amino acid sequence," Extreme Mechanics Letters, Vol. 36, p. 100652, 2020 https://doi.org/10.1016/j.eml.2020.100652

Honeybee Silk

This composition is the musical reflection of infection, the moment during which the virus interacts with the cell, a microscope into the details of molecular motions of attachment and release. 

BACKGROUND

This piece is a representation of part of the protein structure with identifier 6m17 (the2019-nCoV RBD/ACE2-B0AT1 complex), representing the coronoavirus (COVID-19) spike protein bound to the human cell receptor ACE2. What you hear is an evolution that reflects the genetic and structural details of the protein interactions that occur during the virus infection. Notably, the angiotensin-converting enzyme 2 (ACE2) is the cellular receptor for SARS coronavirus (SARS-CoV) and the new coronavirus (SARS-CoV-2) that is causing  COVID-19. 

In the track image, the very left pieces are the tips of the virus spike protein, and towards the right you see the ACE2 receptor, as well as part of B0AT. 

This composition is part of a series of pieces that explore various biomolecular stages of the pandemic, featuring multiscale assessments of the virus spike protein, pandemic progression, and infection mechanisms (this piece). 

SOME MORE INFORMATION:

While we cannot see small nanoscopic objects like proteins or other molecules that make up virtually all living matter including our cells, tissues, as well as pathogens such as viruses, our computational algorithm allows us to make its material manifestation audible.  

HOW IT IS DONE:

What you hear is a multi-layered algorithmic composition, for piano, that reflects the protein structure of the spike protein attached to the human cell receptor ACE2. Details see [1-3].

Scientific references:

[1] Wrapp et al., "Cryo-EM structure of the 2019-nCoV spike in the prefusion conformation," Science, 2020, DOI: 10.1126/science.abb2507

[2] Buehler et al., "A Self-Consistent Sonification Method to Translate Amino Acid Sequences into Musical Compositions and Application in Protein Design Using Artificial Intelligence," ACS Nano, 2019, DOI: pubs.acs.org/doi/10.1021/acsnano.9b02180

[3] R. Yan, Y. Zhang, Y. Li, L. Xia, Y. Guo, Q. Zhou, Structural basis for the recognition of the SARS-CoV-2 by full-length human ACE2., Science. 367 (2020) 1444–1448. doi:10.1126/science.abb2762.

Molecular Infection (for piano)

Reflection of Infection (for Piano)

This piece explores melodic and rhythmic content from a variety of protein structures, including the binding event of COVID-19 to human cells. The progression models pre-pandemic humanity and the virus alone ("growth"), then the virus interacting with human cell - a struggle modeling repeated infection of new cells ("infection") - and finally back to virus alone overlaid with the regular rhythm of human life ("release").  As expressed musically, life after the virus is different than before , since we all changed due to the pandemic. How, we do not know yet. 

LYRICS:

Growth, Infection and Release.
Growth represents prepandemic humanity and the virus alone.
Infection represents the period during with the virus is interacting with human cells.
There is a struggle between infection of new cells and human defenses.
Finally, we return to the regular rhythm of human life, the release.
Life after the virus is different than before. We all changed due to the pandemic.
How we have changed, we do not know yet.

MORE DETAILS:

The rhythm is derived directly from the vibrational spectrum of the virus spike protein attached to the human ACE2 receptor, scaled to the frequencies that human ears perceive as rhythm. 

The construction of this piece involves the vibrational frequencies of virus proteins alone, melding into overlays of the virus with human proteins attached, peaking and then decaying as the virus releases its grip on humanity.  The piece ends with the steady motions of human existence, as we enter release.

The entire composition is written in the amino acid scale, making molecular vibrations audible in a range of experiences from rhythms, to harmony, to melodies. For sources of sonic content, we explore a variety of protein structures - virus spike protein alone, spike protein attached to human cell, human cell alone, and various states of the human cell alone. These provide the melodic foundation on which the progression is based. 

The track image is taken from [1], a recent study to make protein vibrations visible via water waves.

[1] M. Buehler, Liquified protein vibrations, classification and cross-paradigm de novo image generation using deep neural networks, https://arxiv.org/abs/2004.07603, 2020

Growth, Infection and Release

Matter is sound and sound is matter. Since the mapping of amino acids to sound is unique, it can be reversed – out of sound, comes a unique sequence of amino acids. Here we experiment with an exercise to make sound into material, whereby we compose new music derived from protein sonifications of different virus species. This offers a new avenue for interspecies communication, using sound as a sort of universal language of life. 

The result sounds otherworldly, as we travel into the strange sounds, rhythms, and melodies of the inner workings of proteins - not just one, but a musical integration, a jamming, of different ones. The music is performed using an ensemble of violin, flute, koto and other harp instruments, brass, timpani drums, and string sections. 

To do this we generated a large amount of "songs" from spike proteins found in various coronavirus species.  We then teach an artificial neural network - AI - how these different proteins sound like. And then, we use AI to generate new music, and translate the music back to proteins. This allowed us to come up with new protein designs that nature has not yet invented. 

What you hear in this piece is the musical realization of this new protein design, a result of proteins from different viruses jamming together through machine learning, and self-assembling into new forms. It is a journey into a new world where proteins across species interact and recombine, each contributing patterns, ideas, from their respective language of life. 

This work was first presented at TEDx Basel on May 1, 2020. 

BROADER IMPACTS

Through music, we can learn to speak the language of life. This may revolutionize our ability to understand life, treat diseases, and develop new medical technologies and biomaterials, or even drugs. 

Music is everywhere. Matter is sound and sound is matter.

TRACK IMAGE

The track image was generated using the method described in [1].

[1] M. Buehler, "Liquified protein vibrations, classification and cross-paradigm de novo image generation using deep neural networks," https://arxiv.org/abs/2004.07603

Viral Protein Jam

Matter is sound and sound is matter. Since the mapping of amino acids to sound is unique, it can be reversed – out of sound, comes a unique sequence of amino acids. Here we experiment with an exercise to make sound into material, whereby we compose new music derived from protein sonifications of different virus species. This offers a new avenue for interspecies communication, using sound as a sort of universal language of life. 

This is a modified version of this piece: https://soundcloud.com/user-275864738/viral-protein-jam, expressed here as an ambient, lengthy musical experiment that offers variations of the protein design theme. 

Through this musical experiment we travel into the strange sounds, rhythms, and melodies of the inner workings of proteins - not just one, but a musical integration, a jamming, of different ones.  

The music was created by generating a large amount of "songs" from spike proteins found in various coronavirus species, then teaching an artificial neural network - AI - how these different proteins sound like. And then, we use AI to generate new music, and translate the music back to proteins. This allowed us to come up with new protein designs that nature has not yet invented.

The sound reflects the musical realization of this new protein design. It is a journey into a new world where proteins across species interact and recombine, each contributing patterns, ideas, from their respective language of life.

BROADER IMPACTS

Through music, we can learn to speak the language of life. This may revolutionize our ability to understand life, treat diseases, and develop new medical technologies and biomaterials, or even drugs.

Music is everywhere. Matter is sound and sound is matter.

TRACK IMAGE

The track image was generated using the method described in [1].

[1] M. Buehler, "Liquified protein vibrations, classification and cross-paradigm de novo image generation using deep neural networks," arxiv.org/abs/2004.07603

Ambient Viral Jam (Protein Inceptionism)

Markus J. Buehler is the McAfee Professor of Engineering at MIT, and a composer of experimental, classical and electronic music, with an interest in sonification. Using an approach termed "materiomusic", his artistic work explores the creation of new forms of musical expression - such as those derived from biological materials and living systems - as a means to better understand the underlying science and mathematics. One of his goals is to use musical and sound design as a novel and abstract way to model, optimize and create new forms of matter from the bottom up, and to assess cross-system design relationships. He is also interested in research to explore relationships between classical music, mathematics, and the physical and biological sciences, an in the mapping of models of consciousness across systems. In recent work he has developed a new framework to compose music based on proteins – the basic molecules of all life, as well as other physical phenomena such as fracturing, to explore similarities and differences across species, scales and between philosophical and physical models. 

Contact: mbuehler@MIT.EDU