Protein Counterpoint Sonification

We are researching the use of algorithmic composition to generate counterpoint through fracture. Driven by the universal physical forces near a singularity, captured in the so-called Cauchy stress tensor (read more about it here: en.wikipedia.org/wiki/Cauchy_stress_tensor), we are creating patterns of interdependent polyphony through independent patterning in time and pitch.

FRACTURES ARE UNIVERSAL

A fracture is a crack in a material (often seen in glass, but also represented in earthquakes when tectonic plates fracture, or in the unfolding of a protein). Fracturing is the process of destruction of structure, and hence is the division between function and failure in material and has correspondences in philosophy. The tipping point of the crack, the locality of greatest action, and where bond breaking occurs, is the singular crack tip, the sharp edge.
All fractures feature a universal distribution of stresses – the forces that act inside the material and which stretch/compress chemical bonds. No matter what material, as you get closer to the singularity – the tip of the crack – the distribution of forces are identical.

In the three pieces in this set, we explore these universal features to generate algorithmically composed music. We try to explore whether the universal features of fractures have the capacity to generate counterpoint – universal features of music, to better understand the analogy across manifestations.
Read more about the physics of fracture in [1].


Fracture-generated counterpoint hence may reflect the universal features of fractures and provides a model thereof. Owing to the universal significance of fracturing as a process in material and amaterial manifestations, the musical effects generated hereby can be an interesting subject of further analysis and inspiration.

[1] M. Buehler, Atomistic Modeling of Materials Failure, 2008 www.springer.com/gp/book/9780387764252

Counterpoint Through Fracture

Inspiration from Nature knows no boundary and can take on unusual forms.

Fire has fascinated humankind since the prehistoric era. Rooted in the interactions between sound and flames, we report a method to use fire for a variety of purposes, including sonification, art, and the design and manufacturing nature-inspired materials.

We present a method to sonify fire, thereby offering a translation from the silent nature of flames, to represent audible information, and to generate de novo flame images.

To realize material specimen derived from fire, we use the autoencoder to generate image stacks to yield continuous three dimensional geometries that are manufactured using 3D printing.

This represents the first generation of nature-inspired audible signals from fire, and can be a platform to be used for other natural phenomena in the quest for de novo architectures, geometries, and design ideas, thus creating additional directions in artistic and scientific research through the creative manipulation of data with structural similarities across fields.

Original paper: https://www.cell.com/iscience/fulltext/S2589-0042(21)00841-5

Fire sonification: Making silent flames audible

Musical reflection of the molecular Intersection of nucleocapsid antibody and SARS-CoV-2, for piano, violin, and cello. In collaboration with Yun Yoo-jung, Alex Wooju Lim and Hyung Joon Won

Molecular Intersection of Nucleocapsid Antibody and SARS-CoV-2 (for Piano, Violin, and Cello)

Molecular structures are translated into a multi-instrument score, focused on the nuclocapsid spike in SARS-CoV-2 interacting with an antibody molecule. The complex piece, resembling the entanglement of three protein chains, is analyzed and a series of musical fragments are extracted (parts I, II, III, IV and V). The parts build on each other and are organized in increasing complexity, I being the simplest and V being the most complex version.

N Spike Antibody Fragment I

N Spike Antibody Fragment II

N Spike Antibody Fragment III

N Spike Antibody Fragment V

N Spike Antibody Fragment IV

Superfluous Dreams (Violin Adagio) - Part VIIII

We report a self-consistent method to translate amino acid sequences into audible sound, use the representation in the musical space to train a neural network, and then use it to generate new protein designs using artificial intelligence (AI). The sonification method proposed here uses the normal mode vibrations of the amino acid building blocks of proteins to compute an audible representation of each of the 20 natural amino acids, which is fully defined by the overlay of its respective natural vibrations computed using quantum mechanics. 

The vibrational frequencies are transposed to the audible spectrum following the musical concept of transpositional equivalence, playing or writing music in a way that makes it sound higher or lower in pitch while retaining the relationships between tones or chords played. This transposition method ensures that the relative values of the vibrational frequencies within each amino acid and amongst different amino acids are retained. 

The characteristic frequency spectrum and sound associated with each of the amino acids represents a type of musical scale that consists of 20 unique tones, the “amino acid scale”. 

The audio represents a sweep through all 20 natural amino acids:

Alanine 
Arginine 
Asparagine 
Aspartic acid 
Cysteine 
Glutamic acid 
Glutamine 
Glycine 
Histidine 
Isoleucine 
Leucine 
Lysine 
Methionine 
Phenyalanine 
Proline 
Serine 
Threonine 
Tryptophan 
Tyrosine 
Valine 

More on amino acids: https://en.wikipedia.org/wiki/Amino_acid

Sounds of the basic building blocks of life: A sweep through all natural 20 amino acids

This piece is constructed entirely from the sounds generated from amino acids, the building blocks of proteins. Encoded by the DNA sequence, amino acids form chains that assembly into hierarchical materials that serve diverse functions in biology (as enzymes, structural materials, signal transmitters, and many others). 

All sounds used to compose this piece are generated from the sounds of amino acids, computed from their quantum mechanical vibrations, reflecting their elementary molecular features. For further details on the scientific foundation, see [1]. 

The main melody is derived from the amino acid sequence of three proteins:

1) 6cjc (adrenodoxin reductase, a flavoenzyme that is involved in the biosynthesis of steroid hormones; https://www.rcsb.org/structure/1CJC)
2) A de novo protein synthesized using a machine learning algorithm as reported in [1]
3) 194l (lysozyme enzyme; found in tears of humans and animals and egg white; https://www.rcsb.org/structure/194L)

The melodic backbone is composed of a progression from proteins 1) to 2) to 3). Motifs derived from these proteins are used throughout to generate repetitive elements and to conjoin melodic concepts (e.g., segments extracted from 6cjc are used as transitional elements between 1cjc and the de novo protein and again during the transition to 194l. 

All rhythmic elements (bass drum, snare-like, hi-hat, and others) are generated from soundings of amino acids. Both melodic and rhythmic patterns used are derived from the three proteins cited above. 

No other synthetic or sampled sounds have been used in the composition. What you hear is a composition in the natural 20-tone amino acid scale. 

BACKGROUND:

Materials and music have been intimately connected throughout centuries of human evolution and civilization. Indeed, materials such as wood, animal skin or metals are the basis for most musical instruments used throughout history. Today, we are able to use advanced computing algorithms to blur the boundary between material and sound and use hierarchical representations of materials in distinct spaces such as sound or language to advance design objectives. The approach used in this work is that the translation of protein materials representations into music not only allows us to create musical instruments, but also enables us to exploit deep neural network models to represent and manipulate protein designs in the audio space. Thereby we take advantage of longer-range structure that is important in music, and which is equivalently important in protein design (in connecting amino acid sequence to secondary structure and folding). This paradigm goes beyond proteins but rather enables us to connect nanostructures and music in a reversible way, providing an approach to design nanomaterials, DNA, proteins, or other molecular architectures from the nanoscale upwards. 

Reference:

[1] C.H. Yu, Z. Qin, F. Martinez, M.J. Buehler, A Self-Consistent Sonification Method to Translate Amino Acid Sequences into Musical Compositions and Application in Protein Design using Artificial Intelligence, ACS Nano, in press, 2019

Orchestra Of Amino Acids

Sonification of a silk protein, rich in beta-sheet secondary structure.

Reference:

[1] C.H. Yu, Z. Qin, F. Martinez, M.J. Buehler, A Self-Consistent Sonification Method to Translate Amino Acid Sequences into Musical Compositions and Application in Protein Design using Artificial Intelligence, ACS Nano, in press, 2019

Last Move 84: Silk Protein Sonification

This piece reflects the sonification of an amyloid protein. Amyloids are a type of folding of a protein into highly stable, sticky fibrils, which are associated with around 50 diseases. 

Learn more about amyloids: https://en.wikipedia.org/wiki/Amyloid

Reference:

[1] C.H. Yu, Z. Qin, F. Martinez, M.J. Buehler, A Self-Consistent Sonification Method to Translate Amino Acid Sequences into Musical Compositions and Application in Protein Design using Artificial Intelligence, ACS Nano, in press, 2019

Amyloid protein sonification

This piece starts with an overlay of guitar and strings, followed by harp. Additional texture is given by solo cello, viola, bassoon and other orchestral instruments. Documenting the buildup of tension and release, an intense mood is generated with moving chord progressions. 

Part I of a series of four piece.

Guitar And Cello - Part I

Guitar And Cello: I. Piacevole

Going deeper inside a spider web. Multiple spider web variations interact across a spectrum of sonic textures. The sounds are based on a sonification method of a spider web as described in the references below [1-2], part of a larger collaboration with Tomás Saraceno, Saraceno Studios, Ally Bisshop, Thomas Muehletahler, and Evan Ziporyn, and the MIT Center for Art, Science and Technology (CAST).

Several individual webs constructed by Cyrtophora citricola are played in this piece, creating an overlaying soundscape of different shades. The piece includes a rhythm track for added structure, but all other sounds you hear are generated from the sonification of the spider web. 

The image [3] shows a 3D spider web, lit up using a green laser during scanning. The spider is visible in the top middle as she constructs the web. 

References:

[1] Su, I.; Qin, Z.; Saraceno, T.; Krell, A.; Mühlethaler, R.; Bisshop, A.; Buehler, M. J. Imaging and Analysis of a Three-Dimensional Spider Web Architecture. J. R. Soc. Interface 2018, 15 (146), 20180193. doi.org/10.1098/rsif.2018.0193.

[2] Su, I.; Qin, Z.; Bisshop, A.; Muehlethaler, R.; Ziporyn, E.; Buehler, M. J. Sonification of a 3D Spider Web and Reconstitution into Musical Composition Using Granular Synthesis. in submission

[3] Image by I. Su, MIT and M. Buehler, of a tidarren sisyphoides spider building a web, in collaboration with Tomas Saraceno Studio (Berlin)

Spider Variations

What you hear is the sonified expression of the 3D structure of the protein with PDB ID 1akg (ALPHA-CONOTOXIN PNIB FROM CONUS PENNACEUS, see: http://dx.doi.org/10.2210/pdb1AKG/pdb), translated into an orchestral setting (koto [a Japanese string instrument], cello, double bass, viola, flute, and fluegelhorn). The koto is used to express the primary, secondary and tertiary structure of the protein.  

The piece reflects a classical period model of the hierarchical mapping to describe the 3D folded structure of a protein in musical space. The musical mapping begins from the primary amino acid sequence (GCCSLPPCALSNPDYCX) and encodes multiple hierarchical structural levels, and adds musical experimentation. 

We use the concept proposed by Yu, Buehler et al. (ACS Nano) to map amino acid sequences to music, and augment the mapping by adding more data. The 3D folded structure of a protein is coded by inserting strummed chords and interwoven melodies that code for the geometric arrangement of amino acids, the protein's tertiary structure. Rhythm and note volume code for secondary structure.

[1] The protein used for this example is 6cjc (CRYSTAL STRUCTURE OF A FC FRAGMENT LALA MUTANT (L234A, L235A) OF HUMAN IGG1 (CRYSTAL FORM 3))

[2] https://doi.org/10.1021/acsnano.9b02180

Koto Ensemble of Protein 1akg (Venom of Predatory Marine Snails)

Sounds of the escherichia virus T4,  a species of bacteriophages that infect Escherichia coli bacteria. What you hear is the sonified expression of the 3D structure of the protein needle at the core of this virus, classified in the protein data bank with ID 4osd (triple-stranded beta-helix comprising the central spike of bacteriophage T4. 

See http://dx.doi.org/10.2210/pdb4OSD/pdb for more information.  

The music represents the spike-shaped protein that functions to disrupt the membrane of the target cell during phage infection. The protein structure has first been reported in [1].

The protein structure is translated into an orchestral setting (koto [a Japanese string instrument], cello, double bass, viola, flute, and a range of other orchestral instruments). The various instruments is used to express the primary, secondary and tertiary structure of the protein. The piece is composed of a A-B-A' structure, where A and A' reflect the beginning and end coding of the protein, the B part represents the coding of the primary backbone sequence and structure. 

The track image shows a picture of this protein, reflecting a highly organized, repetitive structure with continuous slight variations, which is also reflected musically. The extensive length of 40+ minutes is due to the long protein chains included in this protein assembly. 

The piece reflects a classical period model of the hierarchical mapping to describe the 3D folded structure of a protein assembly (consisting of 9 protein chains) in musical space. The musical mapping begins from the primary amino acid sequence and encodes multiple hierarchical structural levels, and is complemented with musical experimentation. 

We use the concept proposed by Yu, Buehler et al. (ACS Nano) [2] to map amino acid sequences to music, and augment the mapping by adding more data. The 3D folded structure of a protein is coded by inserting strummed chords and interwoven, cascaded melodies and rhythms that code for the geometric arrangement of amino acids, the protein's tertiary structure. Rhythm and note volume code for secondary structure.

References:

[1] Original research paper describing the gp5β-BC2 fragment: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4576200/

[2] https://doi.org/10.1021/acsnano.9b02180

[3] Track image shows the phage, a virus that infects and replicates within bacteria and archaea (source: Wikipedia, by: Adenosine, CC BY-SA 3.0, https://commons.wikimedia.org/w/index.php?curid=8643737, depicting the structure of a typical myovirus bacteriophage)

Viral koto ensemble of protein 4osd with cello, violin, flute and bass

In this piece we use algorithmic composition to generate counterpoint  through fracture. We are creating interdependent polyphony through independent patterning in time and pitch, driven by the universal physical forces near a singularity. 

FRACTURES ARE UNIVERSAL 

A fracture is a crack in a material (often seen in glass, but also represented in earthquakes when tectonic plates fracture, or in the unfolding of a protein). Fracturing is the process of destruction of structure, and hence is the division between function and failure in material and has correspondences in philosophy. The tipping point of the crack, the locality of greatest action, and where bond breaking occurs, is the singular crack tip, the sharp edge. 

All fractures feature a universal distribution of stresses – the forces that act inside the material and which stretch/compress chemical bonds.  No matter what material, as you get closer to the singularity – the tip of the crack – the distribution of forces are identical. 

Here we explore these universal features to generate algorithmically composed music. We try to explore whether the universal features of fractures have the capacity to generate counterpoint – universal features of music, to better understand the analogy across manifestations. 

Read more about the physics of fracture in [1].

HOW THE MUSIC IS CREATED

A set of Euclidean rhythmic motifs (with sparsity: 1/3, 5/12, and 7/12 to reflect different atomic organizations in crystalline forms) are acted upon based on the principal stresses near a crack under tensile loading.  We use the singular principal stress field to drive melodic content formation, whereas several independent moving observers assess the local principal stresses. The observes move in circular fashion, towards and away from the singularity in a spiral, oscillatory motion (see track image).  

Each individual note played is generated based on physical model of a string exposed to different stress levels. Hence, large stresses pull notes into high pitch, lower stresses compress it to lower pitches. 

The violent nature of forces experienced by the observer circling around the crack tip generate complex but highly organized melodic forms. As the stress field features universal, respective patterning, the various melodic lines generated have relationships rooted in the nanoscale physics surrounding the singularity. 

Fracture-generated counterpoint hence may reflect the universal features of fractures and provides a model thereof. Owing to the universal significance of fracturing as a process in material and amaterial manifestations, the musical effects generated hereby can be an interesting subject of further analysis and inspiration. 

[1] M. Buehler, Atomistic Modeling of Materials Failure, 2008 https://www.springer.com/gp/book/9780387764252

Counterpoint Through Fracture: Part I – Spiral Movers

For piano. We are researching the use of algorithmic composition to generate counterpoint through fracture. Driven by the universal physical forces near a singularity, captured in the so-called Cauchy stress tensor (read more about it here: https://en.wikipedia.org/wiki/Cauchy_stress_tensor), we are creating interdependent polyphony through independent patterning in time and pitch. 

FRACTURES ARE UNIVERSAL

A fracture is a crack in a material (often seen in glass, but also represented in earthquakes when tectonic plates fracture, or in the unfolding of a protein). Fracturing is the process of destruction of structure, and hence is the division between function and failure in material and has correspondences in philosophy. The tipping point of the crack, the locality of greatest action, and where bond breaking occurs, is the singular crack tip, the sharp edge.

All fractures feature a universal distribution of stresses – the forces that act inside the material and which stretch/compress chemical bonds. No matter what material, as you get closer to the singularity – the tip of the crack – the distribution of forces are identical.

In this study we explore these universal features to generate algorithmically composed music. We try to explore whether the universal features of fractures have the capacity to generate counterpoint – universal features of music, to better understand the analogy across manifestations.

Read more about the physics of fracture in [1].

HOW THE MUSIC IS CREATED

The piece is constructed from multiple movers in the singular stress field, and using the full stress tensor to derive musical structure. The observes move in circular fashion around the singularity in an oscillatory motion.  Each individual note played is generated based on the stress tensor, a physical quantity that describes the forces inside a material. Large stresses pull notes into high pitch, lower stresses compress it to lower pitches.

The violent nature of forces experienced by the observer circling around the crack tip generate complex but highly organized melodic forms. As the stress field features universal, respective patterning, the various melodic lines generated have relationships rooted in the nanoscale physics surrounding the singularity.

Fracture-generated counterpoint hence may reflect the universal features of fractures and provides a model thereof. Owing to the universal significance of fracturing as a process in material and amaterial manifestations, the musical effects generated hereby can be an interesting subject of further analysis and inspiration.

[1] M. Buehler, Atomistic Modeling of Materials Failure, 2008 www.springer.com/gp/book/9780387764252

Counterpoint Through Fracture: Part II - Stress Tensor Variations

This piece is a reprise of the motifs presented in Part I, now set into a more complex musical arrangement, and with more rhythmic guidance. 

As the other compositions in this series, this part is created based on the universal stress field of a crack, exploiting the unique features of the singular stress tensor to derive melodic and counterpunctual content through overlapping stress fields. 

GENERAL FEATURES: FRACTURES ARE UNIVERSAL 

A fracture is a crack in a material (often seen in glass, but also represented in earthquakes when tectonic plates fracture, or in the unfolding of a protein). Fracturing is the process of destruction of structure, and hence is the division between function and failure in material and has correspondences in philosophy. The tipping point of the crack, the locality of greatest action, and where bond breaking occurs, is the singular crack tip, the sharp edge. 

All fractures feature a universal distribution of stresses – the forces that act inside the material and which stretch/compress chemical bonds.  No matter what material, as you get closer to the singularity – the tip of the crack – the distribution of forces are identical. 

Here we explore these universal features to generate algorithmically composed music. We try to explore whether the universal features of fractures have the capacity to generate counterpoint – universal features of music, to better understand the analogy across manifestations. 

Read more about the physics of fracture in [1].

HOW THE MUSIC IS CREATED

A set of Euclidean rhythmic motifs (with sparsity: 1/3, 5/12, and 7/12 to reflect different atomic organizations in crystalline forms) are acted upon based on the principal stresses near a crack under tensile loading.  We use the singular principal stress field to drive melodic content formation, whereas several independent moving observers assess the local principal stresses. The observes move in circular fashion, towards and away from the singularity in a spiral, oscillatory motion (see track image).  Large stresses pull notes into high pitch, lower stresses compress it to lower pitches.  These motions provide the basis for the music you hear. 

The violent nature of forces experienced by the observer circling around the crack tip generate complex but highly organized melodic forms. As the stress field features universal, respective patterning, the various melodic lines generated have relationships rooted in the nanoscale physics surrounding the singularity. 

Fracture-generated counterpoint serves as a means to explore the universal features of fractures, and provides a model thereof. Owing to the universal significance of fracturing as a process in material and amaterial manifestations, the musical effects generated hereby can be an interesting subject of further analysis and inspiration. 

[1] M. Buehler, Atomistic Modeling of Materials Failure, 2008 https://www.springer.com/gp/book/9780387764252

Counterpoint Through Fracture: Part III - Spiral Roosters

This piece is an illustration how fractures can be used to design sound, here realized in a fracture drone. The piece is constructed from a single sine wave, by using several independent movers in the singular fracture stress field (see more details on the method in the description of Parts I and II). 

All 9 movers live within the fracture stress field and move stochastically (in angle, distance to singularity, stress intensity factor).  Additional harmonic content is derived from the melodies constructed in Part II. 

FRACTURES ARE UNIVERSAL

A fracture is a crack in a material (often seen in glass, but also represented in earthquakes when tectonic plates fracture, or in the unfolding of a protein). Fracturing is the process of destruction of structure, and hence is the division between function and failure in material and has correspondences in philosophy. The tipping point of the crack, the locality of greatest action, and where bond breaking occurs, is the singular crack tip, the sharp edge.

All fractures feature a universal distribution of stresses – the forces that act inside the material and which stretch/compress chemical bonds. No matter what material, as you get closer to the singularity – the tip of the crack – the distribution of forces are identical.

In this composition we explore these universal features to generate algorithmically composed music. We try to explore whether the universal features of fractures have the capacity to generate counterpoint – universal features of music, to better understand the analogy across manifestations.

Read more about the physics of fracture in [1].

[1] M. Buehler, Atomistic Modeling of Materials Failure, 2008 www.springer.com/gp/book/9780387764252

Counterpoint Through Fracture: Part IV - Fracture Drone

This piece is a sonification of the spike protein of the pathogen of COVID-19, 2019-nCoV (protein data bank identifier 6VSB [1]). What you hear is a multi-layered sonification approach featuring both the vibrational spectrum of the entire protein (expressed in sound and rhythmic elements), as well as the sequence and folding of amino acids that compose the virus spike structure [2]. 

References
[1] Wrapp et al., "Cryo-EM structure of the 2019-nCoV spike in the prefusion conformation," Science, 2020, DOI: 10.1126/science.abb2507
[2] Buehler et al., "A Self-Consistent Sonification Method to Translate Amino Acid Sequences into Musical Compositions and Application in Protein Design Using Artificial Intelligence," ACS Nano, 2019, DOI: https://pubs.acs.org/doi/10.1021/acsnano.9b02180

Sonification of a virus spike protein

Sonification of the Coronavirus Spike Protein (Amino Acid Scale)

While we cannot see small nanoscopic objects like proteins or other molecules that make up virtually all living matter including our cells, tissues, as well as pathogens such as viruses, our computational algorithm allows us to make its material manifestation audible.  This piece is a musical representation of the amino acid sequence and structure of the spike protein of the pathogen of COVID-19, 2019-nCoV (protein data bank identifier 6VSB [1]). 

MUSICAL DECEIT

A virus’ genome hijacks the host cell’s protein manufacturing machinery, and forces it to replicate the viral genome and produce viral proteins to make new viruses from it.  This musical art teaches us something about the fine line between beauty of life and death as an opposite pole.  As you listen to the protein you will find that the intricate design results in incredibly interesting and actually pleasing, relaxing sounds. This doesn’t really convey the deadly impacts this particularly protein is having on the world. This aspect of the music shows the deceiving nature of the virus, how it hijacks our body to replicate, and hurt us along the way. So, the music is a metaphor for this nature of the virus to deceive the host and exploit it for its own multiplication. 

HOW IT IS DONE: 

What you hear is a multi-layered algorithmic composition featuring both the vibrational spectrum of the entire protein (expressed in sound and rhythmic elements), the sequence and folding of amino acids that compose the virus spike structure, as well as interwoven melodies - forming counterpoint music - reflecting the complex hierarchical intersecting geometry of the protein [2].

Scientific references:

[1] Wrapp et al., "Cryo-EM structure of the 2019-nCoV spike in the prefusion conformation," Science, 2020, DOI: 10.1126/science.abb2507

[2] Buehler et al., "A Self-Consistent Sonification Method to Translate Amino Acid Sequences into Musical Compositions and Application in Protein Design Using Artificial Intelligence," ACS Nano, 2019, DOI: pubs.acs.org/doi/10.1021/acsnano.9b02180

Viral counterpoint of the coronavirus spike protein (2019-nCoV)

Viral Counterpoint of the Coronavirus Spike Protein (2019-nCoV)

Musical representation of de novo protein designed using artificial intelligence, and folded and equilibrated using molecular dynamics, as reported in [1]. The musical composition renders the 3D structure of this alpha-helix rich protein in an audible space, presenting complex melodic and rhythmic patterns that reflect the hierarchical design of this protein. This is a de novo protein, in other words, a protein that does not exist in nature, but which was designed through our AI algorithm that generated new music, then translated back into a protein amino acid sequence. 

[1] C.H. Yu, M.J. Buehler, "Sonification based de novo protein design using artificial intelligence, structure prediction and analysis using molecular modeling," APL Bioengineering, DOI: 10.1063/1.5133026, 2020.

Concerto of a De Novo Protein Designed using Artificial Intelligence

This is a sonification of a protein extracted from honeybees, and folded using a deep learning method described in [1].

BACKGROUND:

The development of rational techniques to discover new mechanically relevant proteins for use in variety of applications ranging from mechanics, agriculture to biotechnology remains an outstanding nanomechanical design problem. The key barrier is to design a sequence to fold into a predictable structure to achieve a certain material function. Focused on alpha-helical proteins (as found in skin, hair, and many other mechanically relevant protein materials), we report a Multi-scale Neighborhood-based Neural Network (MNNN) model to learn how a specific amino acid sequence folds into a protein structure. 

The algorithm predicts the protein structure without using a template or co-evolutional information at a maximum error of 2.1 Å. We find that the prediction accuracy is higher than other models and the prediction consumes less than six orders of magnitude time than ab initio folding methods. We demonstrate that MNNN can predict the structure of an unknown protein that agrees with experiments, and our model hence shows a great advantage in the rational design of de novo proteins.

Reference: Z. Qin, B. Marelli, M.J. Buehler, et al., "Artificial intelligence method to design and fold alpha-helical structural proteins from the primary amino acid sequence," Extreme Mechanics Letters, Vol. 36, p. 100652, 2020 https://doi.org/10.1016/j.eml.2020.100652

Honeybee Silk

This piece explores melodic and rhythmic content from a variety of protein structures, including the binding event of COVID-19 to human cells. The progression models pre-pandemic humanity and the virus alone ("growth"), then the virus interacting with human cell - a struggle modeling repeated infection of new cells ("infection") - and finally back to virus alone overlaid with the regular rhythm of human life ("release").  As expressed musically, life after the virus is different than before , since we all changed due to the pandemic. How, we do not know yet. 

LYRICS:

Growth, Infection and Release.
Growth represents prepandemic humanity and the virus alone.
Infection represents the period during with the virus is interacting with human cells.
There is a struggle between infection of new cells and human defenses.
Finally, we return to the regular rhythm of human life, the release.
Life after the virus is different than before. We all changed due to the pandemic.
How we have changed, we do not know yet.

MORE DETAILS:

The rhythm is derived directly from the vibrational spectrum of the virus spike protein attached to the human ACE2 receptor, scaled to the frequencies that human ears perceive as rhythm. 

The construction of this piece involves the vibrational frequencies of virus proteins alone, melding into overlays of the virus with human proteins attached, peaking and then decaying as the virus releases its grip on humanity.  The piece ends with the steady motions of human existence, as we enter release.

The entire composition is written in the amino acid scale, making molecular vibrations audible in a range of experiences from rhythms, to harmony, to melodies. For sources of sonic content, we explore a variety of protein structures - virus spike protein alone, spike protein attached to human cell, human cell alone, and various states of the human cell alone. These provide the melodic foundation on which the progression is based. 

The track image is taken from [1], a recent study to make protein vibrations visible via water waves.

[1] M. Buehler, Liquified protein vibrations, classification and cross-paradigm de novo image generation using deep neural networks, https://arxiv.org/abs/2004.07603, 2020

Growth, Infection and Release

A musical experiment with proteins, coyote, bells, guitar, and a rooster. This is an audible journey through a variety of natural sources of sound, invoking various forms of life, materials, as well as electronic synthesis methods.

Typically, sound is made from vibrating a material, like a guitar string, and music is made by arranging sounds in hierarchical patterns. Here we explore a variety of modes of sound generation - from string vibrations (guitar), to protein vibrations (molecular music), to animals (coyote and rooster), to bells, and a few other unusual sources. In one sweep, our ears pick up all of the features a complex hierarchical system: pitch, timbre, volume, melody, rhythm, and chords.

Music, as a universal language, connects people across the world. More so, it allows us to explore the physical features of matter - both living systems and inanimate - in their ability to generate sound through energetic excitation. Hence, it is a model of our shared humanity. And we can cross scales and species, as done in this piece, where we visualize distinct organisms like animals, materials, and humans interacting in a sort of virtual jam, in musical space.

Energy Equilibrium: Proteins, Coyote, Bells, Guitar, and a Rooster (((Ambient Version)))

Protein resonances inspire a minimalist evolution of ideas across the resonances of piano and solo strings. The structure evolves slowly as the motion of jellyfish, a natural hydrogel, in water. Ethereal resonance.

Ethereal Resonance: Protein Inspiration - VII(PIano and Cello)

Superfluous Dreams (Violin Adagio) - Part X

The construction of biological matter, and life itself, features striking similarities with musical creativity. In my research, I have explored these relationships with a combination of biology, chemistry, as well as mathematical and music theoretical tools. 

Listen here how dinoflagellate (a single-celled organism with two flagella, occurring in large numbers in marine plankton and also found in fresh water)  induce hierarchical vibrations translated to musical form.

Superfluous Dreams (Violin Adagio) - Part X

Graph Sequence of a Protein Fold - Part I: Initiation Movement (C minor)

Musical model of the COVID-19 antibody interacting with the tip of the spike protein. Composed by MIT professor Markus Buehler, arranged by In-Kyu Kim and Jae-Yoon Lim.  In collaboration with Hyung Joon Won and the Lindenbaum Festival Orchestra who premiered this piece on March 7, 2021.

Protein Antibody in E minor

Graph Sequence of a Protein Fold - Part II: Vibrational Dance (C major)

Graph Sequence of a Protein Fold - Part III: Equilibration Loops (D minor)

Spiders are intriguing species that merge their own body’s proteins with an extension into their extraorganismal web, fine-tuned as a vibrational sensor. This piece is an exploration of the vibrations of a spider’s web in the context of protein structures of silk threads and spider venom, creating a meditative trap within the strange world of the spider’s vibrational universe.  Albeit this is a weirdly world, and as we feel the entrapment in the entanglement of the web, and the venom as the structure closes in, it offers a counterpoint to our own reflection of time.       

All sounds originate from the natural tuning of spider webs, as well as the content extracted from venom proteins, forming an entanglement of structure and sound, where time is morphed into a state of wider consciousness, a microscope into the cosmic depth of webs that can be interpreted not solely as a physical entrapment of a web, but also as a virtual entrapment as a state of mind, or our experience of time.  

The piece never achieves escape in spite of the ever changing, evolving landscape that captivates and traps.  

BACKGROUND:

This piece explores spider venom proteins within the transient traps of a spider’s web. A multiscale exploration of two distinct poles of structural exploration of spider silk music, featuring both an audible reflection of the web structure and the proteins that live within the web, forming a family of venom proteins from the recluse spider family. The structural similarity of the venom proteins with distinct mutations and variations leads to musical intersections of melodies and rhythms that are ever changing.  

The musical structure reflects several distinct webs – 1) the silk-based web of the spider - a dense and a porous web, whereby a player travels within each of these webs. Four web movers are implemented to create interwoven soundscapes.  2) a representation of the venom vibrations of the multiple scales of the protein’s structures from their amino acids to structural scales.

REFERENCES:

[1] Sonification method - see: I. Su, T. Saraceno, E. Ziporyn, M. Buehler, et al., in revision, 2021
[2] Background on spider venoms: M.A. Coronado, RK. Arno et al., Curr. Prot. Pept. Sci., 2015

An extension of collaborative work with Ian Hattwick, Isabelle Su, Christine Southworth, Evan Ziporyn, Tomas Saraceno, and Markus Buehler; first premiered at Saraceno's OnAir exhibition, Paris, 2018

Spider Venom: Transient Traps

Markus J. Buehler is the McAfee Professor of Engineering at MIT, and a composer of experimental, classical and electronic music, with an interest in sonification. Using an approach termed "materiomusic", his artistic work explores the creation of new forms of musical expression - such as those derived from biological materials and living systems - as a means to better understand the underlying science and mathematics. One of his goals is to use musical and sound design as a novel and abstract way to model, optimize and create new forms of matter from the bottom up, and to assess cross-system design relationships. He is also interested in research to explore relationships between classical music, mathematics, and the physical and biological sciences, an in the mapping of models of consciousness across systems. In recent work he has developed a new framework to compose music based on proteins – the basic molecules of all life, as well as other physical phenomena such as fracturing, to explore similarities and differences across species, scales and between philosophical and physical models. 

Contact: mbuehler@MIT.EDU

Markus J. Buehler

Pinned to spotlight

Popular tracks