Sonifying Science: Listening to Cancer

SCIENCE AND ART have been lifetime passions of mine, and although I have written much more about the former, I have returned to the practice of doing art again in my later life. In the case of science, I have focused upon its material embodiment in technologies, primarily instruments, and drawn from a long praxis set of traditions, particularly phenomenology and pragmatism, which I call postphenomenology. I have long argued that without technologies—instruments—there could be no science. And as I have returned to doing art, I realize the same is the case for art practices.

In recent work, I have taken a long, deep historical look at both art and science practices with respect to tool kits and somewhat arbitrarily begun my narratives with the Ice Ages. I now realize that a kind of archeo-art and archeoscience, both practices with tool kits, go at least this far back. Indeed, what is clearly an art tool kit has been found and dated back to 100,000 BP (BP is a science dating convention meaning “before present”), well before the Ice Age, in Blombos Cave in South Africa. It is a paint-mixing kit that uses a giant sea snail shell as the container, ground up charcoal, red and yellow ocher, bone marrow, and other liquid to make a black/red/yellow palette. It is not clear what types of art may have been produced, but small shells with holes and coloring indicate at least necklace making. Similar finds, dated 75,000 BP, also show color applications to various artifacts. Of course, by the early Ice Ages, 45,000 to 15,000 BP, the proliferation of sophisticated cave paintings is well known.1

What of science, or what I shall call archeoscience? While nothing that goes back 100,000 years is yet known, by mid–Ice Age, artifacts such as reindeer antlers, bone, and stone items depict marks clearly indicating lunar calendars 36,000 to 22,000 BP. It is well known that our ancestors, pretty much all over the world, were keen observers of the nighttime skies and quantified the movements of various celestial objects to produce lunar and solar calendars, regularized the solstices, named constellations, and used both a recording technology (antlers, bones, stones incised) and standardizing observational devices (gnomons, stone rings, sighting constructions) to regularize this knowledge. In short, Ice Age art and Ice Age science were “technologically embodied.”

Of course, as the new philosophies, anthropologies, and sociologies of science recognize, all this practice was culturally embedded in a wider lifeworld of human activity. For hunter-gatherers and very early societies, migration, navigation, and other regularities were needed to stabilize life; for later agricultural societies, crop planting and animal breeding times were needed and knowledge of such natural regularities fit those needs as well. In my own earlier career, I made much about navigational knowledge as well, as applied to both sea voyaging or migratory pathways.

Ancient art and science clearly had to rely on the acuity but also the limits of human perception, which took different shapes in relation to different environments. The astronomy of prehistoric and early history was limited to “eyeball” observation. What marked what we today call “early modern science” was an optical-technological revolution. Galileo used the newly invented telescope (but also a microscope), which opened up macro- and microrealms of extended perception never previously experienced. Yet optical technologies were actually first more employed in art prior to science.2 The Renaissance artists used the camera obscura, camera ludica, and various grip frame devices to produce what we today call “Renaissance perspective” and other verisimilitude effects in art productions. Galileo’s helioscope for viewing sun spots was a century and a half later.

If “art preceded science” in the use of optic technologies in Renaissance times, then it was paralleled by an equal proliferation of acoustic technologies. All our “classical” instrument types were also invented in this period: violin, cello, sackbut (trombone), cornet, et cetera. And as with the helioscope, Galileo also adapted parts of musical instruments and practices into his science investigations: frets for measurement of inclined plane motions, pulse for timing of the Pisa pendulum observations, and so on. (Note in passing that his father was the inventor of Italian opera and a musician.)

A cultural question arises, however, concerning what becomes a different set of trajectories in art and science. If, as so many of our histories and philosophies of science would have it, physics and astronomy were the most refined of early modern sciences, then the use of optical technologies as favored instruments skews those sciences in a visual direction (no early acoustic technologies were relevant to astronomy until radio astronomy in the twentieth century). This may relate to what becomes an increasingly visualist preference in science practice. And, of course, the invention of early modern anatomy fits here as well, with da Vinci and Vesalius and their exploded diagram drawings, a visualist style from the Renaissance on. By late modern times, what I have called a sophisticated visual hermeneutics had become virtually the standard for science depiction—see my Expanding Hermeneutics: Visualism in Science (1998) for an expanded discussion of this cultural contingency in science.

To this point, from Ice Age art to the Renaissance, one might say that art precedes science in the invention and use of technologies-tools-instruments. By late modernity, which in my estimation begins in the nineteenth century with the arrival of much more complex mega- and microtechnologies, there is a shift in which science begins to develop and use technologies that only later are adapted to art. And with this shift—if I am right—there is a reversal of discovery and use such that often art, later than science, opens up a technology to scientific practice. For example, one of my most recent venues was the sixth Computer Art Congress. Artists did not invent computers, although they did invent many Renaissance optics. I will here focus on what I am broadly calling the sonification of science and acoustic technologies. As I pointed out in Expanding Hermeneutics: Visualism in Science, since the seventeenth century, science in practice has favored vision as a sort of cultural choice and has developed, particularly in imaging technologies, a very sophisticated visual hermeneutics. However, recently and often due to scientist-musician and other artist hybrid practitioners, sonification has begun to emerge as a major interest in science imaging.

Sonar, Radar, and Early Sonification

In ordinary human experience, we are familiar with echoes, the sound of our voice, and a musical instrument “returning” in a mountain setting. But it was not until the nineteenth-century discovery of what we now call the electromagnetic spectrum (EMS) that science found a very wide range of wave phenomena that included what we experience but that also far exceeded what we directly could experience. Radar and sonar are “echo-phenomena” but in their instrumental incarnations utilize wave phenomena that we cannot directly experience—but that, through technological mediation, we can again experience.

Radar and sonar were both early twentieth-century technologies that worked like superecho devices, and their most familiar early use was military, to detect airplanes (radar) and submarines (sonar). Active forms used short, sound-like bursts that would return, echo-like, to a receiver. The receiver could actually be acoustic—one could hear the “pings”—or could be made visual on a video display screen. As common knowledge has it, World War II might well have had a very different outcome had the United States and United Kingdom not had superior radar-sonar technologies. Of course today both systems are vastly improved and used for many kinds of detection, mapping, and imaging (see my Acoustic Technics, 2015). As an aside, I note again that, for astronomy, the invention of radio astronomy was the first breakthrough from the optical spectrum, and it is now used to detect radiation not only from dark or nonoptical parts of the sky but to detect the background radiation of the entire universe. Thus, unlike early modern science, today we can both “see” and “hear” the heavens.

Computers and Data-Image Inversions

Above I have hinted that the discovery of the EMS was a major science breakthrough of the nineteenth century. The twentieth-century counterpart, I would contend, is the invention of the computer and its digitalization that drives so many of our practices, both in art and science, today. I shall focus on its capacity to invert data and image. We denizens of the twentieth and twenty-first centuries are all familiar with solar system exploration, one practice that heavily uses the data-image inversion of computer digitality. Cassini is a space probe that has been dedicated to imaging the rings, moons, and other phenomena of our sister planet Saturn. The probe, once in orbit around Saturn, could aim its cameras, radar, or whatever else to the rings. But such dramatic 3-D images are not directly sent back for us to see. Rather, these visual gestalts are first transformed into the familiar binary zeroes and ones of data. These data streams, popularized in such movies as The Matrix, are sent by radio processes to a home station where they are again transformed back into the dramatic images we see of Saturn’s rings, moons, and so on. This inversion process is used in many, many applications and is a taken-for-granted capacity of what I call postmodern imaging. Today’s imaging is thus both compound—images, devices, plus computer tomography—and translational—data into image, image into data.

The Artistic Innovation

What follows is somewhat of a caricature, but it has actually happened. One might say that in a normal science, one could take for granted that data-image inversions are simply translations from data to visual images. And of these, we have multitudinous examples. Let us turn now to more medical examples. If we are looking for a brain anomaly, we use MRI, CT scans, PET scans, or fMRI, all visualizations of the tumor or malignancy that then gets tomographically imaged as a 3-D model. Pap smears image on a slide cells, healthy or malignant, in order to detect a cancer in time for treatment. There is a similar process for prostate cancer in males.

I now turn anecdotal. Many know that for several decades I have been especially interested in auditory, acoustic phenomena, and on many lecture trips here is what happens: I introduce sound bites of what I take to be innovative and interesting sounds via a CD and player. When I do this, quite frequently someone in the audience is an actual performance or other artist-practitioner and will present me with an example, say a CD, of his or her own art product. One of my favorite examples is a CD titled Ground Station.3 I shall describe it. When I play the bite, what one hears is a very narrow kind of minimalist digital piano music. The songline consists of a very few notes that sound in what seems to be a random order, reminiscent of a Philip Glass or Steve Reich piece. And it is obvious that the few notes played repeat themselves. What produces this “song,” however, is very complex. It is a translation of data that comes from a geostationary satellite such as those that beam the data to run GPS instruments in cars. The voice and visual display in your car, however, do not come from this data stream. Instead, they come from a data stream that corrects satellite wobble. For, like the earth or in fact any object in orbit, motion is not actually smooth but filled with minute wobbles that, if not corrected, would yield a confused pathway for the GPS in your car.

This acoustic result is one of my favorite art examples that can be used to show the culturality of science’s visualism. Any musician, even short of a perfect-pitch listener, could quickly be taught to recognize what each note means vis-à-vis the position in a wobble of the satellite. The listener could “hear” the angle. Now I shall turn to a similar example, which I call “listening to cancer.” I have not been able to discover who was the first person to recognize that sonification could be used to do science. It is clear that a number of performance artists did discover that in the data-image inversion process, one could turn data either or equivalently into visual or acoustic images. But in the articles that first pointed out the data to acoustic image inversions, it turned out to be artistically or musically trained scientists who made the switch.

Data to Acoustic Imaging

Sonification in science, especially in this era of high-tech imaging instruments, is relatively new and often accidental. But it is occurring with increasing frequency and with often astonishing and innovative results. A recent “Science and Technology” article in the Economist (March 19, 2016), “Now Hear This,” points to this sporadic history. During World War I, Heinrich Barkhousen used a modified radio-like device to try to listen in to Allied communications—instead he got lots of static that he eventually realized came from lightning storms in the distance.4 Others have pointed out that this would eventually play a role in radio astronomy as noted above. By mid-century, seismologists realized that if they sped up sound data, they could better discern pre-earthquake patterns. As noted above, space explorations like Cassini, when sonified, created hailstorm-like sounds later identified as debris from space hitting the rings.5 Later in the article, one Robert Alexander is cited for his sonification of sun flares to which he listens.6 His name immediately rang a memory bell for me since an earlier (March 2015) Scientific American article “Sound Bytes,” had identified him as a pioneer of science sonification.7 This was the article that first described “listening to cancer.”

As it turns out, Robert Alexander is by vocation both a scientist and an artist; he is a composing musician and was previously a graduate student at the University of Michigan.8 But, like so many new practices in technoscience, sonification has a complex and growing development. Anecdotally, often when I claim that science’s visualism is culturally contingent, I get objections from scientists in the audience, and one line of objection claims that we humans are physiologically-neurologically-visually superior. So it is not surprising that the current focus upon THE BRAIN, or its neurology, should be used in this more contemporary context to counter the older visualist claims.

“Sound Bytes” does just that: “Our ears can detect changes in a sound that occur after just a few milliseconds.” This is a claim by Andrew King, Oxford University neurologist, who goes on to state that by comparison the eye’s limit for detecting a flickering light is about fifty to sixty times a second.9 Another neurologist, Bechara Saab of the Neuroscience Center Zurich, claims more generically that a mammal’s “auditory system is faster at transmitting neural signals than most other parts of the brain. This system holds the largest known connection between neurons, a giant synapse called the calyx of Held. The calyx can release neurotransmitters eight hundred times a second. In contrast, the visual pathway does not have such a speedy neural connection.”10 While I do not claim expertise in this domain, and I definitely do not want to substitute an aural bias for a visual one, the new neurologies are of interest in this scientific-cultural battle.

Now I shall turn to the acoustical diagnosis of cancer—listening to cancer. The current, dominant diagnosis, and I take here cervical cancer as my example, begins with a biopsy from a Pap smear sample made into a slide and visually analyzed by an expert reader of slides, and eventually the result is reported to the patient. One problem, of course, with this type of diagnosis—and with many medical tests—is the delay time that stimulates a period of anxiety for the patient. This delay problem was noted by yet another scientist-artist, Ryan Staples in the United Kingdom, who imagined a much faster process if the diagnosis could be sonified. After some experimentation, the process known as Raman spectroscopy was chosen. Laser light is shown onto molecules, causing vibrations, and, as it turns out, the vibrations differ between healthy and malignant cells and this difference can be heard. We can listen to cancer. And, with development, this process could, ideally, be set up in a physician’s office. Another anecdote: I described this process for the first time in Toronto, Canada, and in the audience there was a postdoctoral student who was, in fact, undergoing training in listening to cancer. We had a serious discussion after. There are two companies in Canada developing the process and, as the Scientific American article also notes, there is some concern about how deeply entrenched the visualist approach may be, which in turn may complicate the introduction of a new process. “Although sonification offers advantages over visual display, [science sonifiers] face a major hurdle: simply getting researchers to try this new way of exploring data.”11 For, in spite of studies that show that training listeners to discriminate healthy from malignant cells can be done to achieve 95 percent accuracy in an hour’s intense training, the long experience of visualism remains.12

So far a sonification diagnosis has been limited to specific types of cancer, such as cervical and prostate, but detection of much harder to find bloodstream cancers is being investigated. It is my hope that science and art practices could, and should, enrich each other and, in keeping with the whole-body emphasis of postphenomenology, equally enrich human experience overall.

I add as a sort of postscript that many of the late-life medical procedures I have experienced were sonograms, particularly used by urologists but also for other arterial and esophageal areas.