Chapter 3: Digital Expression
Originally published by Henry Holt and Company 1999. Published on KurzweilAI.net May 15, 2003.
A few years ago I found myself on the stage of one of Tokyo's grandest
concert halls. I wasn't going to perform; Yo-Yo Ma was, if I could
fix his cello bow in time. The position sensor that had worked so
well at MIT was no longer functioning after a trip around the world.
To a casual observer, Yo-Yo's instrument looked like a normal cello
that had lost a few pounds. The one obvious clue that something
was out of the ordinary was the racks of computers and electronics
behind it. These made the sound that the audience heard; the cello
was really just an elaborate input device.
Sensors measured everything Yo-Yo did as he played. A thin sandwich
of flexible foam near the end of the bow detected the force being
applied to it, and a rotary sensor measured the angle of his wrist
holding the bow. Strips under the strings recorded where they contacted
the fingerboard. Polymer films measured how the bridge and top plate
of the instrument vibrated. In a conventional cello these vibrations
get acoustically amplified by the resonant cello body that acts
like a loudspeaker; here they were electronically amplified by circuits
in the solid body. The cello itself made no audible sound. A small
antenna on the bridge sent a radio signal that was detected by a
conducting plastic strip on the bow, determining the position of
the bow along the stroke and its distance from the bridge.
Earlier that day we had spent a few hours unpacking all of the
shipping crates, attaching the many cables, and booting up the computers.
Everything was going fine until we tried the bow. The position sensor
that I had developed to measure its position was erroneously indicating
that the bow was fluctuating wildly, even though it wasn't actually
moving. Fortunately, I had packed a small laboratory of test and
repair equipment. Given the amount of new technology being used
in a hostile environment (a concert hall stage), something unexpected
was certain to happen. Probing the signals let me narrow the problem
down to the fine cable coming from the bow antenna. Opening up its
protective insulation and shielding exposed an erratic connection,
which I could patch around with a bit more surgery on the inner
conductor. After I packed everything back up, the signals checked
out okay, and I could resume breathing.
There was an expectant buzz in the full hall as we connected the
computers, the cello, and Yo-Yo. The second unexpected event of
the evening came as I began to make my way back to the control console
where we were going to do some final system checks. I was only a
few feet into the audience when I heard the unmistakable sound of
Yo-Yo Ma controlling racks of electronics. The concert had begun.
As some newfound friends made room to squeeze me in, I felt the
way I imagined I will feel when my children leave home. The instrument
had become his more than ours; he no longer needed us. As he played,
the hall filled with familiar cello lines and unfamiliar sonic textures,
with recognizable notes and with thick layers of interwoven phrases,
all controlled by him. I sat back to enjoy the performance as a
listener, no longer an inventor.
I was equally surprised and delighted to be there. This project
was both an artistic and a technical experiment, asking how the
old technology of a cello could be enhanced by the new technology
of a computer. Yo-Yo's real allegiance is to communicating musical
expression to a listener; the cello is just the best way he's found
to do that. By introducing intelligence into the instrument we could
let him control more sounds in more ways without sacrificing the
centuries of experience reflected in its interface and his technique.
This was an ideal way to explore the role of technology in music:
there was an enormous incentive to succeed, and it would be easy
to tell if we didn't.
My contribution had begun years earlier with my love/hate relationship
with bassoon reeds. As an enthusiastic amateur musician I was just
good enough to be able to get hopelessly out of my depth. One of
my perennial stumbling blocks was reed making. To create a reed,
cane from a particular region in France is seasoned, shaped, cut,
trimmed, wrapped, sealed, shaved, tested, aged, played, shaved some
more, played again, trimmed a bit, on and on, until, if the reed
gods smile on you, the reed finally speaks over the whole range
of pitch and volume of the bassoon. Almost imperceptible changes
in the thickness of the reed in particular places can have a salutary
or devastating effect on the color of the sound, or the dynamics
of the attacks, or the stability of the pitch, or the effort required
to make a sound. The reward for all of this trouble is playing one
of the most expressive of all musical instruments. The same sensitivity
that makes it so difficult to put together a good reed also lets
the reed respond to subtle changes in the distribution of pressure
and airflow as it is played. When it works well I have trouble telling
where I stop and the instrument begins.
For a few weeks, that is, until the reed becomes soggy and the
whole process starts all over again. The cane softens, cracks, and
wears away, reducing the reed to buzzing noodles. This is why I
was ready to embrace the convenience of electronic music synthesizers
when I first encountered them. I was a graduate student at Cornell
working on experiments in the basement of the Physics building,
crossing the street to go to the Music building to play my bassoon
for therapy. I didn't realize that I was literally following in
the footsteps of Bob Moog.
A few decades earlier he had walked the same route, leaving his
experiments in the Physics building to invent the modern synthesizer.
He was struck by a parallel between the human ear and the transistor,
which was just becoming commercially available. Both devices respond
exponentially rather than linearly. This means that each time the
amplitude or frequency of a sound is doubled, the perceived intensity
or pitch increases by the same fixed amount. A low octave on a piano
might span 100 to 200 Hz; another set of twelve keys farther up
the keyboard might go from 1000 to 2000 Hz. The same thing is true
of the current flowing through a transistor in response to an applied
control voltage. This is what let Bob make circuits that could be
played like any other musical instrument.
Moog began building modules with transistor circuits to control
the frequency of oscillators, the volume of amplifiers, or the cutoff
of filters. Patching together these modules resulted in great sounds
that could be used in musically satisfying ways. As Bob's musical
circuits grew more and more capable, he spent less and less time
in the physics lab, until his advisor got rid of him by giving him
a degree. This was fortunate for the rest of the world, which was
transformed by the company Bob then founded. The progeny of the
Moog synthesizer can be heard in the electronic sounds in almost
any popular recording.
At Cornell this legacy came to life for me when I encountered one
of Bob's early synths still in use in the basement of the Music
building. It belonged to David Borden, the composer who had used
Bob's original synthesizers to start the first electronic music
ensemble. David also had a useful talent for breaking anything electronic
by looking at it; when one of Bob's modules could survive the Borden
test, then it was ready to be sold.
David was in the process of updating Cornell's studio to catch
up to the digital world. The studio had languished since Bob's time
as the department specialized in the performance practice of early
acoustic musical instruments, leaving it with the earliest electronic
ones also. As David filled the studio with the latest computerized
synths, I found it liberating to come in and push a button and have
the most amazing sounds emerge. Even better, the sound was there
the next day, and the next month; I didn't have to worry about it
becoming soggy.
I quickly realized that was also the problem. The nuance that made
my bassoon so beloved was lost in the clinical perfection of the
electronic sounds. Unlike the bassoon, there was no ambiguity as
to what part I did and what part it did: I pressed the button, it
beeped. I began to give up on the technology and go back to my real
instrument.
As I walked back and forth between the Music building and the Physics
building I realized that I was doing something very silly. A bassoon
is governed by the same laws of physics as the rest of the universe,
including my physics experiments. It may be a mystery why we like
to listen to a bassoon, but the equations that govern its behavior
are not. Perhaps I could come up with a set of measurements and
models that could reproduce its response, retaining what I like
about the bassoon but freeing me from the Sisyphian task of reed
making.
Directly comparing a bassoon and an electronic synthesizer makes
clear just how impressive the bassoon really is. Even though its
design is centuries old, it responds more quickly and in more complex
ways to finer changes in more aspects of the musician's actions.
It's not surprising that when playing a synthesizer one feels that
something's missing.
From across the street in the Physics department, it occurred to
me that, instead of leaving the lab to make music, I could stay
there and use its resources to update the synthesizer with everything
that's been learned about sensing and computing since Bob Moog's
day. As I began the preliminary analysis that precedes any physics
experiment, I was startled to realize that for the purpose of making
music, computers are beginning to exceed the performance of nature.
There are relevant limits on both the instrument and the player.
Displacements of a cello bow that travel less than about a millimeter
or happen faster than about a millisecond are not audible. This
means that the data rate needed to determine the bow's position
matches that generated by a computer mouse. Even including the extra
information needed to describe the bow pressure and angle, and the
positions of the fingers on the strings, the amount of gestural
input data generated by a player is easily handled by a PC. If we
use the specifications of a CD player to estimate the data rate
for the resulting sound, this is also easily handled by a PC. Finally,
the effective computational speed of the cello can be found by recognizing
that the stiffness and damping in the materials that it is made
of restrict its response to vibrations that occur over distances
on the order of millimeters, at a frequency of tens of thousands
of cycles per second. A mathematical model need not keep track of
anything happening smaller or faster than that. Dividing the size
of the cello by these numbers results in an upper limit of billions
of mathematical operations per second for a computer to model a
cello in real time. This is the speed of today's supercomputers,
and will soon be that of a fast workstation.
The conclusion is that with appropriate sensors, a computer should
be able to compete with a Stradivarius. By the time I appreciated
this I was a Junior Fellow of the Harvard Society of Fellows. The
regular formal dinners of the Society provided an ideal setting
to develop this argument, but not a Stradivarius to try it out on.
That came when I met a former Junior Fellow, Marvin Minsky, at one
of the dinners. He called my bluff and invited me down to the other
end of Cambridge to visit the Media Lab and take the experiment
seriously.
Marvin introduced me to Tod Machover, a composer at the Media Lab.
Starting with his days directing research at IRCAM, the pioneering
music laboratory in Paris, Tod has spent many years designing and
writing for smart instruments. Classically, music has had a clear
division of labor. The composer puts notes on a page, the musician
interprets the shorthand representation of the composer's intent
by suitable gestures, and the instrument turns those gestures into
sounds. There's nothing particularly fundamental about that arrangement.
It reflects what has been the prevailing technology for distributing
music—a piece of paper. Notes are a very low level for describing
music. Just as computer programmers have moved from specifying the
primitive instructions understood by a processor to writing in higher-level
languages matched to application domains such as mathematics or
bookkeeping, a more intelligent musical instrument could let a composer
specify how sounds result from the player's actions in more abstract
ways than merely listing notes. The point is not to eliminate the
player, it is to free the player to concentrate on the music.
Tod had been discussing this idea with Yo-Yo, who was all too aware
of the limits of his cello. While he's never found anything better
for musical expression, his cello can play just one or two notes
at a time, it's hard to move quickly between notes played at opposite
ends of a string, and the timbral range is limited to the sounds
that a bowed string can make. Within these limits a cello is wonderfully
lyrical; Yo-Yo was wondering what might lie beyond them.
As Tod, Yo-Yo, and I compared notes, we realized that we were all
asking the same question from different directions: how can new
digital technology build on the old mechanical technology of musical
instruments without sacrificing what we all loved about traditional
instruments? We decided to create a new kind of cello, to play a
new kind of music.
My job was to find ways to measure everything that Yo-Yo did when
he played, without interfering with his performance. Sensors determined
where the bow was, how he held it, and where his fingers were on
the strings. These were connected to a computer programmed by a
team of Tod's students, led by Joe Chung, to perform low-level calibration
(where is the bow?), mid-level analysis (what kind of bow technique
is being used?), and high-level mapping (what kind of sound should
be associated with that action?). The sounds were then produced
by a collection of synthesizers and signal processors under control
of the computer.
Tod, who is also a cellist, wrote a piece called Begin Again
Again . . . that looked like conventionally notated cello music
(since that is what Yo-Yo plays), but that also specified the rules
for how the computer generated sounds in response to what Yo-Yo
did. Yo-Yo's essential role in the development was to bridge between
the technology and the art. He helped me understand what parts of
his technique were relevant to measure, and what parts were irrelevant
or would be intrusive to include, and he helped Tod create musical
mappings that could use these data in ways that were artistically
meaningful and that built on his technique.
While the physical interface of the cello never changed, in each
section of the resulting piece the instructions the computer followed
to make sounds did. In effect, Yo-Yo played a new instrument in
each section, always starting from traditional practice but adding
new capabilities. For example, an important part of a cellist's
technique is associated with the bow placement. Bowing near the
bridge (called ponticello) makes a bright, harsh sound; bowing
near the fingerboard makes a softer, sweeter sound. In one section
of the piece these mappings were extended so that playing ponticello
made still brighter sounds than a cello could ever reach. Another
essential part of playing a cello is the trajectory of the bow before
a note starts and after it ends. Not unlike a golf swing or baseball
throw, the preparation and follow-through are necessary parts of
the bow stroke. In addition, these gestures serve as cues that help
players communicate with each other visually as well as aurally.
These influences came together in a section of the piece that used
the trajectory of his bow to launch short musical phrases. The location
and velocity of the bow controlled the volume and tempo of a sequence
of notes rather than an individual note. Yo-Yo described this as
feeling exactly like ensemble playing, except that he was the ensemble.
The computer would pick up on his bowing cues and respond appropriately,
in turn influencing his playing.
Developing the instrument was a humbling experience. I expected
to be able to use a few off-the-shelf sensors to make the measurements;
what I found was that the cello is such a mature, tightly integrated
system that most anything I tried was sure to either fail or make
something else worse. Take a task as apparently simple as detecting
the motion of the strings. The first thing that I tried was inspired
by the pickup of an electric guitar, which uses a permanent magnet
to induce a current in a moving string, in turn creating a magnetic
field that is detected by a coil. Since a cello string is much farther
from the fingerboard than a guitar string, I had to design a pickup
with a much stronger magnet and more sensitive amplifier. This let
me make the measurement out to the required distance, obtaining
a nice electrical signal when I plucked the string. When this worked,
I proudly called the others in to try out my new device.
To my horror there was no signal when the string was bowed, even
though I had just seen it working. Some more testing revealed the
problem: bowing made the string move from side to side, but my sensor
responded only to the up-and-down motion of the string that resulted
when I plucked it.
Plan B was to place between the strings and the bridge a thin polymer
sheet that creates a voltage when compressed. By carefully lining
the bridge with the polymer I was able to get a nice measurement
of the string vibration. Pleased with this result, and ignoring
recent experience, I called everyone back in to see it work. And
then I watched them leave again when the signal disappeared as the
cello was bowed. Here the problem was that the bowing excited a
rocking mode of the bridge that eliminated most of the constraint
force that the polymer strips were measuring.
A similar set of difficulties came up in following the movement
of the bow. Sonar in air, sonar in the strings, optical tracking,
radars, each ran afoul of some kind of subtle interaction in the
cello. It was only after failing with these increasingly complicated
solutions that I found the simple trick we used. I was inspired
by a baton for conducting a computer that was developed by Max Mathews,
a scientist from Bell Labs who was the first person to use a computer
to make music. His system measured the position of the baton by
the variation in a radio signal picked up in a large flat antenna
shaped like a backgammon board. This was much too big to fit on
a cello, but I realized that I could obtain the same kind of response
from a thin strip of a material placed on the bow that was made
out of poor conductor so that the signal strength it received from
a small antenna on the bridge would vary as a function of the bow
position.
The most interesting part of the preparations were the rehearsals
at MIT where the emerging hardware and software came together with
the composition and the musical mappings. Each element evolved around
the constraints of the others to grow into an integrated instrument.
When everything was working reasonably reliably, we packed up and
headed off for the premiere performance at Tanglewood in western
Massachusetts. It was hard to miss our arrival when, for the first
time, Yo-Yo went on stage to try out the whole system.
A cello can only sound so loud. Beyond that, players spend a lifetime
learning tricks in how they articulate notes to seem to sound louder
and to better cut through an accompanying orchestra. There's nothing
profound about this aspect of technique; it's simply what must be
done to be heard with a conventional cello. But we had given Yo-Yo
an unconventional cello that could play as loud as he wanted, and
play he did. He unleashed such a torrent of sound that it threatened
to drown out the concert going on in the main shed. This was great
fun for everyone involved (except perhaps the people attending the
concert next door).
A similar thing happened the first time he tried the wireless bow
sensor. Instead of playing what he was supposed to be rehearsing,
he went off on a tangent, making sounds by conducting with his bow
instead of touching the strings at all. After a lifetime of thinking
about the implications of how he moved his bow through the air,
he could finally hear it explicitly.
As the rehearsals progressed we found that eliminating many other
former constraints on cello practice became easy matters of system
design. It's taken for granted that the cellist, conductor, and
audience should all hear the same thing, even though they're listening
for different things, and are in different acoustic environments.
We found that Yo-Yo was best able to play with a sound mix in his
monitor speakers that highlighted his performance cues, different
from the sound mix that filled the hall. Such a split is impossible
to do with an acoustic cello but was trivial with ours.
The biggest surprise for me was the strength of the critical reaction,
both positive and negative, to connecting a computer and a cello.
Great musicians loved what we were doing because they care about
the music, not the technology. They have an unsentimental understanding
of the strengths and weaknesses of their instruments, and are unwilling
to use anything inferior but are equally eager to transcend all-too-familiar
limitations. Beginners loved what we were doing because they don't
care about the technology either, they just want to make music.
They're open to anything that helps them do that. And in between
were the people who hated the project.
They complained that technology was already intruding on too many
parts of life, and here we were ruining one of the few remaining
unspoiled creative domains. This reaction is based on the curious
belief that a computer is technology, and a cello isn't. In fact,
musical instruments have always been improved by drawing on the
newest available technologies. The volume of early pianos was limited
by the energy that could be stored in the strings, which in turn
was limited by the strength of the wood and metal that held them.
Improvements in metallurgy made possible the casting of iron frames
that could withstand tons of force, creating the modern piano, which
is what enabled Beethoven to write his thunderous concertos. Popular
musical instruments have continued to grow and change, but classical
instruments have become frozen along with much of their repertoire.
The only role for new technology is in helping people passively
listen to a small group of active performers.
What our critics were really complaining about were the excesses
of blindly introducing inferior new technology into successful mature
designs. Given that a Stradivarius effectively computes as fast
as the largest supercomputer, it's no wonder that most computer
music to date has lacked the nuance of a Strad. Just as with electronic
books, rather than reject new technology out of hand it is much
more challenging and interesting to ask that it work better than
what it intends to replace.
Here our report card is mixed. We made a cello that let Yo-Yo do
new things, but our instrument couldn't match his Strad at what
it does best. I view that as a temporary lapse; I'll be disappointed
if we can't make a digital Stradivarius in the next few years. That's
admittedly a presumptuous expectation, needing an explanation of
how we're going to do it, and why.
Luthiers have spent centuries failing to make instruments that
can match a Strad. Their frustrating inability to reproduce the
lost magic of the Cremona school has led them to chase down many
blind alleys trying to copy past practice. The nadir might have
been an attempt to use a rumored magic ingredient in the original
varnish, pig's urine. Better results have come from trying to copy
the mechanical function of a Strad instead of its exact design.
By bouncing a laser beam off of a violin as it is played, it is
possible to measure tiny motions in the body. These studies have
led to the unexpected discovery that modes in which the left and
right sides of the instrument vibrate together sound bad, and modes
in which they move asymmetrically sound good. This helps explain
the presence of the tuning peg inside the instrument, as well as
why the two sides of a violin are not shaved identically. Instruments
built with frequent testing by such modern analytical tools have
been steadily improving, not yet surpassing the great old instruments
but getting closer.
Progress in duplicating a Stradivarius is now coming from an unexpected
quarter. Physicists have spent millennia studying musical instruments,
solely for the pleasure of understanding how they work. >From Pythagoras's
analysis of a vibrating string, to Helmholtz's discovery in the
last century of the characteristic motion of a string driven by
the sticking and slipping of a bow (found with a vibrating microscope
he invented by mounting an eyepiece on a tuning fork), to Nobel
laureate C. V. Raman's more recent study of the role of stiffness
in the strings of Indian musical instruments, these efforts have
helped apply and develop new physical theories without any expectation
of practical applications. The work generally has been done on the
side by people who kept their day jobs; a recent definitive text
on the physics of musical instruments starts with a somewhat defiant
justification of such an idle fancy. Computation is now turning
this study on its head.
The world isn't getting any faster (even though it may feel that
way), but computers are. Once a computer can solve in real time
the equations describing the motion of a violin, the model can replace
the violin. Given a fast computer and good sensors, all of the accumulated
descriptions of the physics of musical instruments become playable
instruments themselves. This is why I expect to be able to create
a digital Stradivarius. I'm not smarter than the people who tried
before and failed; I'm just asking the question at an opportune
time.
I also think that I know the secret of a Strad. When we sent the
data from Yo-Yo's sensors to almost any sound source, the result
still sounded very much like Yo-Yo playing. The essence of his artistry
lies in how he shapes the ten or so attributes of a note that are
available to him, not in the details of the waveform of the note.
Much of the value of a Strad lies not in a mysterious attribute
of the sound it makes, but rather in its performance as a controller.
Where a lesser instrument might drop out as a note is released or
waver when a note is sustained, a Strad lets a skilled player make
sharp or soft attacks, smooth or abrupt releases. This translates
into effective specifications for the resolution and response rate
of the interface, something we've already found that we can match.
This is the second reason that I'm optimistic about making a digital
Strad: much of the problem lies in tractable engineering questions
about sensor performance.
For a computer to emulate a Stradivarius it must also store a description
of it. Right now this sort of problem is solved most commonly in
electronic musical instruments by recording samples of sounds and
playing them back. This is how most digital pianos work. The advantage
is that a short segment of sound is guaranteed to sound good, since
it's a direct replica of the original. The disadvantage is readily
apparent in listening to more than a short segment. The samples
can't change, and so they can't properly respond to the player.
The result lacks the fluid expression of a traditional instrument.
An alternative to storing samples is to use a computer to directly
solve a mathematical model of the instrument. This is now possible
on the fastest supercomputers and will become feasible on more widely
accessible machines. Aside from the demand for very powerful computers,
the difficulty with this approach is that it's not much easier than
building an instrument the old-fashioned way. Even if the wood and
strings are specified as software models they must still match the
properties of the real wood and strings. That's exactly the problem
that's remained unsolved over the last few centuries.
A better alternative lies between the extremes of playing back
samples and solving physical models. Just as luthiers began to progress
when they moved their focus from how a Stradivarius is built to
how it performs, the same lesson applies to mathematical modeling.
Instead of trying to copy the construction of an instrument, it's
possible to copy its function. We can put our sensors on a great
instrument and record the player's actions along with the resulting
sound. After accumulating such data covering the range of what the
instrument is capable of, we can apply modern data analysis techniques
to come up with an efficient model that can reproduce how the instrument
turns gestures into sounds. At that point we can throw away the
instrument (or lovingly display it to admire the craftsmanship and
history) and keep the functionally equivalent model. You can view
this as sampling the physics of the instrument instead of sampling
the sound. Once the model has been found, it can be played with
the original sensors, or used in entirely new ways. Soon after we
were first able to model violin bowing with this kind of analysis
I was surprised to come into my lab and see my student Bernd Schoner
playing his arm. He had put the bow position sensor into the sleeve
of his shirt so that he could play a violin without needing to hold
a violin.
Granting then that a digital Stradivarius may be possible in the
not-too-distant future, it's still fair to ask what the point is.
The obvious reason for the effort is to make the joy of playing
a Strad accessible to many more people. There are very few Strads
around, and they're regularly played by even fewer people. Even
worse, the instruments that aren't in routine use because they're
so valuable suffer from problems similar to those of an automobile
that isn't driven regularly. If we can match the performance of
a Strad with easily duplicated sensors and software, then playing
a great instrument doesn't need to be restricted to a select few.
Beyond helping more people do what a small group can do now, making
a digital Stradivarius helps great players do things that no one
can do now. As we found in the project with Yo-Yo, traditional instruments
have many limitations that just reflect constraints of their old
technology. By making a virtual Stradivarius that is as good as
the real thing, it's then possible to free players from those constraints
without asking them to compromise their existing technique.
Making a Strad cheaper, or better, is a worthy goal, but one that
directly touches only the small fraction of the population that
can play the cello. There's a much more significant implication
for everyone else. Right now you can listen to a recording of Bach's
cello suites, or you can play them yourself. It takes a lifetime
to learn to do the latter well, and along the way the suites don't
sound particularly good. But when a Stradivarius merges with a PC
then this divide becomes a continuum. The computer could emulate
your CD player and play the suites for you, or your bowing could
control just the tempo, or you could take over control of the phrasing,
on up to playing them the old-fashioned way.
A wealthy executive was once given the chance to conduct the New
York Philharmonic; afterward when asked how he did, a player commented,
"It was fine; he pretended to conduct, and we pretended to follow
him." That's exactly what a smart musical instrument should do:
give you as much control as you want and can use, and intelligently
fill in the rest.
Now this is an exciting consequence of bringing computing and music
together. It used to be that many people played music, because that
was the only way to hear it. When mass media came along, society
split into a small number of people paid to be artistically creative
and a much larger number that passively consumes their output. Reducing
the effort to learn to play an instrument, and opening up the space
between a PC, a CD player, and a Stradivarius, points to the possibility
that far more people will be able to creatively express themselves.
Improving the technology for making music can help engage instead
of insulate people.
There's a final reason why it's worth trying to make a digital
Stradivarius: it's hard. The most serious criticism of so many demos
of new interfaces between people and computers—say, the latest
and greatest virtual reality environment—is that it's hard
even to say if they're particularly good or bad. So what if they
let you stack some virtual blocks? The situation is very different
with a cello, where the centuries of wisdom embodied in the instrument
and the player make it very easy to tell when you fail. The discipline
of making an instrument good enough for Yo-Yo Ma ended up teaching
us many lessons about the design of sensors and software that are
much more broadly applicable to any interaction with a computer.
Constraints really are key. If you venture off without them it's
easy to get lost. One of the most depressing days I've ever spent
was at a computer music conference full of bad engineering and bad
music. Too many people there told the musicians that they were engineers,
and the engineers that they were musicians, while doing a poor job
at being either. Increasingly, we can make sensors for, and models
of, almost anything. If this kind of interactivity is done blindly,
everything will turn out sounding brown. Technologists should be
the last people guiding and shaping the applications; accepting
the challenge presented by a Stradivarius is one demanding way to
ground the effort.
Toward the end of the project with Yo-Yo, I asked him when he would
be ready to leave his cello behind and use our instrument instead.
His answer was instant. Rather than speculate about the irreproducible
joys of his Strad, he jumped to the practical reality of making
music. He tours constantly and needs to be able to get out of a
plane, open his case, and start playing. Our system took the form
of several cases of hardware tended to by a small army of students,
and it took a few hours to boot up and get working. It's not until
our technology can be as unobtrusive and invisible as that of the
Strad that he could prefer it instead. It must always be available
for use, never fail, and need no special configuration or maintenance.
That imperative to make technology so good that it disappears lies
behind much of the work in this book.
WHEN THINGS START TO THINK by Neil Gershenfeld. ©1998 by
Neil A. Gershenfeld. Reprinted by arrangement with Henry Holt and
Company, LLC.
| 
