Ramona: Questions and Answers
Ramona's alter ego (Raymond Kurzweil) says that Ramona and the technology behind her is a step in the progression of our species from real reality to virtual reality.
Originally published February 22, 2001. Published on KurzweilAI.net February 22, 2001.
What is the presentation about that you made at TED11?
I transformed myself into Ramona, my female alter ego. She is the first live virtual recording and performing artist. She is a demonstration that "you can be who you want to be" in virtual reality. We all have personalities within us that are difficult if not impossible to express with our real-world bodies and in real-world environments. Yet in virtual reality, we can express ourselves in new ways. We can go where we want to go. We can even "have been" who we want to have been, and be with whom we want to be with. Ultimately, virtual reality will be fully competitive with real reality in terms of the level of realism, while offering profound news ways to explore the world and to interact with each another.
I've always found myself attracted to female performers, and realized that I not only wanted to be with them, but wanted to be them. This is not a matter of gender confusion; having other gender personae in our fantasies is integral to our ability to relate romantically to the opposite sex. These personalities are there in all of us, but up to now have existed only in our imagination. Virtual reality provides the opportunity to express ourselves in important new ways.
My presentation pushes the state of the art in real-time virtual reality transformation, and is intended as a harbinger of things to come. Although real-time cartoon characters have been created before, this is the first time that motion capture and real-time image rendering technologies have been combined in a performance setting to create real-time photorealistic virtual performers. It is the first time that someone has given a singing performance in front of a live audience while being transformed into someone else. It is also the first time that a dancer has danced in a live performance while being transformed into someone else.
Real-time motion capture and photorealistic image transformation has obvious application to the entertainment world, but will ultimately profoundly affect the everyday world. By 2009, we will all have ubiquitous, full-immersion, shared, visual-auditory virtual reality environments. We will have images written directly to our retinas from our eyeglasses and contact lenses, high-resolution motion capture of our bodies through automatic detection of our electromagnetic fields, and very-high-bandwidth wireless connection to the Internet at all times. Going to a web site will mean entering a virtual reality environment where you can visit other people. It will be just like being there, at least as far as seeing and hearing is concerned. Of course, keep in mind that the people you meet in these virtual reality environments may have very different appearances and personalities in real reality. Ultimately, we will regard these virtual personalities as just as real and important as the one we portray in real reality.
By 2029, we will have nanobots, blood-cell sized robots, traveling through the capillaries of our brains communicating wirelessly directly with our biological neurons. This type of technology, called "neuron transistors," has already been demonstrated. Billions of nanobots can be communicating with our biological brains and with each other, all on a wireless local area network. This will enable shared virtual reality environments incorporating all of the senses. With this technology, we will be able to meet each other and engage in any type of interaction, from business negotiations to sexual and sensual encounters, in any type of environment, from recreations of Earthly places to fantastic imaginary locales.
"Experience Beamers" will beam their entire flow of sensory experience as well as their emotions onto the web just as people beam their images from their web cams today. You'll be able to plug in and experience what it is like to be someone else, à la "Being John Malkovich," as well as relive interesting archived experiences.
Like any other technology, virtual reality is not emerging fully formed in one instant. Today, it has obvious limitations. I need dozens of computers and three-quarters of an 18-wheel truck worth of equipment to transform myself into Ramona and perform with my virtual dancers in virtual environments. But I can describe today how by the end of this decade, full-immersion, visual-auditory virtual reality will be ubiquitous, woven into our clothing, and highly realistic. Ultimately, virtual reality will offer everything real reality offers and a lot more. By the middle of this century, we will be spending most of our time in virtual reality environments.
The word "virtual" may be unfortunate because it implies that the experiences and environments aren't "real." But we've already had one form of virtual reality for over a century. The telephone is auditory virtual reality, and that's exactly how nineteenth century enthusiasts viewed it. It would not be correct to say that interactions we have with each other on the telephone aren't "real." Rather, interacting with other real people over the phone represents an important extension of our ability to relate to each other. When we can add all of the other senses to these interactions, it will be another profound way to have real experiences with other real people.
When I transform myself into Ramona, I do feel empowered to express myself as a new personality. I feel that I am doing more than just playing a role, I am expressing my female alter ego, and I do find it liberating. I had a vision of who she was, and what she looked like, and with the help of dozens of brilliant people, I feel she expresses my original conception.
Why did you choose to launch Ramona at TED11?
Where else would I launch her? Ramona involves intense exploration and development of technology, entertainment, and design, and that's what TED is all about.
Do you think the general public is ready for this type of technology?
Absolutely. Like all other communication technologies from the printed book to the Web, entertainment applications are dominant early on, then progress to profound new ways to interact, to learn, and to create new knowledge.
How will Ramona, or this technology, change our lives? Why is it so revolutionary?
The ultimate significance is as a step in the progression of our species from real reality to virtual reality. Thousands of years ago, written scrolls enabled ideas to travel from one place to another. A century ago, the telephone provided auditory virtual reality. Today, the technology behind Ramona represents an early manifestation of full immersion visual-auditory virtual reality. Within a few decades, we will be able to do anything with anyone in virtual environments that rival and even exceed the realism of real reality. It will enable us to have profound new experiences, and to express ourselves in new ways.
Will it change the way people conduct business?
The concept of the "business meeting" is the most obvious example. We'll be able to meet in groups ranging from two people to large groups, regardless of the physical proximity of our physical bodies. We'll be able to select our environments of choice, choosing from a well-appointed office, a virtual Mediterranean beach, a virtual wild game preserve, and virtual flights past celestial objects. More importantly, we can select the personae that best suit our intentions and goals for each encounter. If we go out thirty years, the nanobot technology will not only enable us to engage in full-immersion virtual-reality experiences involving all the senses, but they will literally expand our minds. They can provide trillions of additional interneuronal connections, thereby expanding our memory, pattern recognition, and cognitive faculties. We will ultimately combine the subtlety of human pattern recognition with the speed and memory sharing ability that machines already excel in.
What is unique about your vision/business plan? Where are you taking this?
Ramona is more than a technology demo. We are launching a serious career for Ramona as the first live virtual musical performing and recording artist. We realize that she will have interest because of the novelty of the technology, but we intend Ramona to be more than a novelty act. Our creative team in terms of musical, theatrical, and dance talent matches that of our technology team, and we plan on making a serious artistic statement. We plan a music album, live performances on television and other venues.
We also plan on positioning KurzweilAI.net as the "place to be" on the Web for finding out about the advanced technologies of the twenty-first century, and their far reaching implications for human-machine civilization. We have an impressive line-up of the world's "big thinkers" committed to providing significant content for the site. The content is accessed with a set of integrated web technologies that allow the user to converse with the avatar version of Ramona, and to explore the intuitive organized concepts in the display of her "brain."
Are you planning to make money with Ramona? If so, how do you intend to do so?
There are three business models behind Ramona. First, she can make money the old fashioned way, as a performing and recording artist, the same way that other musical artists make money. The fact that she is the first live virtual musical performing and recording artist certainly helps her appeal, but we intend on going beyond her interest as a novelty or as a mere technology demonstration.
In order to create Ramona, we have organized considerable technological talent and expertise in virtual reality technology, and we plan on harnessing this capability to assist other firms in creating virtual reality and entertainment technology.
The same model exists for KurzweilAI.net. This site is a showcase for both content and web enabling technologies, and we plan to assist other firms to present their own content using our own proprietary system and expertise. It's the same model we have successfully followed at Medical Learning Company (FamilyPractice.com), which is another Kurzweil Technologies company.
How will this impact technology, entertainment, and the merging of the two?
The first major project I undertook that impacted the merger of entertainment and technology was Kurzweil Music Systems. The type of music synthesis we pioneered is now the basis for most commercial music, such as movie and television soundtracks and recorded music. With Ramona, we are integrating both musical and visual technologies, which is obviously the basis of most forms of entertainment.
How is Ramona different from other virtual reality "personalities"?
In terms of technology, Ramona is the first live virtual musical performing and recording artist. My presentation at TED represents the first time that a singer has performed in front of a live audience while transforming themselves into someone else. The same is true in terms of a dance performance. People have previously combined motion capture with real-time image rendering for cartoon like characters, but Ramona is the first virtual performer that is lifelike and photorealistic.
The web-based avatar version of Ramona is the first lifelike photorealistic avatar on the web that you can converse with using a conversational engine. She represents a unique combination of elements, combining a lifelike avatar, a conversational engine, synthetic speech, and the "Brain" display of her thoughts.
How will Ramona go on to be a virtual recording artist?
By recording an album and performing her original songs on television and other venues. At TED11, she performed an original song Come Out and Play (lyrics by Ramona, music by Zoux), and an original version of Grace Slick's White Rabbit.
White Rabbit is one of my favorite songs. It describes the chaos that Alice encounters ("the white knight is talking backward") and ends with the door mouse's advice to "feed your head." So chaos leads to new insight and knowledge. It's a good expression of the creative process, and an empowering personal philosophy. Incidentally, it's also a good description of how an evolutionary (or "genetic") algorithm works, which is a method I frequently employ in my pattern recognition projects.
What do her voice, appearance, movements derive from?
In the virtual reality manifestation of Ramona, special magnetic sensors pick up the motion of my body. Special motion capture computers interpret these signals to build a moving three-dimensional image of my body. These are transformed into the movements of Ramona's body by the computer taking into consideration the fact that her body is shaped somewhat differently from mine. Other computers then create the real-time moving images that correspond to Ramona's body moving in a similar manner to mine. Phoneme recognition software, which is based on similar principles to speech recognition software, detects the phonemes in my speech. This is in turn affects the shape of Ramona's mouth and other facial features to reflect her speaking. The actual sound of my voice is transformed into Ramona's voice through real-time signal processing using principles similar to music synthesis. Other software controls Ramona's facial expressions.
Real-time video compositing allows the moving images of Ramona to be composited with the other virtual performers, specifically Ramona's back up dancers. These are composited together with images of her virtual environments, which can be static or moving. Ramona can be made to appear to be actually on the set of a television program. For example, she could sit on the coach with Jay Leno and she would appear to be there like any other guest. On television, she can perform on the stage along with other real performers.
The software model for Ramona was created by scanning a human model in a special three dimensional body scanner and head scanner employing lasers and three-dimensional digital image capture. These images were then manipulated in the computer by human artists to create Ramona's unique appearance.
How long did it take to develop the technology behind Ramona?
Many of the subsystems have been in development for years and derive from work in speech recognition (for the phoneme recognition), music synthesis (for the gender bending of my voice to Ramona's voice), and real-time image rendering (to create Ramona's moving image). I had the idea to create Ramona as a result of my visit to SIGGRAPH, the nation's leading graphics show, in July of 2000, where I gave the keynote address. The integrated technology to create Ramona and the creation of her actual image, voice, and personality then took about seven months, at which time we launched Ramona at TED11.
There are about 20 people involved on our own staff plus our "Ramona Technology Consortium," which consists of over a dozen leading companies in virtual reality and web enabling technologies. All together, there were over 50 people making significant technology and creative contributions to the creation of Ramona.
Where will we see Ramona again?
Ramona will appear as part of my presentation at the ACM1 conference in San Jose on March 12 and at the New York Music & Internet Expo on April 21. We've had many other requests for her to appear, and now that she's launched her career, she'll be picking her venues carefully.
Do you expect her to evolve?
Sure. She's considering getting a tattoo on her belly, but she's getting conflicting advice on this. And then, of course, she's interested in expanding her wardrobe. Virtual reality is still a bit expensive, so each of her outfits costs tens of thousands of dollars.
Ramona has her own life story, and as she points out, even though she was created more than 24 years after her birth, her story is still just as real as the histories of people in real reality. She just celebrated her 25th. Birthday (on 2/12/01) and her story will continue to evolve. There will be ongoing improvements in the realism of the technology behind Ramona. We are interested in providing live audiences with a three-dimensional experience in the future, preferably without having to use polarized glasses, and we are exploring several possible approaches to doing this.
Will there be other virtual reality personalities to follow?
Ramona has a fondness for TED impresario Richard Saul Wurman, and is delighted that he has joined her as one of her virtual backup dancers. We envision continuing to develop her co-performers.
How does this relate to your web site KurzweilAI.net?
The avatar version of Ramona is your hostess on KurzweilAI.net and is your primary user interface. She answers your questions, displays her intuitive "Brain" idea map, and provides a rich array of articles, links, and other content related to your questions.
Although Ramona's virtual life experiences were different from my own story, she shares my passion for both technology and artistic expression, particularly for music. She is very much my female alter ego in both her virtual-reality performing manifestation and as a web based avatar.
How does Ramona the avatar differ from the "virtual celebrity" who performed at TED11 with you?
As virtual performer, Ramona is a real-time transformation of myself. She is a demonstration of the ability to "be who you want to be" in virtual reality. As avatar, Ramona is her own standalone virtual intelligence guiding you through the big thinkers on KurzweilAI.net. Although she's not yet ready to pass the Turing test, she is an effective guide to the concepts that are shaping the remarkable century ahead.
| 
