Forsaking photorealism for ambiguity, Osmose presents its developers with unique design challenges

Char Davies' Osmose (Tree Pond)

Digital frame captured in real-time through head-mounted display during live performance.]

Osmose is probably not like any other virtual-reality experience you've ever tried. It doesn't work the same way, and it doesn't have the same look and feel. As a result, it's easy to understand why the task of creating it inquired the creation of customized tools, lots of trial and error, and tight collaboration between Davies' three-person team, which consisted of Davies, John Harrison, and Georges Mauro.

Concept development on Osmose began in the fall of 1993, and it took Davies about six months to refine and document her vision. Once that was completed, the next step was to create the VR subsystem, which involved writing the real-time interactive code. Work on that began in May 1994 and took almost a year to complete. The person Davies brought on board to do that work was John Harrison.

This VR subsystem was developed on SGI Indigo2 and Crimson computers. (A full installation of Osmose runs on an 8-processor, 3-pipe SGI Onyx RealityEngine2 and an assortment of audio equipment to create the sound.)

"The entire project depended on achieving a soft and luminous transparency in real time," explains Davies. "We first tried many tests to try and get a soft and luminous effect. Along with this were experiments in various sorts of navigation using breath and balance, which culminated in the system we now use. For each new idea that we used in Osmose, perhaps 10 to 20 ideas were thrown away."

The third and final stage of the project got underway in May 1995 and took about three months to complete. This was when Georges Mauro joined the team to take on all responsibility for creating the models and textures. All were created using Softimage 3D software (with the exception of the leaf textures, which were digitally scanned in from real leaves).

In developing the content, Mauro worked very closely with Davies and Harrison. Davies provided the conceptual direction, while Harrison worked to reduce triangle counts to make the models more efficient for real-time viewing. Harrison also wrote new software as needed to achieve new special effects.

"I would sometimes provide Georges with references to sketches or paintings I had done years before," says Davies, explaining the working relationship. "However, I didn't make a storyboard for Osmose or attempt to reproduce my vision on paper, since it could not be expressed on a static 2D surface.

"The creative process was explorative, impro- visational, and intuitive," she continues. "Georges would show me various tests and I would indicate which ones offered promise and should be pursued further. Toward the end of the project, when we had all the worlds functioning, these meetings would often take place with Georges and me taking turns in the helmet discussing a subtle but crucial change that was needed."

As it turned out, developing imagery that had the ambiguous, ethereal look Davies was striving for was more difficult both conceptually and technically than creating photorealistic imagery.

"To effectively create ambiguity," says Davies, "the elements created must lie along a very delicate threshold somewhere in between the two poles of abstraction and realism. This is very difficult, as it is all too easy to fall into either abstraction or realism."

For example, says Davies, at first Mauro made the edge of the forest so realistic that it caught your attention as a forest-edge, in which the tree trunks were obvious and so forth. But Davies wanted the edge of the forest to be "felt," to be sensed as a presence rather than be seen as an object. Eventually Mauro was able to accomplish this through repeated attempts working with the texture, the tonality, and the color.

Clearly the most demanding aspect of Osmose— as far as graphics requirements go— was its extensive use of transparency. As Davies explains, "Since immersants can see through everything, we couldn't do any tricks like not rendering objects that are occluded. For example, the underground world is made up of many layers of textures, and to create the sense of depth when the immersant looks down, we needed to draw them all."

For Osmose, this meant that each pixel had to be redrawn about 20 times, creating a significant bottleneck that needed to be overcome.

Another challenge, says Davies, was working out a way to move from one world to the next in a smooth, seamless manner. Traditionally, VR worlds use portals, or doorways, to make the transition. These portals are set so that one cannot see the new world until the old one is turned off. "We used a different technique in Osmose, " says Davies. "As one moves toward a new world, it starts slowly to fade in from complete transparency while the old world slowly fades out at the same rate. There is a magical moment as this transition is happening, as two very different worlds are superimposed in the same space—and this gets rid of the feeling of an abrupt transition."

Of course, this meant rendering both worlds at the same time, a challenge that required considerable code optimization in order to maintain real-time performance.

At the same time the graphics were being developed, work on developing sound for Osmose also progressed. Dorota Blaszczak came from Poland to design and program the sound, while Rick Bidlack came from Seattle to compose and program the music. As was the case with the graphics, Davies' goal was to have sound that was neither literal nor illustrational, musical nor sound-effect-like, but which was aurally ambiguous.

While visitors in Osmose might swear they hear musical instruments and the sounds of insects and water and such, in reality all the sound in Osmose is based on samples of two human voices—one male and one female— which have been manipulated to create the varied timbres that are heard in the piece.

Because each world had its own unique soundscape, one particular challenge involved crafting the sound so that the sound from one world to the next faded smoothly as immersants moved about freely in space. Therefore, the composer couldn't use one sound to create a sustained emotion but had to plan a way for the melodies to work and interact with each other.

And just as the location in the graphics world affects the sound, sometimes the sound affects the graphics—for example, whenever the sound of wind was used, a signal was sent to the graphics engine to start the leaves shaking according to the intensity of the wind.

"The sound is extremely important," says Davies. "It plays a huge role in creating the sense of presence in the space. By always having some sort of background sound, which changes as one moves about, a person gets a strong impression of actually being somewhere. And perhaps most importantly, a lot of the emotional impact of the piece comes from the haunting melodies and soundscapes throughout."

Ultimately, Davies hopes Osmose helps expand the VR aesthetic by challenging VR developers to think in new directions. Indeed, as director of visual research at Softimage, one of her prime mandates is "to push the expressive capabilities of the tools." That's the reason Soft- image supported the work, she explains, and it's her hope other vendors will likewise support other artists.

"Given the role that digital media will play in reflecting and shaping cultural values," she says, "it is vitally important that technological development be accompanied by an evolution in visual and interactive aesthetics."

For more information on Osmose, visit its Web site at www.microsoft.com/softimage.[link now defunct, ed.]

Stephen Porter is editor of CGW.