The Future of Interactive Gestures

Tomorrow will give us something to think about.

—Cicero

Predicting the future is a sucker's game. But there is far more to interactive gestures than check-in kiosks, towel dispensers, and the nearly ubiquitous demonstration of scaling and sorting photos! As prices decrease and the availability of these devices (and the tools to create them) increases, we will see more novel implementations of touchscreens everywhere.

Of course, this may not be an entirely good thing. As Chapter 1 pointed out, gestural interfaces shouldn't be used for everything: they are a poor choice, for instance, for heavy data input. But assuredly, we'll see them being used for activities for which they weren't meant to be used, as well as for applications for which we never expected to use them. Despite the long history of this technology, we're entering a time—an interesting and exciting time—much like the early days of the Web in the 1990s or the beginning of personal computing in the 1970s and 1980s, when experimentation and exploration are the norms, when we're still figuring out standards and best practices and what this technology can really do.

FUTURE TRENDS

It took about six years for the gestural system in Minority Report to move from science fiction to reality; what will the next six years bring? Here are my predictions.

• WIDESPREAD ADOPTION IN THE HOME AND OFFICE

We've seen gestural interfaces in public spaces such as public restrooms, retail environments, and airports. But touchscreens haven't entirely penetrated the home and office environment as of yet (at least not until, like any mature technology, they become invisible and "natural"). But that is changing quickly. Consumer electronics manufacturers are rapidly producing new lines of products that employ touchscreens. As BusinessWeek reported, companies around the world are designing and producing new products with gestural interfaces:

The touch-screen tech ecosystem now includes more than WO companies specializing in everything from smudge-proof screens to sensors capable of detecting fingers before they even contact the screen. Sales of leading touch-screen technologies, such as those used in mobile phones and navigation devices, are expected to rise to $4.4 billion in 2012, up from $2.4 billion in 2006, according to iSuppli estimates.

This technology ecosystem, plus the extreme interest by companies in getting in on what is seen as the next wave of product innovation, practically guarantees that touchscreens and gestural interfaces will be entering the home and traditional office over the next several years.

• SUPPLANTING THE DESKTOP METAPHOR?

Often when one mentions interactive gestures, the conversation turns to the impact that gestural interfaces will have on the desktop. Will the traditional key- board-mouse-monitor setup be replaced by a touchscreen? Or a headset and special gloves?

The simple answer is...maybe, but probably not in the near future. It takes a long time for a technology, especially one as deeply ingrained as "traditional" computing, to be supplanted. Besides, a keyboard is still necessary for heavy data input (e.g., email, instant messaging, and word processing), although with a haptic system (see later in this chapter), this could be mitigated. The monitor (or some sort of visual display) is necessary as well, although this could easily be a touchscreen (as it is on some newer systems and on tablet PCs). The most vulnerable part of our existing PC setup is the mouse: the mouse could be replaced (and on many laptops it already has) by touchpads or a gestural means of controlling the cursor and other on-screen objects.

It certainly is possible that some jobs and activities that are currently accomplished using a traditional system will be replaced by a gestural interface. If you aren't doing heavy data entry, for instance, why do you need a keyboard and mouse? Gestural interfaces can and should be used for specialized applications and workstations.

The influence of interactive gestures will likely be felt, however, by the further warping—and even enhancement—of the desktop metaphor. If interactive gestures become one input device (alongside traditional keyboards and voice recognition), they could start to affect basic interactions with the PC. Waving a hand could move the windows to the side of the screen, for example. Finger flicks could allow you to quickly flip through files. Even a double head nod could open an application.

The home, being semiprivate, may allow for a wider range of gestural products, since in the home you can do certain gestures and activities which you would seldom do in a workplace—for instance, clapping your hands to turn on a washing machine. Although gestural interfaces will naturally be used for "traditional applications" (already millions of people are using them with trackpads), they may find their greatest use in specialized applications and products.

• SPECIALIZED APPLICATIONS AND PRODUCTS

Interestingly, unlike many of the technologies that are currently available, gestural interfaces may be best suited to settings other than the traditional office— at least, not the offices many of us are used to, with walls, desks, and cubicles. No, it is this author's opinion that, although offices will certainly be a rich place to find gestural interfaces, the best use of them may be in other locations: homes, public spaces, and other environments (retail, service environments such as restaurants, and so-called "third places"'). These other environments have a wider variety of activities than the traditional workplace, and many of these activities already have associated with them a physical activity (everything from cooking to working out to gaming to simply walking about) that could be further enhanced by making more gestures interactive.

Specialized products will be paired with specialized activities in specialized environments, and that will be a rich source of gestural interfaces. Public restrooms are currently a great example of this, but other spaces could easily take on this sort of "hothouse" environment. The next likely place for such experimentation and adoption is kitchens: they feature lots of activities, plus a contained environment with tons of specialized equipment.

Yo-Yo Ma and the Hypercello

As Neil Gershenfeld detailed in his book When Things Start to Think, in the 1990s, renowned cellist Yo-Yo Ma partnered with the MIT Media Lab to create a modified Stradivarius (and bow) outfitted with a battery of sensors that were sensitive enough to pick up his nuanced playing and the subtleties of the instrument itself. The sound and playing were then amplified and augmented, changing the sound while still retaining some of the cello-like qualities of the instrument and the playing style and technique of the musician. This "hypercello" is an example of an extremely specialized product and input device—one worth millions of dollars.

• NEW INPUT DEVICES

These specialized applications and products will probably require more than off-the-shelf touchscreens to make them work. They could have specialized gear that goes with them. As in many areas of technology, games are leading the way, with many forms of alternative input devices, from the Wiimote to objects shaped (and sort of used) like guitars.

• CONVERGENCE WITH OTHER TECHNOLOGIES

Few technologies live in isolation, and interactive gestures are no exception. In the future, the line between the following technologies and the world of interactive gestures will only grow blurrier as the technologies and systems grow closer together.

• VOICE RECOGNITION

Once we move past just using a keyboard and mouse to interact with our digital devices, our voices seem like the next logical input device. Voice recognition combined with gestures could be a powerful combination, allowing for very natural interactions. Imagine pointing to an object and simply saying, "Tell me about this chair."

Using voice with gestures could also help overcome the limitations of modes using free-form gestures. Users could issue voice commands to switch modes, then work in the new mode with gestures.

• VIRTUAL REALITY

Virtual reality (VR) is the technology that always seems on the edge of tomorrow, and gestures, once you are in an immersive space, are a likely way of navigating and controlling the virtual environment. After all, the cameras and projectors are possibly already in place to display the environment, and the user is likely wearing or using some sort of gear onto which other sensors can be placed.

 

Last updated: Feb 8, 2018.