Alternative Systems

• Overview
• Mouse Gestures
• Tangible Interfaces
• Ambient Devices
• Voice Recognition
• Physical Motion Based Systems

 

Overview

Science Fiction writers and filmmakers have had the imagination to demonstrate interfaces outside the constrains of current technologies. We have taken many of these interfaces as our own inspiration, such as the “PreCog Neural Viewer” of Minority Report, the retinal scanner of Blade Runner, and the piloting system of the ships in Earth: Final Conflict.

These systems are examples of how discreet physical actions can, in theory, be translated by machines into action. By removing the interface from a two-dimensional plane, and by focusing on the desired tasks and physicalities of the user, a remarkable amount of functionality can be achieved with startlingly simple technologies.

Mouse Gestures

Mouse Gestures are an evolution of the classic mouse based user interface. Typically, mouse gestures are performed when the user holds down a modifier key (e.g. right mouse button) and makes a movement, or gesture with the mouse, such as a swipe left to move back in a web browser, or a circular motion to reload the page. Mouse gestures are based on the principle that it's easier to make a simple gesture anywhere on the screen than it is to navigate the mouse pointer to a specific area of the screen and hit an button to perform an action. Gestures can be performed wherever the mouse is located, can vary in size, position and speed, and as such are more flexible actions than simple mouse clicks on designated screen areas.

In recent times, the Opera web browser has integrated mouse gesture technology into its navigation features, while other software such as Stroke-It and WayV offer solutions for the entire operating system, allowing users to control their computers using gesture technology. While mouse gestures could never replace menus and other typical UI elements because of the complexity of assigning, and indeed remembering, gestures for every function on a computer, they offer an effective tool in technological systems where repetitive and predictable functions are found, such as the navigation elements of a web browser, or the transport controls of a media player.

Tangible Interfaces

Many of the problems people have when interfacing with technologies stem from the disembodiment of the devices, where huge amounts of data, and the virtual tools we use are nothing more than symbols on the screen to be manipulated crudely. Tangible computing is concerned with engaging with the users' sense of touch, by providing an actual physical control over on-screen elements, and in some cases related sensory feedback through force-feedback devices.

Many new “Tablet PCs” engage directly with the user's body using touch areas of the screen, and while this is a far removed and very rudimentary when compared to interaction with real world objects, the added tactility of the display is a small step towards interacting with computers in the way a person would interact with everyday objects. When we consider the introduction of touch screens in consumer PCs; the challenge of moving towards representations of technology and computing which take advantage of physical ergonomics and more comfortable ways of working seems to be close at hand.

MIT's Tangible Media Group are pioneers of this type of interface, and have developed the Sensetable project, a table that detects object placed on it and feeds input into the computer. Pictured above is one application of it, called AudioPad, an computer-based composing and performance instrument for electronic music. The combination of a tangible interface with the flexibility of a display that can change, combines the ease of use of a traditional instrument, everyday objects, and the power of modern computing software.

Ambient Devices

Ambient devices are an alternative form of feedback device. They are used to give the user complex information through our peripheral senses. We notice when food is burning even if we're not paying full attention to it, and tangible devices take advantage of a similar idea by providing information on situations such as the weather, stock markets through colours and sounds transmitted through discreet objects; through . Shown on the right is the Stock Orb, a device which glows at different colours based on the performance of the stock market. Information is transmitted to the device through radio waves, meaning it can be placed anywhere as an ornament.

Ambient devices represent the concept of subtle computing, a movement away from classic incarnations of the computer to a schema where information is located directly within our environment, not seta apart on a screen.

Voice Recognition

The idea of commanding an autonomous computer using nothing more than our voice is extremely attractive. Along with physical communication, our voice is the primary means of communication between people. The goal of achieving natural language based interaction with a computer could break down many barriers between the user and technological tasks. By walking into a room and saying “Chopin”, a computer could choose a piece of music, or by saying “send this file to Paul”, a computer could initiate and send an e-mail to a recipient.

Building voice recognition systems has proved to be difficult, and many incarnations of the technology have been problematic. IBM's ViaVoice, one of the vanguard in commercial voice-recognition packages, required the user to ‘train’ the computer for several hours in order for the computer to discern accent, intonation, meter of speech; quite a way behind the idea of the computer in Star Trek; a wonderous machine that can easily complete complex tasks when given instructions by any user. The target of universal, natural voice-recognition software is achievable, but it may only emerge in ‘everyday’ situations to complement another interface system such as physical motion. The vagaries of human physical language, body language are minute when compared to the variety of languages, accents, and rhythms of speech found worldwide.

For simple tasks such as retrieving basic information and turning on and off lights, voice recognition could be very useful. For specific tasks however, such as programming, drawing, or quickly navigating large amount of data it may be problematic for quite some time.

Physical Motion Based Systems

Body language is crucial in everyday communication, some would even argue that is is more important than conscious verbal communication. Physical gestures are a huge part of that communication process. Humans are uniquely evolved to interpret not only complex verbal communication, but also to sense mood, temperament and implicit schema of communication through body posture, arm and hand movement, eye and facial expressions. The challenge in creating a physical motion based system is to observe and decode physical movement, conscious or otherwise, and translate it into machine action.

The film Minority Report saw Tom Cruise's character using one such system while manipulating the ‘PreCog Neural Viewer’, a system which allowed him to see premonitions of ‘pre-crimes’ as video, and manipulate data drawn from population and criminal records, scrubbing back and forth through video, and throwing unwanted files away using hand gestures. This direct manipulation of data bridges the gap between between computers and the physical world, removing the abstraction of user interfaces, replacing it with body movements designed around the tasks at hand.