The physical world is set to blur into the digital with the invention of a new ‘sixth sense’ device.

Technology is constantly evolving into realms that were previously unheard of, thanks to the work of countless trailblazing technicians across the globe. Lately, we have seen society adopt a trend where interactivity is a heavily desired feature – and the more that people can use technology in their everyday lives, the better.

For proof of this, look no further than the custom installation industry itself. Through the implementation of technology such as touch panels and smart phones, home owners can control everything from the lights in the kitchen to the level of water in the bathtub, all through the touch of a button.

Following in the footsteps of current whizz-bang technologies is a new mechanism called SixthSense, specifically designed with mobility in mind. And if this new prototype manages to see the commercial light of day, it could hold the potential to become the preferred option for multi-touch devices worldwide.

SixthSense is the brainchild of a team of researchers from the Massachusetts Institute of Technology (MIT), lead by associate professor Pattie Maes and PhD student Pranav Mistry. The idea behind the technology is simple: to provide access to scores of information on-the-go, by turning any flat surface into a touch screen display (much like Tom Cruise in the film, Minority Report).

As the face of SixthSense, Pranav says it’s all about integrating the virtual world with reality.

“We’ve evolved over millions of years to sense the world around us. When we encounter something, someone or some place, we use our five natural senses to perceive information about it and that helps us make decisions and choose the right actions to take,” Pranav writes on his website.

“But arguably, the most useful information that can help us make the right decision is not naturally perceivable with our five senses. This is namely the data, information and knowledge that mankind has accumulated about everything, which is increasingly all available online.”

This ultimately led to the creation of SixthSense, a wearable gestural interface that augments the physical world with digital information. Interestingly, it responds to natural hand gestures in order to interact with and manipulate information.

The prototype is comprised of a web camera, a 3M Pico projector and a mirror, which are coupled in a pendant that hangs around the user’s neck. Both the projector and the camera are connected wirelessly to a Bluetooth smart phone that sits in the user’s pocket.

Basically, the camera can instantly recognise and track the user’s hand gestures or postures and the surrounding physical objects using computer-vision based techniques. In order to help differentiate the different cues, both thumbs and index fingers are taped with either green, blue, red or yellow tape.

The projector can then cast visual information onto any flat surface in front of the user, thus enabling walls, objects and the user’s hands to transform into multi-touch interfaces.

The prototype is comprised of a web camera, a 3M Pico projector and a mirror, which are coupled in a pendant that hangs around the user’s neck. Both the projector and the camera are connected wirelessly to a Bluetooth smart phone that sits in the user’s pocket.

“Although the miniaturisation of computing devices allows us to carry computers in our pockets, keeping us continually connected to the digital world, there is no link between our digital devices and our interactions with the physical world. Information is traditionally confined on paper or digitally on a screen,” Pranav says.

“SixthSense bridges this gap, bringing intangible, digital information out in the tangible world, allowing us to interact with this information via natural hand gestures. SixthSense frees information from its confines by seamlessly integrating it with reality and thus making the entire world your computer.”

The SixthSense software program processes the video stream data that is captured by the camera and tracks the locations of the coloured markers, which act as visual tracking fiducials at the tip of the user’s fingers. The creation of simple hand gestures then act as interaction instructions for the projected application.

“The maximum number of tracked fingers is only constrained by the number of unique fiducials, thus SixthSense also supports multi-touch and multi-user interaction,” he says.

“The SixthSense prototype implements several applications that demonstrate the usefulness, viability and flexibility of the system. The map application lets the user navigate a map displayed on a nearby surface using hand gestures, similar to gestures supported by multi-touch based systems, letting the user zoom in, zoom out or pan using intuitive hand movements.”

Another feature, the drawing application, lets the user draw on any surface by tracking the fingertip movements of the user’s index finger. For example, the SixthSense system implements a gestural camera that can take photos of the scene the user is looking at by detecting the ‘framing’ gesture that is often associated with a cliché Hollywood director. To browse through the photos that were taken, a surface or wall will suffice as the backdrop.

SixthSense also lets the user draw icons or symbols in the air using the movement of the index finger and recognises those symbols as interaction instructions. For example, drawing a magnifying glass symbol activates the map application, while drawing an ‘@’ symbol opens the email application.

“The SixthSense system also augments physical objects that the user is interacting with by projecting information about these objects onto them. For example, a newspaper can show live video news, or dynamic information such as flight delays can be provided on an airline ticket. The gesture of drawing a circle on the user’s wrist automatically projects an analogue watch to check the time,” Pranav says.

While these are simply examples of the novel applications that this new prototype is capable of, the true power of SixthSense lies in its potential to connect the real world with the internet and project this information in front of the user – which ultimately spells out great opportunity for the custom installation industry and residential applications in particular.

The key here is that SixthSense recognises the objects in our surrounding environment, automatically displays information about it and allows you to access it in any way.

If it succeeds in progressing through the initial development stage, SixthSense clearly has a strong potential to become the ultimate ‘transparent’ user interface for accessing information about the world around us. It may change the way we interact with reality to give a complete awareness of the environment around us any time, any place.