Technical Blogs: Sixth Sense

Sixth Sense Technology:

“Invent is an attempt to make programming more like thinking”

Invent= Imagine + Explore + Learn

Fig 1: Six Senses

Human have basic five senses that constitutes of Sight for vision, Sound for communication, Smell for fragrance, Taste for sweet and sour and Touch. As a child is born it does not develop all its senses from the birth but most of them are developed during a human life cycle. Millions of years have been evolved to sense the world. As According to human nature when we see someone, something, someplace our five senses perceive the information and helps in making our decision. Most of the time the useful information perceived by our five sense doesn’t help us making the correct decisions. Being human we have a limitation while considering the data, Information and Knowledge that is available online. But the miniaturization of computing device and the availability of internet everywhere as made those data available everywhere this data has though limitation that they are limited to paper or mobile phone or digital screen. There is no interaction of those data confined on the digital screen with our physical world. Consider a Example if we meet some distant relative of our and we can recognize his/her personal or professional information in that very case we have to take our mobile or the digital screen and we have to Google about him our search about him in the data record where ever we store it.

There is no link between that digital information and our interaction with the physical world.

6^th sense bridges this gapes. Bringing intangible (not able to perceived by touch) digital information available in an informative and represent able way to the physical world which is tangible and allow us to interact with those information using our normal hand gestures.

Sixth Sense is also named by it developers as “WEAR THE WORLD” device i.e., using the entire world as your computer.

Physical Aspects of the Sixth Sense Device:

Sixth sense device is a device with basic gestural interface which augments the physical world around us with digital information available. The hardware components are coupled in a pendant like mobile wearable device.

The components constituting the sixth sense device are,

1. Micro Battery Projector.

2. Mirror.

3. A high resolution camera.

4. A Phone with internet connectivity and software to allow the other devices to interact with Phone.

5. Coloured markers.

Fig 2: The device

Now we will discuss the function and use of each of the components mentioned above.

Micro Battery Projector:

It’s a small battery powered projector that projects the digital information on any surface it can be a wall, a piece of paper or if nothing is available on our palm. In simpler words it can be used to project on any opaque surface. This projector not directly projects the digital data on to an opaque surface but it uses a small mirror that receives the data from the projector and reflects it to the opaque surface.

Mirror:

The small mirror that receives the data from the projector and reflects it to the opaque surface.

A high resolution camera:

The high resolution camera that is used to capture the physical world around and the physical gestures which are they way by which we contact with the physical world around us. In technical aspect the camera recognizes and tracks a user hand gestures and the physical object which is in front of it.

Coloured Marker:

Coloured marker are simple colour caps or it can be nail paint this is just to differentiate the normal fingers from the fingers with which we perform the gestures and each having a different just to differentiate it a little further. These are also called as visual tracking fiducials. Movement or arrangements of these fiducials are interpreted as gesture that acts as interaction instruction for the projector application interface. The maximum no of tracked finger is constrained to the uniqueness of the fiducials. Hence by using unique fiducials we can provide a support for multi touch and multi user interaction.

Technical aspects of the sixth sense device:

Computer Vision Techniques:

Computer Vision techniques includes the processing of large data set using techniques of AI (Artificial Intelligence) or ANN (Artificial Neural Network). This involves creating a machine that can see. It’s concerned with the theory of building artificial system that obtains information from images. The images can be replaced by video sequence, views from multiple camera, and multi dimensional data from medical scanners.

Examples of applications of computer vision include systems for:

Controlling processes (e.g., an industrial robot or an autonomous vehicle).
Detecting events (e.g., for visual surveillance or people counting).
Organizing information (e.g., for indexing databases of images and image sequences).
Modeling objects or environments (e.g., industrial inspection, medical image analysis or topographical modeling).
Interaction (e.g., as the input to a device for computer-human interaction).

Computer vision is closely related to the study of biological vision. The field of biological vision studies and models the physiological processes behind visual perception in humans and other animals. Computer vision, on the other hand, studies and describes the processes implemented in software and hardware behind artificial vision systems. Interdisciplinary exchange between biological and computer vision has proven fruitful for both fields.

Computer vision is, in some ways, the inverse of computer graphics. While computer graphics produces image data from 3D models, computer vision often produces 3D models from image data. There is also a trend towards a combination of the two disciplines, e.g., as explored in augmented reality.

Sub-domains of computer vision include scene reconstruction, event detection, video tracking, object recognition, learning, indexing, motion estimation, and image restoration.

Deploying CVT in Sixth sense Device:

In the Sixth sense device the aspect of Computer vision techniques that are used are Object recognition and searching of object with the object itself.

Examples:

1. Drawing in open air