field of invention - ipindiaservices.g..

kayakstarsAI and Robotics

Nov 15, 2013 (3 years and 11 months ago)

81 views

FORM 2

THE PATENTS ACT, 1970


(39 of 1970)

As amended by the Patents (Amendment) Act, 2005

&

The Patents Rules, 2003

As amended by the Patents (Amendment) Rules, 2005

COMPLETE SPECIFICATION

(See section 10 and rule 13)

TITLE OF THE INVENTION

Head mounted device for semantic representation of the user surroundings
APPLICANTS

Name
Nationality

Address

:

INVENTORS

Names

Nationality

Address

Indian Institute of Technology, Bombay
an autonomous research and educational

institution established in India by a special

Act of the Parliament of the Republic of India

under the Institutes of Technology Act 1961

Powai, Mumbai 400076, Maharashtra, India

Chaudhuri Subhasis, Rajashekar and Prabhudesai Amit

all Indian Nationals

all of Indian Institute of Technology, Bombay, Department

of Electrical Engineering, Powai, Mumbai 400076, Maharashtra,

India

PREAMBLE TO THE DESCRIPTION

The following specification particularly describes the nature of this invention and the
manner in which it is to be performed :

FIELD OF INVENTION

This invention relates to a head mounted device for semantic representation of the user
sur
roundings.

The term "semantic representation of the surroundings" as used in the specification
means a verbal description of the surroundings whereby the user can form an image of
the
surroundings mentally or visualise or perceive the surroundings.

The device can be used to aid a visually impaired person and give him a semantic
description of the surroundings around him or traversed by him. The device can also be
used to augment a robot navigation system whereby a remote operator gets a perception
of

the surroundings around the robot or traversed by the robot.

BACKGROUND OF THE INVENTION

US Publication No 2005/0208457 relates to a camera based object detection system for
the visually impaired. The system comprises a digital camera mounted on the user
to take
images of objects on demand, real time algorithms connected to the camera to decipher
attributes of the images, an announcement module connected to the algorithms to
construct a sentence to describe the images and a computer based voice synthesizer

connected to the module to verbally announce the sentence to the user. The system
essentially enables the user to recognize or identify the objects in front of the user and
navigate but does not give to the user a qualitative or semantic description or me
ntal
picture of the surroundings around him or traversed by him.

2

WO 03/107039 relates to an apparatus and method and also a system for aiding the
visually impaired. It comprises means for sensing time/space characteristics and physical

characteristics information about an object in a field of view, means for
interpreting the
time/space characteristics and physical characteristics information and for
characterizing
the object and audible information delivery device for audibly describing

to the user the
characterization of the object and the interpretation about the object. This also enables the
user to recognize or identify the objects in front of the user and navigate but does not give a
mental picture or perception of the surroundings
around the user or he
is traversing.

OBJECTS OF INVENTION

An object of the invention is to provide a head mounted device, which enables the user to
perceive the surroundings around him or he is traversing accurately and reliably.

An other object of the inv
ention is to provide a head mounted device, which is light
weight to be carried on the person of the user and is simple in construction and easy to
operate.

Another object of the invention is to provide a head mounted device, which is
economical.

Another object of the invention is to provide a head mounted device, which aids a
visually impaired person wearing it by giving him a semantic description of the
surroundings around him or traversed by him.

3

Another object of the invent
ion is to provide a head mounted device, which can be used
to
augment a robot navigation system whereby a remote operator gets a perception of the
surroundings around the robot or being traversed by the robot.

DETAILED DESCRIPTION OF INVENTION

According to

the invention there is provided a head mounted device for semantic
representation of the user surroundings, the device comprising a cap adapted to be fitted
on the head of the user, an image capturing means mounted on the cap, a computing
platform connect
ed to the image capturing means, a feature data store connected to the
computing platform and comprising reference features which represent different entities
existing in the surroundings and are associated with semantic description of the entities

9

in th
e surroundings represented by them and an output module connected to the
computing platform, the computing platform being designed to extract at least one feature
from the images of the entities in the surroundings captured by the image capturing
means, co
mpare the extracted feature of the captured image with the reference features in
the feature data store, identify the reference feature in the data store closest to the
extracted feature of the captured image and provide an output ie the semantic descripti
on
of the entities in the surroundings associated with the identified feature to the output
module.

The following is a detailed description of the invention with reference to the
accompanying drawings, in which:

4

Fig 1 is a block diagram of a head mounted device for semantic representation of the user
surroundings excluding the cap according to an embodiment of the invention;

Fig 2 is a schematic view of the cap of the device of Fig 1; and

Fig 3 is a schematic view

of the cap and the image capturing means mounted on the cap.

The device of the invention as illustrated in the accompanying drawings comprises a cap
1
provided with a pair of fastening straps 2 and 3. The strap 2 is provided with a buckle 4
whose hinged p
in is marked 5. The strap 3 is provided with a series of pin holes 6. The
cap 1 is worn on the head of a user and fastened to the head of the user by threading
strap
3 through the buckle 4 and engaging pin 5 of the buckle in one of the pin holes 6 in the s
trap
3. An image capturing means comprising a non
-
calibrated digital camera 7 is
mounted on
the cap with support members 7a, 7b and 7c which are fixed on the cap.
The outer surface
of the cap is rendered reflective by coating the outer surface of the cap
w
ith a reflective
material such as mercuric oxide in known manner. The reflective coating is marked la.
Alternatively, the image capturing means comprises an
omnidirectional camera mounted
on the cap. 8 is a computing platform connected to the
image capturi
ng means. The cord
connecting the image capturing means to the
comprising platform is marked 7d. The
computing platform comprises, for instance, a
laptop PC. 9 is a feature data store
connected to the computing platform. The feature
data store comprises re
ference features
which represent different entities like trees or
buildings or water bodies or roads existing
in the surroundings. Each of the reference

5

features is associated with a semantic description of the entities in the surroun
dings
represented by them. The feature data store is stored on a USB drive or flash memory
stick or hard disk. 10 is an output module connected to the computing platform. The
output module comprises, for instance, an audio system comprising a pair of speak
ers
worn on the ears of the user like visually impaired person and connected to the output
port
of the laptop PC or a Braille board display connected to the output port of the laptop
PC. In
the case of a digital camera, snapshots of the images of the entit
ies in the
surroundings of
the user reflected on the outer surface of the cap are taken by the camera.
In the case of an
omnidirectional camera, snapshots of the entities in the surroundings of
the user are
directly taken by the camera. Each of the images
shot by the camera is
divided into five
sectors namely Right Top (RT), Front Right (FR), Left Top (LT), Left Bottom (LB) and
Right Bottom (RB). The laptop PC extracts at least one feature of each
the sectors of the
captured image. Preferably the extracted
feature is the colour
component of the different
entities in the surroundings as captured by the camera. The
laptop PC compares the colour
feature with the reference features stored on the data store. Color feature in the data store
that best matches with
the extracted features is
identified. A semantic description of the
entity in the surroundings associated with the
identified feature is fed to the speakers or
Braille board display, as the case may be,
thereby enabling the user to relate to or visualise
o
r perceive the surroundings he is in or
traversed by him.

The device of the invention can be used by a visually impaired person so as to enable
him
to perceive or visualise the surroundings around him or he is traveling accurately and
reliably. The device can also be used to augment a robot navigation system whereby a

6

remote operator can get a perception of the surroundings around the robot or traversed by
the robot. The components of the device of the invention are

relatively cheap and,
therefore, the device of the invention is economical. It is also simple in construction and
easy to operate besides being light weight to be carried on the head of the user.

EXAMPLE 1

A typical device of the invention comprising an o
mnidirectional camera was tested on a
visually impaired person by making him traverse on a road surrounded by woods and
water body and buildings on both the sides. The visually impaired person took 15 snap
shots of the surroundings at different locations.
Each of the images shot by the camera
was divided by the camera into five sectors namely Right Top (RT), Front Right (FR),
Left Top (LT), Left Bottom (LB) and Right Bottom (RB). The color feature of each of
the
sectors of the captured image was extracted b
y the laptop PC. The color features
extracted
from each of the sectors of the captured image was compared with the reference
features
stored on the data store. Color features in the data store that best matched with the
extracted features were identified a
nd a semantic description of the features was fed
to the
Braille board. The data read by the visually impaired person on the Braille board was
tabulated against actual data as shown in the following Table. It is quite clear from
the
Table that the estimate
d description (ED) matched with the true description (TD) thereby
establishing that the device is accurate and reliable.

7

Table


Photo





Sectors





Frame

RT

FR

LT

LB

RB


EC

TC

EC

TC

EC

TC

EC

TC

EC

TC

1

WD

WD

WD

WD

WD

WD

BD

BD

WD

WD

2

WD

WD

BD

BD

WD

WD

WD

WD

WD

WD

3

WD

WD

BD

BD

WD

WD

WD

WD

WD

WD

4

BD

BD

WD

WD

WD

WD

WD

WD

BD

BD

5

WD

WD

WD

WD

WD

WD

WD

WD

BD

BD

6

BD

BD

BD

BD

BD

BD

WD

WD

BD

BD

7

BD

BD

BD

BD

BD

BD

WD

WD

BD

BD

8

WD

WD

BD

WD

WD

WD

BD

BD

WD

WD

9

WD

WD

BD

WD

BD

BD

BD

BD

WD

WD

10

WD

WD

BD

BD

WD

WD

WD

WD

WD

WD

11

WD

WD

WD

WD

BD

WD

WD

WD

WD

WD

12

WD

WD

WD

WD

WD

WD

BD

BD

WD

WD

13

WD

WD

WD

WD

WD

WD

BD

BD

WD

WD

14

WD

WD

BD

WD

WD

WD

BD

BD

WD

WD

15

WD

WD

WD

WD

WD

WD

WD

WD

WD

WD

TC

=

True Category

EC

=

Estimated Category

WD

=

Woods

BD

=

Building

It is possible to have variations in the invention without deviating from the scope thereof.
For instance the cap configuration can be different. The extracted feature of the captured
image and the features in the reference data store need not be colour. Besides the feature

8

data store, the computing platform may comprise image processors. Such variations of
the invention are to be construed and understood to be with
in the scope of the invention.

We Claims:
-

1.

A head mounted device for semantic representation of the user surroundings,
the device comprising a cap adapted to be fitted on the head of the user, an
image capturing means mounted on the ca
p, a computing platform connected
to
the image capturing means, a feature data store connected to the computing
platform and comprising reference features which represent different entities
existing in the surroundings and are associated with semantic desc
ription of
the entities in the surroundings represented by them and an output module
connected to the computing platform, the computing platform being designed
to
extract at least one feature from the images of the entities in the
surroundings
captured by
the image capturing means, compare the extracted
feature of the
captured image with the reference features in the feature data
store, identify the
reference feature in the data store closest to the extracted
feature of the captured
image and provide an out
put ie the semantic
description of the entities in the
surroundings associated with the identified
feature to the output module.

2.

A device as claimed in claim 1, wherein the image capturing means comprises
a
non
-
calibrated digital camera mounted on the cap, the outer surface of the cap
being reflective.

3.

A device as claimed in claim 1, wherein the image capturing means comprises
an omni directional camera mounted on the cap.

10

4.

A de
vice as claimed in any one of claims 1 to 3, wherein the computing
platform comprises a laptop PC.

5.

A device as claimed in any one of claims 1 to 4,wherein the extracted feature
comprises the color component of the different entities existing

in the
surroundings as captured by the image capturing means.

6.

A device as claimed in any one of claims 1 to 5,wherein the output module
comprises an audio system or a Braille board display.

7.

A device as claimed in anyone of claims 1 to 6, wherein the feature data store
comprises feature data stored on a USB drive or flash memory stick or hard
disk.

8.

A head mounted device as claimed in anyone of claims 1 to 7, which is for use
by a visually
impaired person.

9.

A head mounted device as claimed in anyone of claims 1 to 7, which is for use

on a robot.

Dated this 27
th

day of January 2006

11


ABSTRACT

A head mounted device for semantic representation of the user surroundings. The
device comprises a cap (1) adapted to be fitted on the head of the user, an image
capturing means (7) mounted on the cap, a

computing platform (8) connected to the
image capturing means, a feature data store (9) connected to the computing platform
and an output module (10) connected to the computing platform. The feature data
store comprises reference features which represent
different entities existing in the
surroundings and are associated with semantic description of the entities in the
surroundings represented by them. The computing platform is designed to extract at
least one feature from the images of the entities in the
surroundings captured by the
image capturing means, compare the extracted feature of the captured image with the
reference features in the feature data store, identify the reference feature in the data
store closest to the extracted feature of the captured

image and provide an output ie
the semantic description of the entities in the surroundings associated with the
identified feature to the output module (Fig 1).