Bridging the Gap between Humans and the Physical World

beeuppityΤεχνίτη Νοημοσύνη και Ρομποτική

19 Οκτ 2013 (πριν από 4 χρόνια και 25 μέρες)

84 εμφανίσεις

Bridging the Gap between
Humans and the Physical
World

A Step Toward Reducing Energy
Consumption and Increasing Comfort

Dr. Fred Jiang /
姜小凡

In collaboration with Mike Liang, Jeff Hsu, Caiquan Liu, Jie Liu, and Feng Zhao

Mobile and Sensing Systems Group

Microsoft Research Asia


National Taiwan University / 2011
-
4
-
1

Motivation

We interact with our environment in very limited ways

Simple time series data collection

Actuation is limited because of lack of uniform interface

Physical objects cannot proactively reach out to us

Result: waste in energy and suboptimal comfort


Example 1: environment (physical objects) aware of
who is acting on it, collect usage / energy data in real
-
time

(personal energy footprint)

Example
2
: alert me if something is not normal

Example 3: objects interacting with objects

Related Work

Ambient intelligence

Roomware

Smart objects

Localization

X10

Beaver monitoring

Ubicomp

literature

We Propose

Bi
-
direction interaction and
information flow
between humans and physical
objects

For humans to have real
-
time finer
-
grained
knowledge and control over physical objects

Environment
to be aware of humans, and make
intelligent
decisions, proactively and reactively

Bridging the gap between humans and their
environment

Our vision of
IoT

(it’s not about simply putting a
networking interface on everything)

Three Primary Components

Precise indoor localization primitive

Reliable IPv6 based networking

Uniform interface for abstracting services of
physical objects

Virtual Overlay

Entrance to 4
th

floor

name:
fridge

states
:


on : true



temp : 5



door : open

commands:


turn_on
()



turn_off
()

events:



notifyDorOpn



notifyPwrUsg

children: []

Indoor Localization

Previous work

What we need

Challenges

Localization Options

Wifi

radio

BT radio

Inertial sensor

Optical (
K
inect
)

RFID

Magnetic (PKE)

Magneto
-
Induction


Slide courtesy of
Andrew Markham

(
http://
www.comlab.ox.ac.uk
/people/
andrew.markham
/)

Magneto
-
Induction


Slide courtesy of
Andrew Markham

(
http://
www.comlab.ox.ac.uk
/people/
andrew.markham
/)

Magneto
-
Induction


Slide courtesy of
Andrew Markham

(
http://
www.comlab.ox.ac.uk
/people/
andrew.markham
/)

Challenges for Indoor

4m x 1m antenna
-
> 5m^3 tracking

We don’t have that much space!

We use a 8cm x 1.5cm transmitter antenna

And 3D coil IC (2cm x 1cm) as receiving antenna

Power supply issues

Outdoor
vs

indoor

Interference issues

Phone magnetometer (digital
compass)






Microsoft
LivePulse

Mains
Power

AD/DC/DC
C
onverstion

90
-
240VAC
-
> 12VDC
-
> 5VDC
-
> 3.3VDC

Mechanical
Relay

Hall
-
effect

Sensor

MI MOSFET Driver

125kHz MI Transmitting Antenna

CC430
SoC

Energy
Meter
IC

900MHz

PCB

Antenna

Microsoft
LivePulse


Precise indoor
localization

Range: ~ 3m
omnidirectional

~10cm
dropoff

zone

Consistent over time

Robust against interference and multipath

Block by metal


Wireless energy
monitoring

Microsoft
LiveLink

3D MI
Receiving
Antenna

Wakeup

Networking

Requirements

Reliable

Globally addressable

Scalable

Low power

Radio Consideration


802.15.4


2.4GHz


863
MHz to
928MHz


100
-
150m range


US


802.15.4c (c for china)


779
-
787
MHz


250m range


Core module choices


60USD Epic core


20USD
SuRF

core


Energy


Networking Overview

One
-
hop

IPv6 communication
over 802.15.4c

JSON /
RESTful

services over
Internet

Repository of
virtual objects

Virtual Object Abstraction Layer

apartment


bedroom office bathroom hallway


desk desktop room portable

lamp computer lamp heater


coffee laptop piano pinball

machine computer keyboard machine


LCD computer


monitor tower


Hierarchical


Event based

Object Representation

States

E.g. On/off

Commands

E.g. turn on / read power / set temp / count
people

Events

E.g. door opened / light
-
off / over
-
threshold

Children

Ptr

to children objects



name: office

states:


lightOn
? : True


doorOpen
? : False


roomOccupied
? : True


roomTemp
? : 30C


roomEnergy
? : 200W

commands:


toggleLight



numOccRoom


lastTimeDoorOpened





events
:



notifyDoorOpen


notifyNewOccupant


notifyHourlyEnergyUse

c
hildren [ipv6]:


coffeeMachine


deskLamp


laptopComputer


desktopComputer


pianoKeyboard


roomLamp


pinballMachine


portableHeater


1. DISCOVERY_REQ

2. DISCOVERY_REPLY

t
o http://<ipv6OfOffice>

3
. EVENT_SUBSCRIPTION [
notifyNewOccupant
]

4. NOTIFICATION

“let me know if someone enters room”

“someone just entered your room”

Open Source Standard

JSON is the object encoding

SOAP / WSDL /
Zigbee

SE Profile


Nooo

REST/HTTP is the interchange scheme

HTTP_GET / HTTP_GET / HTTP_RESPONSE

IPv6

Interoperate with existing IP devices without
understanding additional
protocols

Applications

Personal energy footprint

Social networking

Health care

Related Efforts

Smart meters (via utilities)

Enables time
-
of
-
day billing

Google
PowerMeter

Visualization of whole
-
house usage

Defining a API for energy data

Microsoft
Hohm

Complimentary work (help learn signatures)

House
-
level / NILM / can benefit from DB of
appliance signatures

In the right direction, but not enough

Only at the whole
-
house level

Lack of actuation / control

Source
: http://www.microsoft
-
hohm.com

Source
:
X. Jiang Dissertation

Individualized Energy

Feedback and Control


Real
-
time energy
apportionment


Individualized energy
accounting / billing


Energy map / trail of occupants


Alerts and abnormally
detection


Via cellphone


Remote alerts


Control and actuation


Automated control of devices


Scheduling

An Energy
-
Centric Ecosystem



Social Network of Energy

Social network based on real
-
time individual
energy usages

Windows
7 Gadget

Windows Mobile 7
App

Facebook app

Reward systems


Building managers have fine
-
grain control and
view

mobile application

w
eb application
-

synergy

Social Network of Energy

Localization

Location Status updates, kind of like Foursquare
Checkins
.


“Mike is now in the coffee room.”

Apportionment

Now that we have localization primitive, we can
specifically attribute the use of public energy to
individual users. Very useful.


“Attribute the last 3 minutes of coffee energy usage to Jeff.”

Add this to personal monitoring of private
appliances, and we now have a complete carbon
footprint.

w
eb application dashboard
-

map

Social Network of Energy

Map of User’s owned areas…and where
energy usage is taking place the most

Social Interaction

Compare Energy Usage of Specific Appliances, or
aggregate

Games


User VS. User [One
-
Month Challenge, percent
improvement?]


Team VS. Team




web application dashboard
-

energy

Social Network of Energy

Add Appliances [Home and Work]

Energy Usage Trends

Day to Day Comparisons in a table

Energy Usage Graphs [Compare different devices]

Email Summary of Energy Usage to User
[each day, or week, or month]


w
eb application


user profile

Social Networking “Wine Party” App

Frequently, it is very difficult to meet people you
click with at social

gatherings.

Ex.
You are at a company party. You are bored.

You pull out your mobile phone. A map of the
room you are in shows up, and there are
markers on the map.

Click on a marker, and you can see that person’s
interests, details, hobbies, etc.

See

twitter feed,
facebook

profile, etc.

When someone who match your interest pass
by, your phone vibrates to alert you.


Health Care

Room is aware of status of elderly occupants

1. phone insert trigger / event

2. object representing the room

3. room send a signal to alert

Conclusion


Bi
-
direction
interaction
between
humans and
physical objects


For humans to have real
-
time
fine
-
grained
knowledge and control
over physical objects


Environment to be aware
of humans, and make
intelligent decisions,
proactively and
reactively


Precise indoor localization
primitive


Reliable IPv6 based
networking


Uniform virtual
representation of
physical
objects


Thank you

Questions?







Feel free to contact me at
fxjiang@microsoft.com


http://research.microsoft.com/people/
fxjiang