which alerts users of

Leveraging big traffic data
to create a web service
which alerts users of
potential traffic congestion

Donna Thomas

Jacob Rettig

Justin Lawrence

CMSC 668

October 1st, 2012

Project

Description

Using Big Data sets, we are mining for potential
congested traffic patterns which may impair
one’s timely commute during rush hour. When
we discover potential congested traffic
patterns, our web service will send SMS and
Email notifications to service subscribers.

Users

•
NYC Tri
-
State
Area
Users

Users stuck in traffic

Users querying traffic conditions

Link and Event driven

Civic responsibility


Scenarios

User Inputs:

•
Registration for accounts

•
Registration for traffic alerts

•
Search for alerts by

o
X miles from City/Address

o
X miles from Latitude/Longitude

o
Path between point A and B

Output to Users:

•
List of traffic alerts of interest

•
Alerts on Google Maps

•
Alerts sent to registered users via

o
email, SMS text, etc.

Functional Requirements

Technologies and Hardware Used

HADOOP

MapReduce

Apache Tomcat

WSDL

JAX
-
RPC/XML
-
RPC

SOAP

AXIS

Google Maps
API

TRANSCOM data set

SMTP Server

BlueGrit

Cluster

Data sources

TRANSCOM

Google Maps

Our project’s design

Based on project requirements:

Bluegrit

Cluster, using SOAP in the Cloud

Services and Network Architecture

•
UMCP
CapWIN

•
I
-
95 Corridor

•
TRANSCOM/Xerox

•
INRIX

•
Dr. Kalpakis
-

Traffic Models

•
Dr.
Finin

-

Twitter

(
TweetCollector

+
TweetGobbler
)

•
Dr. Sherman
-

CapWIN

(UMCP
collaboration)

Related Works

•
User Login

•
Basic Authentication

•
Register Alerts


Security

•
After
-
class meetings to discuss ideas and
direction

•
Skype for remote meetings and
discussions

•
Microsoft SkyDrive for creating
collaborated documents

Team Approach

•
Midterm:

o
Design Database schema and begin to pull data from TRANSCOM

o
Design
MapReduce

queries

o
Develop querying and polling services

o
Create SMTP server for notifications

o
Integrate output with Google Maps API




•
Final:

o
Finalized Project Report

o
Provide Demonstration


Performance and Results

•
Big data analysis interpretation

•
Event driven path analysis

•
Getting access to traffic data




Challenges

•
Communicate with data providers (Donna)

•
Register to receive data from data providers (Donna, Justin and
Jacob)

•
Installation/setup of:

o


Hadoop

(Justin), AXIS (Jacob), Tomcat (Jacob), HDFS (Justin), HBASE (Justin),
MapReduce

(Justin), SMTP (Justin)

•
Security: (Jacob)

o


SOAP authentication headers, web GUI authentication

•
Create ontology/schemas types:

o


HBASE (Justin), SOAP messages (Donna)

•
Data Ingest service(s):

o


TRANSCOM (Jacob, Justin, Donna)

•
User registration services: (Donna)

o


user accounts, traffic alerts registration

•
Querying services (using
MapReduce
): (Justin)

o


Querying traffic data, Traffic alert polling

•
Messaging service to send: (Justin)

o



emails, SMS texts, RSS, SOAP

•
Web GUIs: (Donna, Jacob, Justin)

o


Google Maps, User Services, Querying Services

•
Testing: (Donna, Jacob, Justin)

•
Writing the Paper: (Donna, Jacob, Justin)

Responsibilities

Allow people travelling through the New York tri
-
state
area to avoid traffic congestion by leveraging
event and link driven data from TRANSCOM.


Summary

Technical Data:


Google Maps APIs

http://code.google.com/apis/maps/index.html

TRANSCOM dataset

http://www.xcom.org

WebServices

–

Axis

http://axis.apache.org/axis/java/install.html



Credits/References

Research/Publications:


Barry, Keith. "The Future of In
-
Car Technology."
Car and Driver
.
N.p
., 1 Apr.
2012. Web. 01 Oct. 2012. <http://www.caranddriver.com/features/the
-
future
-
of
-
in
-
car
-
technology>.

"CATT Homepage, Clark School of Engineering, University of Maryland."
CATT
Homepage, Clark School of Engineering, University of Maryland
. Center of
Advanced Transportation Technology,
n.d
. Web. 01 Oct. 2012.
<http://catt.umd.edu/>.

Chan, Alice. "Mercedes
-
Benz Dashboard
Crowdsources

Parking Spots For
Drivers Via Twitter
-

PSFK."
Mercedes
-
Benz Dashboard
Crowdsources

Parking Spots For Drivers Via Twitter
-

PSFK
.
N.p
., 12 Mar. 2012. Web. 01 Oct.
2012. <http://www.psfk.com/2012/03/mercedes
-
benz
-
twitter
-
parking
-
spots.html>.

Chang, Fay, Jeffrey Dean, Sanjay
Ghemawat
, Wilson C. Hsieh, Deborah A.
Wallach, Mike Burrows,
Tushar

Chandra, Andrew
Fikes
, and Robert E.
Gruber. "
Bigtable
: A Distributed Storage System for Structured Data."
OSDI'06: Seventh Symposium on Operating System Design and
Implementation

(2006): 1
-
14. Web. 30 Sept. 2012.
<http://static.googleusercontent.com/external_content/untrusted_dlcp/r
esearch.google.com/en/us/archive/bigtable
-
osdi06.pdf>.

Credits/References

Chouffani
,
Reda
. "4 Questions to Ask Before Starting a Big Data Initiative."
CIO
.
N.p
., 21 Aug. 2012. Web. 01 Oct. 2012.
<http://www.cio.com/article/714309/4_Questions_to_Ask_Before_Starting_
a_Big_Data_Initiative>.

Das
Sarma
,
Anish
,
Lujun

Fang,
Nitin

Gupta,
Alon

Halevy,
Hongrae

Lee,
Fei

Wu,
Reynold

Xin
, and Cong Yu.
Finding
Realted

Tables
. Proc. of SIGMOD ’12,
Scottsdale, Arizona, USA.
N.p
.: ACM, 2012. 1
-
12. Print.

Harris, Derrick. "How
Facebook

Made It Possible to Geo
-
tag Everything
â

Cloud Computing News."
GigaOM
.
N.p
., 9 Mar. 2012. Web. 01 Oct.
2012. <http://gigaom.com/cloud/why
-
every
-
location
-
tag
-
on
-
facebook
-
is
-
big
-
data
-
in
-
action/>.

Holt, Kris. "Toyota Lets Drivers Tweet from the Dashboard."
Daily Dot
.
N.p
., 28
Sept. 2012. Web. 01 Oct. 2012. <http://www.dailydot.com/news/toyota
-
corolla
-
twitter
-
dashboard/>.

Horvitz, Eric. "Predictive Analytics for Traffic."
Microsoft Research
.
N.p
.,
n.d
.
Web. 1 Oct. 2012. <research.microsoft.com/en
-
us/projects/
clearflow
/default.aspx>.

Credits/References

Kuehnhausen
, Martin, and Victor S. Frost.
Transportation Security
SensorNet
: A
Service Oriented Architecture for Cargo Monitoring
. Rep. no. ITTC
-
FY2010
-
TR
-
41420
-
22.
N.p
.: University of Kansas Information & Telecommunications
Technology Center, 2010. Print.

Mazaré
, Pierre
-
Emmanuel, Olli
-
Pekka

Tossavainen
,
Alexandre

M.
Bayen
, and
Daniel B. Work. "Trade
-
offs between Inductive Loops and GPS Probe
Vehicles for Travel Time Estimation: A Mobile Century Case Study."
Trade
-
offs between Inductive Loops and GPS Probe Vehicles for Travel Time
Estimation: A Mobile Century Case Study
.
N.p
.,
n.d
. Web. 1 Oct. 2012.
<https://netfiles.uiuc.edu/dbwork/www/pdf/TRB12.pdf>.

Newcomb, Doug. "
Inrix

|
Autopia

| Wired.com."
Wired.com
.
Conde

Nast
Digital, 05 June 0012. Web. 01 Oct. 2012.
<http://www.wired.com/autopia/tag/inrix/>.

Ovide
,
Shriva
. "Tapping 'Big Data' to Fill Potholes."
Tapping 'Big Data' to Fill
Potholes
. Wall Street Journal, 12 June 2012. Web. 1 Oct. 2012.
<online.wsj.com/article/SB10001424052702303444204577460552615646874.
html>.

Staff,
Datanami
. "What It Takes to Deliver Real
-
Time Traffic Info
-

Datanami
."
What It Takes to Deliver Real
-
Time Traffic Info
-

Datanami
.
Datanami
, 01
Aug. 2012. Web. 01 Oct. 2012.
<http
://www.datanami.com/datanami/2012
-
08
-
01/what_it_takes_to_deliver_real
-
time_traffic_info.html>.

Credits/References

?

Questions