Big Data Analytics in Practice

nostrilshumorousInternet και Εφαρμογές Web

18 Νοε 2013 (πριν από 3 χρόνια και 8 μήνες)

48 εμφανίσεις

The 8th Korea Communication Conference

2012.05.15

Saltlux, Inc.

Tony LEE

The 8
th

K
orea
C
ommunication
C
onference
2012

Big Data Analytics in Practice

The 8th Korea Communication Conference

2

9 Myths of Big Data


Big Data ‘KUMIHO’, nine
-
tailed fox

The 8th Korea Communication Conference

3

“Big Data means


Big Volume

of data set”

Myth #1

“Big Data means

Big
Difficulties

to process it”

Truth #1

“4V := Volume, Velocity, Variety + Value”

The 8th Korea Communication Conference

4


Bigger Insight

comes with

Bigger Data


Myth #2

“Bigger insight

comes with meaningful

Data Selection


Truth #2

“No
Garbage
-
in,
Gold
-
out”
-

At
least
goldstone
-
in

The 8th Korea Communication Conference

5

“Big Data Analytics

Means
Social

Media
Analytics”

Myth
#3

“Social data is

one of
big data

sources


Truth #3

“It is not enough to understand the World”

The 8th Korea Communication Conference

6

“The goal of Big

Data Analytics is

Prediction


Myth #4

“Right understanding

and
optimization

Of current status”

Truth #4

“The best way to predict the future is to create it”

The 8th Korea Communication Conference

7

“New
technology

drives the success of

Big Data Analytics”

Myth #5

“Clear
Goal

and

Human

centric task

drive the success”

Truth #5

“People
People

People

under the Clear Vision"

The 8th Korea Communication Conference

8


Hadoop

is

essential

tech. for

big data analytics”

Myth #6


Hadoop

is
one of

good tech.
tools

for big data anal.”

Truth #6

“Do not use a hammer to crack a nut"

The 8th Korea Communication Conference

9

“Big data tech.

will lead
Huge

IT Market


Myth #7

“A
few

companies

will
create

values

from big data”

Truth #7

“It’s not Buzz, but be careful“

-

you wouldn’t be.

The 8th Korea Communication Conference

10

“Big data business

is an extension of

old BI


Myth #8

“BI is one of

applications
,

using big data”

Truth #8

“It’s Not an Old Wine in New Bottle!“

The 8th Korea Communication Conference

11

“Big data analysis

is
for decision

maker

like CEO”

Myth #9


Common people

will
enjoy

the

value of big data”

Truth #9

“Invisible and Calm Big Data Analytics“

The 8th Korea Communication Conference

12

Understanding

Big Data Analytics

The 8th Korea Communication Conference

13

What
kinds of

data

we have?

King of Data :

Linking Open Data

+ data from social media

+ enterprise and private

+ closed government data

gov.

and pub. open data

The 8th Korea Communication Conference

14

Enterprise Strategy Group, 2010

Almost of them are

unstructured

90% is unstructured

However, we don’t utilize it enough

Unified analysis combining structured and

unstructured data will be key success factor

The 8th Korea Communication Conference

15

Stream Data

World, we never seen

sensor networks, social networks,

web of data, IOT, M2M ...

The 8th Korea Communication Conference

16

Social

Data

Enterprise

Data

Financial

Data

Telcom
.

Data

Security

Data

Health

Data

Trend Analysis

Voice of Customer

Reputation Analysis

Competitive Analysis

Risk Detection/
Mngt

Fraud Detection

Big Data

Analytics

Platform

Advanced

Analytics

Advantages

Cloud Computing Infra

Data Collection,

indexing and
Mngt
. Infra

Analytic Tech. Infra

NLP, ML, Statistics, Semantic, AI

Analy
. Service Component

Classification., Clustering, Trends, Sentiment

Analysis Workflow System

Visualization and Application

Parallel/
Dist

Comp Infra

Hadoop
,
NoSQL
(
HBase
,
mongoDB
, …)

Real
-
time

Marketing

Optimization

Competitive

Strategy

Optimization

Dynamic

Cost

Optimization

New business

or Policy

Development

Risk Management

and Optimization

Big data

Analytics Platform

?

Manufact
.

Data

Manufac. system anal.

Production

System

Optimization

The 8th Korea Communication Conference

17

Quality comes from
Process


Analysis is sophisticate process : Workflow system is key


Dynamic collaboration between analysts and machine

The 8th Korea Communication Conference

18

Big data analytics

Technologies

NLP

Machine

Learning

Text

Mining

Cloud,

NoSQL

IR

(Search)

Semantics

Crawling

In
-
memory

Analytics

Statistics

(R)

Visual
-

ization

The 8th Korea Communication Conference

19

Value Creation

The 8th Korea Communication Conference


Anomaly detection from (social) big data


Issue detection and decision supporting


transparency improvement and cost reduction


Competitive analysis from social big data


Context
awareness &
AI

based smart services


Next generation BM based on personalization


New value creation from heterogeneity


Reducing ‘trial & errors’ by connected patterns


Fusion market creation from conversions


Understanding real
world pattern,
forecasting


Understanding big
picture under
complexity


Understanding social signal and simulation

20

Is Big data a Future
Growth Engine
??

Uncertainty

Risk

Smart

Conversions

Insight

Counterplan

Competition

Creativity

Feature of Future

Expected Role and Values of Big Data

source : NIA, 2011.12.30

The 8th Korea Communication Conference

21

3 + 1

Big Values


take two!

Quality

Cost

Speed


Financial Fraud Detection


Run
-
time Marketing Optimization


Real
-
time Production Optimization


Business & Policy Develop.


Competitive Analysis


City Surveillance


Personalized Services


VOC, Customer
Understan
.


Medical, Healthcare

Sustain
-

ability


Risk Management


e
-
Discovery


Security and Defense


Environment Conservation

The 8th Korea Communication Conference

22

Big Data
5
-
Step Business Models

Step 1

Step 2

Step 3

Step 4

Step 5

The 8th Korea Communication Conference

23

McKinsey Said
the Value
of Big Data

$300 billion

potential annual value to US health care more than

double the total annual health care spending in Spain


250 billion

potential annual value to Europe’s public sector

administration more than GDP of Greece

$600 billion

potential annual consumer surplus from

using personal location data globally

60%

potential increase in retailers’ operating margins

possible with big data

1.5 million

more data
-
savvy managers needed to take

full advantage of big data in the United States

The 8th Korea Communication Conference

24

IDC
also said about Big Data

Big Data Phenomenon is REAL.

1.
The
market for Big Data technology and services will reach
almost 17 billion dollars by 2015, up from a mere 3 billion
dollars in 2010

2.
This
is a 40
%
a
-
year growth
-
rate, and about seven times
the estimated growth rate of the overall ICT market

3.
Server 27.3%, Software

34.2%, Storage

61.4% growth

4.
Appliance and Cloud based service will be important

(IDC, March 7, 2012)

The 8th Korea Communication Conference

25

Applications

Working on Big Data

The 8th Korea Communication Conference

Applications
of Big Data Analytics


26

time

un
-
structurality

Social Media Analytics

(Trends, Sentiment, Issue detection and etc.)

Financial, Telecom fraud detection

City Surveillance, Disaster Management

Mobile Service Personalization

Defense, Security,
eDiscovery

Policy, Business Development

Voice of Customer /Citizen Analysis

Medical, Healthcare Services

1s




1m




1h




1d




1w


Tech. / Research

Data

Analysis

structured

semi
-
structured

un
-
structured

The 8th Korea Communication Conference

27

Public Data

Social Data

Enterprise Data

Reputation Analysis

Social Issue Detection

Medical Service Opt.

Welfare Optimization

Service Personalization

Gov. Policy Optimization

Risk Management

e
-
Discovery

Fraud Detection

Marketing Optimization

VoC

Analysis

Crime Protection

Disease Protection

Defense, Security

City Surveillance

Biz Strategy Optimization

Applications
of Big Data Analytics


The 8th Korea Communication Conference

Telecom

Big Data

: Personalization

28

The 8th Korea Communication Conference

29

Telecom

Big Data

: Hybrid Reasoning

The 8th Korea Communication Conference

30

30

Company

Big Data

: Search + Analytics

The 8th Korea Communication Conference

31

Company

Big Data

: e
-
Discovery & Compliance

The 8th Korea Communication Conference

32

Customer

Big Data

: VOC Analysis

The 8th Korea Communication Conference

33

Technical

Big Data

: Trend Sensing

The 8th Korea Communication Conference

34

Research

Big Data

: experts recommend.

The 8th Korea Communication Conference

Social

Big Data

: Trends, Reputation

35

TrueStory.co.kr

The 8th Korea Communication Conference

Sensor Monitoring

Discover Leakage Area

Infer Leakage Pipe Link

Automatic Alert

Recom
.
Detour Path

Leakage Detection

36

Smart City

Big Data

: Pipeline mgmt.


The 8th Korea Communication Conference

37


Traffic data from Milano (Italy)


Data ranging from Mar. 07 to July
09


5 min. sampling rate for flow & speed


Traffic flow & speed from



209

sensors that are able to


classify vehicles, and



757

non classifying sensors


Weather data provided from

http://www.ilmeteo.it


1 hour sampling rate for weather data

Milano City Sensor Map

Sensors



Crossroads



Street Categories (multi
-
colored)

Smart City

Big Data

: Traffic Optimization


Traffic prediction for Milano based on traffic sensor data


Hybrid (ontology and ML) reasoning for stream data set

The 8th Korea Communication Conference

38

-

CLOSING
-

The best way to predict the future is to create it.

Let’s manage in a times of Grate Changes
!