reduce - Windows Azure User Group NL

hedgebornabaloneSoftware and s/w Development

Dec 2, 2013 (3 years and 8 months ago)

68 views

Windows Azure Center of Excellence

Spotlight

Pilots

Assessment

Architecture

and Design


Guidance

Modern Apps

Global Scale



Design Sessions

Global Services Team

10 Senior Cloud Architects

Dennis

Mulder

US, EMEA, APAC

8

Pilots

Cloud Apps

Champs

Services

Dennis Mulder, Solution Architect, dmulder@microsoft.com

Design

Assess

Contact

Pilots

Engage

Social

Mobility

mobile apps

will be

downloaded

in 2012

=

91%

of organizations expect
to spend on mobile
devices in 2012

1/2

of
companies
expect to use
internal social
network apps
in 2012

2.7

zettabytes

in 2012

>80%

of new apps in
2012 will be distributed/

deployed on clouds

32%

of businesses
are likely to
invest in BI
and analytics
in 2012

from infrastructure
to
application
platforms

The strategic focus
in the cloud

will shift

in 2012

In 2012, mobile
devices will
outship

PCs by
more than

2:1

and generate more
revenue than PCs
for the first time

85

BILLION

Social networking will
follow not just people
but also appliances,
devices and products

34%

of CIOs say technology as a service
(cloud) will have the most profound
effect on the CIO role in the future

2/3

of mobile apps

developed in 2012 will integrate
with analytics offerings

49%

of CIOs rank BI as the
top project priority for 2012

Big data

Cloud

Four m
egatrends will dominate
the
next decade

mobile apps

will be

downloaded

in 2012

=

91%

of organizations expect
to spend on mobile
devices in 2012

1/2

of
companies
expect to use
internal social
network apps
in 2012

2.7

zettabytes

in 2012

>80%

of new apps in
2012 will be distributed/

deployed on clouds

32%

of businesses
are likely to
invest in BI
and analytics
in 2012

from infrastructure
to
application
platforms

The strategic focus
in the cloud

will shift

in 2012

In 2012, mobile
devices will
outship

PCs by
more than

2:1

and generate more
revenue than PCs
for the first time

85

BILLION

Social networking will
follow not just people
but also appliances,
devices and products

34%

of CIOs say technology as a service
(cloud) will have the most profound
effect on the CIO role in the future

2/3

of mobile apps

developed in 2012 will integrate
with analytics offerings

49%

of CIOs rank BI as the
top project priority for 2012

Social

Mobility

Big data

Microsoft is embracing these megatrends

Cloud

How will technology megatrends enable you to save money,

drive innovation, grow your business, and attract and retain customers?

Rethinking and
e
volving business strategies

Social

Big data

Mobility

Cloud




Internet of things

Audio / Video

Log Files

Text/Image

Social Sentiment

Data Market Feeds

eGov Feeds

Weather

Wikis / Blogs

Click Stream

Sensors / RFID / Devices

Spatial & GPS Coordinates


WEB 2.0

Mobile

Advertising

Collaboration

eCommerce

Digital Marketing


Search Marketing

Web Logs

Recommendations


ERP / CRM

Sales Pipeline

Payables

Payroll

Inventory

Contacts

Deal Tracking

Terabytes

(10E12)

Gigabytes

(10E9)

Exabytes

(10E18)

Petabytes

(10E15)


Velocity
-

Variety
-

variability

Volume

1980

190,000$

2010

0.07$

1990

9,000$

2000

15$

Storage/GB


ERP / CRM


WEB
2.0


Internet of things




Map tasks

21

53705

$65

53705

$30

53705

$15

54235

$75

54235

$22

02115

$15

02115

$15

44313

$10

44313

$25

44313

$55

5

53705

$15

6

44313

$10

5

53705

$65

0

54235

$22

9

02115

$15

6

44313

$25

3

10025

$95

8

44313

$55

2

53705

$30

1

02115

$15

4

54235

$75

7

10025

$60

Mapper

Mapper

4

54235

$75

7

10025

$60

2

53705

$30

1

02115

$15

10025

$60

5

53705

$65

0

54235

$22

5

53705

$15

6

44313

$10

3

10025

$95

8

44313

$55

9

02115

$15

6

44313

$25

10025

$95

DataNode3

DataNode2

DataNode1

Blocks

of the

Sales

file in

HDFS

Group

By

Group

By

(
custId
,
zipCode
, amount)

One output bucket
per reduce task

Reducer

Reducer

Reduce tasks

Reducer

53705

$65

54235

$75

54235

$22

10025

$95

44313

$55

10025

$60

Mapper

53705

$30

53705

$15

02115

$15

02115

$15

44313

$10

44313

$25

Mapper

53705

$65

53705

$30

53705

$15

44313

$10

44313

$25

10025

$95

44313

$55

10025

$60

54235

$75

54235

$22

02115

$15

02115

$15

Sort

Sort

Sort

53705

$65

53705

$30

53705

$15

44313

$10

44313

$25

44313

$55

10025

$95

10025

$60

54235

$75

54235

$22

02115

$15

02115

$15

SUM

SUM

SUM

10025

$155

44313

$90

53705

$110

54235

$97

02115

$30

Shuffle

Front
end

Front
end

Stream
Layer

Partition
Layer

Name Node


de




Data Node



Data Node



Front end

HDFS API

DFS (1 Data Node per Worker Role)

and Compute Cluster

Azure Storage
(
ASV)



Azure
Blob Storage

Distributed Storage

(HDFS)

Query

(Hive)

Distributed Processing

(MapReduce)

ODBC
Legend

Red = Core
Hadoop

Blue = Data
processing

Purple =
Microsoft
integration
points and
value adds

Orange = Data
Movement

Green =
Packages

Hive, Pig, Mahout, Cascading, Scalding,
Scoobi
,
Pegasus…


C#, F# Map/Reduce, LINQ to Hive, .NET
management clients

JavaScript Map/Reduce, Browser hosted console,
Node.js management clients

PowerShell, Cross Platform CLI tools

http://
www.windowsazure.com/
http://hadoop.apache.org
/
http://
nuget.org/packages?q=hadoop
http
://
hadoopsdk.codeplex.com
Windows Azure Center of Excellence

Spotlight

Pilots

Assessment

Architecture

and Design


Guidance

Modern Apps

Global Scale



Design Sessions

Global Services Team

10 Senior Cloud Architects

Dennis

Mulder

US, EMEA, APAC

8

Pilots

Cloud Apps

Champs

Services

Dennis Mulder, Solution Architect, dmulder@microsoft.com

Design

Assess

Contact

Pilots

Engage

©
2012 Microsoft
Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademar
ks
and/or trademarks in the U.S. and/or other countries.

The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the
dat
e of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a c
omm
itment on the part of
Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation.
MICROSOFT
MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.