Jarrett Fishpaw, Business Operations Manager

viraginityfumblingSoftware and s/w Development

Nov 2, 2013 (3 years and 10 months ago)

55 views

© 2010 Cisco and/or its affiliates. All rights reserved.

Cisco Confidential

1

© 2010 Cisco and/or its affiliates. All rights reserved.

Cisco Confidential

1

© 2010 Cisco and/or its affiliates. All rights reserved.

Cisco Confidential

1

Cisco Confidential

1

© 2010 Cisco and/or its affiliates. All rights reserved.

Jarrett Fishpaw, Business Operations Manager

Tyler Munger, NMO Lab Intern


UC Santa Cruz, ISM 101: Thursday, February 28, 2013

© 2010 Cisco and/or its affiliates. All rights reserved.

Cisco Confidential

2


Jarrett’s Personal Background


Cisco Systems Overview


What is Cisco Services


Cisco Services Technology Group


Smart Analytics


‘Actionable Insight’
TM


The NMO Lab and Example Data Analytics Projects


Day in the life of a Cisco Employee


© 2010 Cisco and/or its affiliates. All rights reserved.

Cisco Confidential

3


B.A. Business Management Economics (2009)


M.S. Applied Economics and Finance (2010)


Business Operations Manager, Cisco Systems

Contractor from June 2010
-

April 2011

FTE since May 2011


Mayor of Los Altos (2012)

Joined City Council in 2010

Job Role:


50%
-

Incubation and Innovation of new analytical models

Automated Router Configuration and Simulation


50%
-

Specific projects/Team Administration

Enhanced
NPA

© 2010 Cisco and/or its affiliates. All rights reserved.

Cisco Confidential

4


At

Cisco (NASDAQ:
CSCO
)

customers come first and an integral part
of our DNA is creating long
-
lasting customer partnerships and working
with them to identify their needs and provide solutions that support their
success.


Husband and wife Len
Bosack

and Sandy Lerner, both working for
Stanford University, wanted to email each other from their respective
offices located in different buildings but were unable to due to
technological shortcomings.


A technology had to be invented to deal with disparate local area
protocols; and as a result of solving their challenge
-

the multi
-
protocol
router was born.


Founded:

San Fran
cisco
, California, U.S. (1984)

Len
Bosack
,
Sandy Lerner
,
Richard
Troiano
,
Nicholas Pham


© 2010 Cisco and/or its affiliates. All rights reserved.

Cisco Confidential

5


© 2010 Cisco and/or its affiliates. All rights reserved.

Cisco Confidential

6


© 2010 Cisco and/or its affiliates. All rights reserved.

Cisco Confidential

7


© 2010 Cisco and/or its affiliates. All rights reserved.

Cisco Confidential

8


© 2010 Cisco and/or its affiliates. All rights reserved.

Cisco Confidential

9


Q1

FY'13

Revenue: $11.9 billion

Q1

FY'13

Employee Count:
72,360

© 2010 Cisco and/or its affiliates. All rights reserved.

Cisco Confidential

10

Technical
Assistance
Center

© 2010 Cisco and/or its affiliates. All rights reserved.

Cisco Confidential

11


Employees Worldwide:

11,400+


Certified Professionals:

CCIE
= 2,000+


Patents

~700 since 1998


Cisco Partners



Services is responsible for 80% of the revenue coming from Cisco Partners



There are over 280,000 partner employees which provide channel leverage



Over 68,000 authorized/registered partners



Provide support in over 180 countries worldwide

© 2010 Cisco and/or its affiliates. All rights reserved.

Cisco Confidential

12


The Cisco Services Technology Group drives technology
innovation and software development to accelerate the evolution
of our Advanced and Technical services portfolio.


Our global organization consists of five primary groups:

Technology Development

Support

Business Operations

Globalization

Expertise


We are accountable for developing, delivering, and supporting the
technologies Cisco Services deploys in our markets.

© 2010 Cisco and/or its affiliates. All rights reserved.

Cisco Confidential

13

Who Are These People?


Think of him or her as a
hybrid of data hacker,
analyst, communicator,
and trusted adviser.


The
combination is
extremely powerful

and
rare.


What are their traits?


Intense curiosity


Associative thinking

Credit Card fraud


DNA
sequencing

Finance
-

Pharmaceutical
Trials
-

Network Reliability


© 2010 Cisco and/or its affiliates. All rights reserved.

Cisco Confidential

14

Analysts

Competitors

IBM opens a Services Innovation Lab
(
SIL
)…the lab will accelerate the
expansion of real
-
time analytics … in both
IBM’s technology services offerings and its
global services delivery capabilities.
(4/2011)

IBM has committed $2 billion in service R&D


Partners & Customers

Smart Analytics represent an excellent
delivery of intellectual capital…This type of
IC scored in the top 3 of partner demands
in the partner focus groups regarding
…desirable service elements


-

Market Intelligence Team,
5/2011


IT complexity will soon outstrip customer’s IT
management capabilities


Forrestor


Network managers have more information than
ever … Making sense of the information is the
essential challenge… trends will see the
effective transition of development priorities from
network technology to IT analytics




IDC, 3/2011


Role of Analytics in Complex Networks

© 2010 Cisco and/or its affiliates. All rights reserved.

Cisco Confidential

15


Understand network
health


Set and measure
progress against
network
improvement plans
and change
initiatives


Compare against
peers in the same
vertical on certain
metrics such as
technology maturity
and operational
maturity

© 2010 Cisco and/or its affiliates. All rights reserved.

Cisco Confidential

16


Big Data requires:

Hadoop

SAS

R

Denodo


Domain Experts Collaborating with Statisticians


‘Big Data’ combined with Analytics

© 2010 Cisco and/or its affiliates. All rights reserved.

Cisco Confidential

17

Linear

Regression

Curve Fitting

ARIMA



Autoregressive Integrated Moving Average

Data Clustering

Data Mining

Correlation Analysis

Time Series
-

ARIMA

Models

Predictive Forecast

Multivariate Regressions

Cross Sectional Data

Historic Trending

Process data using advanced tools, and domain expertise

Techniques:

Multivariate
Regressions

© 2010 Cisco and/or its affiliates. All rights reserved.

Cisco Confidential

18

Delivery of Intuitive Insight



Industry Benchmarking



Root Cause Analysis



Predictive Forecasting



ROI

Simulation



Visualization


© 2010 Cisco and/or its affiliates. All rights reserved.

Cisco Confidential

19


Projects
: Short
-
term undergraduate internships (1
-
3 quarters) and Masters/PhD
level research (3+ quarters
) at Cisco


Wide range of interesting projects from hardware testing to Supply Chain
Management


Real world data and problems


Potential users (work has a high possibility of actually being used)


Time
: Generally 20 hours a week. Usually work from home and go into Cisco
about once a week


Resources


Networking labs (UCSC and Cisco)


Servers


Domain experts

© 2010 Cisco and/or its affiliates. All rights reserved.

Cisco Confidential

20


BS in Computer Science from UCSC (2009)


Graduate Student
in the
Technology and Information
Management (TIM)
program
working under
Subhas

Desa


Been part of the NMO lab for 4 years


1 year as an undergraduate student


3 years as a graduate student


Research areas


Data mining/Information retrieval


Software engineering


Computer networks


Knowledge engineering

© 2010 Cisco and/or its affiliates. All rights reserved.

Cisco Confidential

21


When a customer has a problem with a Cisco
networking product they open a service request
with Cisco’s support teams


Cisco support
teams receive thousands of
service
requests
every
month on a wide variety of
Cisco
networking products


How do we detect emerging (growing) problems
before they impact a large number of customers?


Software bugs


Viruses


Hardware failures

“We have one ASA with 2 ISP
connections and several site to
site VPN’s set
-
up to other sites
with ASAs. When we test failing
over the VPN’s to use the second
ISP connection the tunnels are
not coming up. We have
confirmed that outbound routing
fails over immediately.”

Typical Customer
Service Request

Primary Investigator: Tyler
Munger

Advisors: Chris Wong (Cisco),
Subhas

Desa

(UCSC)

© 2010 Cisco and/or its affiliates. All rights reserved.

Cisco Confidential

22

1.
Data Mining
: identify
the
associated product
failure mode
for customer
service requests


Support
Vector
Machines
(classification)

2.
Prediction
: generate an alert when a particular failure mode starts
abnormally increasing in frequency (indicating an emerging problem)


Double Exponential Smoothing

(forecasting)


Control
Charts

(alerts)

3.
Software Engineering
:
develop an easy to use software tool that allows
Cisco Engineers
to monitor
customer service requests for emerging
problems


Weka

(data mining)


Ruby
on Rails

(webserver
)


MySQL

(database
)


Adobe
Flex

(visualization and GUI
)



© 2010 Cisco and/or its affiliates. All rights reserved.

Cisco Confidential

23



Identified 20
key
failure
modes for
a
popular
Cisco
network security
product (
ASA
5505)


Classified a
dataset of ~
10,000
service requests by failure mode
using a
Support Vector Machine
c
lassifier (SVM)


Dominant
failure modes were
“failover communication between
devices”, “VPN tunnels
disconnected after switchover”,
“configuration not in sync after
switchover”








Failure Mode

Frequency

Failover
communication
between devices

1414

VPN tunnels
disconnected after
switchover

922

Configurations not in
sync after
switchover

609

No switchover

450

Hardware

224





© 2010 Cisco and/or its affiliates. All rights reserved.

Cisco Confidential

24


Developed a web
-
based
software tool for Cisco to
use for monitoring emerging
issues


Used the tool to identify
emerging problems in
historical customer service
requests


Increase in Virtual Private
Networking (VPN)
problems caused by a bug
in a new software release

© 2010 Cisco and/or its affiliates. All rights reserved.

Cisco Confidential

25


Customer
: increased network reliability because potential

problems are
identified and resolved before they occur


Cisco
: increased customer satisfaction
and
reduced support costs
because customers are opening less service requests

© 2010 Cisco and/or its affiliates. All rights reserved.

Cisco Confidential

26


Cisco Internet Operating Systems (IOS) is the core piece
of software that runs a wide range of Cisco networking
products (routers, switches, firewalls, etc.)


There are hundreds of different IOS software releases


Older releases are more reliable but have less features


Newer releases have more features but are less reliable


How do we recommend the most reliable IOS
software
release that has the features that the customer needs?

Primary Investigator:
Yali

Mu

Advisors: Jarrett
Fishpaw
/Chris Wong (Cisco),
Subhas

Desa

(UCSC)

© 2010 Cisco and/or its affiliates. All rights reserved.

Cisco Confidential

27


The reliability of an IOS software
release
can measured by the number
of software bugs that have not been
found
yet (unfound bugs)


How do we estimate the number of
unfound bugs for each IOS Software
Release?

1.
Create a bug forecast using historical
bug data

2.
Determine the total number of bugs for
the software release based on the
forecast

3.
Calculate the number of unfound bugs


Develop an easy to use software tool
that allows Cisco Engineers to easily
select the most reliable software
release for a particular customer
network


Bugs Found

Forecast

Total Number of Bugs

Unfound Bugs

© 2010 Cisco and/or its affiliates. All rights reserved.

Cisco Confidential

28


Created reliability forecasts for over 5,000 IOS software
releases across 36 different product families


Forecasts were used to
create 3
reliability
estimates for
each IOS
software
release

1.
Total number
of
bugs

2.
How
many unfound bugs

3.
When
all the bugs will be found


© 2010 Cisco and/or its affiliates. All rights reserved.

Cisco Confidential

29


Customers:

network reliability improves
because they
can select the
most reliable
IOS software release for
their
network


Cisco:
increased customer satisfaction and reduced support costs
because customers have less network problems


© 2010 Cisco and/or its affiliates. All rights reserved.

Cisco Confidential

30


Many employees have a unique and flexible work schedule


WebEx Meetings


Outlook Calendar

© 2010 Cisco and/or its affiliates. All rights reserved.

Cisco Confidential

31


Each manager has a different style


Responsiveness to Email and IM’s (Phone/text occasionally)


Personal discipline to manage multiple projects simultaneously


Weekly Breakfast Club on our team



© 2010 Cisco and/or its affiliates. All rights reserved.

Cisco Confidential

32

Cisco Business questions


What does Cisco do?


Are Cisco products food or internet related?


Do you use Cisco products?


What does a Data Scientist do?


What
year was Cisco Founded
?


How large
is Cisco (people/revenue
)?


How many authorized partners does Cisco have?

Cisco
Certification questions


Name
2 Cisco networking
certifications