Mining the Madden Experience

unknownlippsΤεχνίτη Νοημοσύνη και Ρομποτική

16 Οκτ 2013 (πριν από 3 χρόνια και 7 μήνες)

66 εμφανίσεις

expressive
intelligence
studio

Mining the Madden Experience

Applying Machine Learning to Telemetry

Ben Weber

UC Santa Cruz

bweber@soe.ucsc.edu

Michael John

Electronic Arts

mjohn@ea.com

expressive
intelligence
studio


UC Santa Cruz

Madden NFL 11

expressive
intelligence
studio


UC Santa Cruz

Madden 2011 Questions




What
gameplay features

impact
player
retention
?



What are optimal
win rates
for
retention
?


expressive
intelligence
studio


UC Santa Cruz

Our Problem



How do we identify the
relation

between
gameplay features and retention?

Gameplay

Features

? ? ?

Player

Retention

expressive
intelligence
studio


UC Santa Cruz

Our Solution



Use machine learning to build
models

of
player behavior



Analyze

generated
models

to identify
influential

gameplay elements

expressive
intelligence
studio


UC Santa Cruz

What is Machine Learning?



Machine Learning
(ML) is branch of AI that
uses
algorithms

to extract patterns from
empirical data



ML is widely used for
prediction

and
forecasting

expressive
intelligence
studio


UC Santa Cruz

What is a Model?



A
function

that maps
input variables

to a
predicted value



Regression models

predict a continuous value



Different ML algorithms generate
different
types
of models

expressive
intelligence
studio


UC Santa Cruz

What can a Model tell Us?


Model analysis
can identify the most
influential
gameplay features

Testing

Data

Model

Predictions

Feature

Tweaking

Analyst

expressive
intelligence
studio


UC Santa Cruz

How We Applied ML

Testing

Data

Models

Predicted
number of
games played

Feature

Tweaking

Analyst

Training

Data

ML
Algorithms

Madden Players

expressive
intelligence
studio


UC Santa Cruz

Our Workflow

Madden

Gamecast

data

Weka

Java Parser

(ETL)

expressive
intelligence
studio


UC Santa Cruz

Madden 2011
Gamecast

Dataset



Gamecast

telemetry


Play
-
by
-
play summaries


Xbox 360 players


August 10
th



November 1
st



350 GB



Sampled 25,000 players

expressive
intelligence
studio


UC Santa Cruz

Extract
-
Transform
-
Load (ETL)



Parse
play
-
by
-
play

data



Convert to
feature vector

representation



Export to
ARFF

format

expressive
intelligence
studio


UC Santa Cruz

ETL Workflow

Play
-
by
-
Play
Data

User DB

Madden

Gamecast

data

ARFF

Files

Parser

(Java)

Feature

Encoder

(Java)

expressive
intelligence
studio


UC Santa Cruz

Gameplay Features


Each player’s behavior is encoded as the following
features (46 total):



Game modes


Usage


Win rates


Performance metrics


Turnovers


Gain


End conditions


Completions


Peer quits



Feature usage


Gameflow


Scouting


Audibles


Special moves


Play Preference


Running


Play Diversity


expressive
intelligence
studio


UC Santa Cruz

Weka

Toolkit

expressive
intelligence
studio


UC Santa Cruz

Predicting the Number of Games Played

0
50
100
150
200
250
0
50
100
150
200
250
Actual Games Played

Predicted Games Played

Correlation Coefficient: 0.88

expressive
intelligence
studio


UC Santa Cruz

Feature Impact on Number of Games Played


How does tweaking a single feature impact retention?

0
10
20
30
40
50
60
70
0
0.2
0.4
0.6
0.8
1
Predicted Number of Games Played

Value of tweaked Feature

Peer Quit Ratio
Play Diversity
Actions Per Play
Sacks Allowed
Online Franchise Games
expressive
intelligence
studio


UC Santa Cruz

Most Influential Features


The following features were identified as the most influential
in predicting player retention

Feature

Impact

Play Diversity

Negative

Online Franchise Wins

Positive

Running Plays

Positive

Sacks

Made

Positive

Actions

per Play

Positive

Interceptions Caught

Positive

Sacks Allowed

Negative

Peer

Quit Ratio

Negative

Correlation Strength

expressive
intelligence
studio


UC Santa Cruz

Predicted Number of Games for Different Win Rates

0
5
10
15
20
25
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Predicted Number of Games

Win Rate

PlayNow
Ranked
Unranked
OTP
Superstar
Franchise
Online Franchise
Ultimate Team
expressive
intelligence
studio


UC Santa Cruz

What We Learned


Simplify playbooks


Players presented with a large variety of plays have
lower retention and less success



Clearly present the controls


Knowledge of controls had a larger impact than
winning on player retention



Provide the correct challenge


Multiplayer matches should be as even as possible,
while single player should greatly favor the player

expressive
intelligence
studio


UC Santa Cruz

Project Impact


Play selection redesign

expressive
intelligence
studio


UC Santa Cruz

Takeaways



Machine Learning enables
deep analysis

of
Big Data



Machine Learning is
versatile



There are
open tools

expressive
intelligence
studio


UC Santa Cruz

Questions?



Ben Weber


UC Santa Cruz


bweber@soe.ucsc.edu




Michael John


Electronic Arts


mjohn@ea.com