Crowdsourcing for Business:
An Emerging Paradigm
Shourya Roy
Area Manager, Human Computation
Xerox Research Centre India, Bangalore
shourya.roy@xerox.com
Workshop on Social
Computing, IIT
Kharagpur
5
th
Oct, 2012
Crowdsourcing :
What is it?
The act of taking a task traditionally performed by an
employee or contractor, and outsourcing it to an
undefined, generally large group of people, in the form of
an open call
Digitization, image labeling, user studies, machine translation
evaluation, logo design, EDA simulation, innovation contests, . .
.
Services Thrust
–
Xerox Confidential
–
Handwriting
Recognition Problem
*
Many tasks are easy/feasible/
doable
for
humans
, but difficult/challenging/impossible
for
computer programs
1.
Make progress
towards deciphering this
handwriting
2.
Put words which you
are unsure about in parenthesis
Instructions to Crowd
Services Thrust
Examples (1/5)
Services Thrust
Examples (2/5)
Services Thrust
Examples (3/5)
Services Thrust
Examples (4/5)
It has been Existing
Humans were the first “computers,” computers, used for math
computations
9 examples of crowdsourcing, before ‘crowdsourcing’ existed :
http://bit.ly/mXFdRp
–
Xerox Confidential
–
Internet and Mobile Have Made
it More Common and Promising
Services Thrust
Increasing Activities and Popularity
Page
10
“2M contributors who does more
than 4PY of work on an average
day!”
--
Increasing Popularity as Depicted
by Google Trends
Crowdsourcing on Google Scholar
Over the Last Few Years
CEO
Services Thrust
Changing Demographics
What is the Problem
Given a computational problem, design a solution
using
human
computers
and
automated
computers
–
Xerox Confidential
–
Human in the loop
(
and not
Guinea Pigs
)
Main doer is Human
(
and not
Machines as in Assembly Lines
)
H
umans
are actively
computing
(
not merely carrier
of sensors
)
The
outcome is determined by
an algorithm
(
and not the
natural dynamics of the crowd
)
Why is it Different?
Where is Research?
–
Xerox Confidential
–
Quality Estimation and Assurance
(
Redundancy and voting; Gold data; joint estimation of worker
quality and task difficulty; Symbiosis with Machine Learning
)
Complex Tasks
(
No discrete answer; Exploration and exploitation; crowd workflows
;)
Task Design
(
Optimize cost, quality and time; infinite completion time; Real time
)
Incentive and Motivation
(
Payment vs. non
-
payment; Optimal payment; Payment and quality
; )
Market Design
(
Reputation Mechanism; Monitoring and feedback; Task Discovery
;
Behavioral Aspects
(
Noisy
behaviour
; Non
-
reproducible
;)
An Emerging Research Field
An Interdisciplinary Research Field
That’s Alright
–
but Xerox!!?
We have transformed…
… into the world’s
leading enterprise
for
Business
Process
and
Document
Management
Revenue
Market
Opportunity
2011
Services
-
led
~50% Services
Document Outsourcing
Business Process
Outsourcing
Information Tech
Outsourcing
~$23 billion
$500 billion +
Services
Leadership In
$15.2 billion
2009
Technology
-
led
~25% Services
$132 billion
Document
Outsourcing
18
Xerox Revenue by Business Segment*
* http://
www.fastcompany.com
/magazine/161/
ursula
-
burns
-
xerox
Page
20
Is Crowdsourcing a Viable Alternative
to Outsourcing?
Outsourcing is
Focus
on the core business while
partnering
with 3rd
party vendors to tackle the
non
-
core operations
Tasks requiring human intelligence and skills
Data and process migration by smart use of
technology
Heavily human intensive; typically with the help of
computing technologies
Large distributed workforce enabled by
technology
executing tiny pieces of work
requiring
human intelligence
Page
21
Data Entry by Crowd
We started by
considering a typical
outsourced process
(Data Entry)
Objective is to
understand a process in
detail and identify
implications for
crowdsourcing
Digitisation of
insurance forms and
medical records for
US based insurance
companies
Typing in,
validation/
correction of
information from
scanned forms
Outsourced,
distributed
process
Features that make
Form Digitization
process
amenable to crowd sourcing
Page
22
Relatively low skill data entry work, known
as ‘key what you see’
Already an outsourced process requiring a
low level of interactivity between
sequential steps
Strong workflow tool to manage work, which
flows through a series of system and human
steps
Between sites
Between sequential tasks
Between agents (given their known skill
set
)
Findings from Work
-
Practice Study (1/2)
Findings from Work
-
Practice Study
(2/2
)
Page
24
Workplace Ecology
: Data security is physical, technical
&
social
Crowdsourcing
: lose physical and social enforcement,
reduced control of workforce. Need technical solutions.
Skills and
Knowledge
1)‘key what you see’ data entry
actually involves extensive rule set. 2) Form difficulty
is situational. 3) Non
-
standard means non
-
standard
.
Crowdsourcing
: Situational
-
based incentives and
supporting learning
Being
a Corporate Employee
Pay
alone not enough to
achieve SLA. Agents made accountable.
Crowdsourcing
: reduced accountability could increase
rejections of difficult work.
Making the Workflow
Work:
Push model of
work
Crowdsourcing
: Pull model of work raises coordination and
completion issues.
Collaborative Working
: Work is not collaborative at
workflow level; but it is at claim level (floorwalkers &
colleagues).
Crowdsourcing
: building collaboration in? Pull models of
supervision?
Conclusion
Page
25
Crowdsourcing is an emerging Research
area
It requires expertise and research
competencies from a number
disciplines
Crowdsourcing can be applied in
various domains to solve problems in
a more effective manner
Finally, a large fraction of the
crowd comes from India
Focused research and technologies
will be highly relevant
Services Thrust
References
•
TurKit
: Tools for Iterative Tasks on Mechanical Turk; Greg Little,
Lydia B. Chilton, Robert C. Miller, and Max
Goldman
•
Matt Lease Tutorial
•
Soylent
–
A
cr
•
Fold.it
–
S. Cooper et. al
Enter the password to open this PDF file:
File name:
-
File size:
-
Title:
-
Author:
-
Subject:
-
Keywords:
-
Creation Date:
-
Modification Date:
-
Creator:
-
PDF Producer:
-
PDF Version:
-
Page Count:
-
Preparing document for printing…
0%
Σχόλια 0
Συνδεθείτε για να κοινοποιήσετε σχόλιο