Learning from Implicit Feedback

moonlightmidgeInternet και Εφαρμογές Web

18 Νοε 2013 (πριν από 3 χρόνια και 6 μήνες)

71 εμφανίσεις

Evaluating the Robustness of
Learning from Implicit Feedback

Filip

Radlinski

Thorsten
Joachims



Presentation by

Dinesh Bhirud

bhiru002@d.umn.edu

Introduction


The paper evaluates the robustness of
learning to rank documents based on Implicit
feedback.


What is implicit feedback?


Relevance feedback obtained from search engine
log files


Easier to collect large amount of such training data
as against explicitly collecting relevance feedback.

Osmot


Osmot



Search engine developed at Cornell
University based on Implicit Feedback


Name
Osmot

comes from the word “osmosis”


learning from the users by osmosis


Query Chains


Sequence of reformulated
queries.


Osmot

learns ranked retrieval function by
observing query chains and monitoring user clicks.

High Level Block Diagram

Data
generation

User
behavior
simulation
(based on
original
ranking
fucntion
)

Preference
generation

SVM
Learning

User
behavior
simulatoin

(based on
learned
ranking
function)

Data Generation


Set of W words are chosen, word frequencies
obeying a
Ziph’s

law


T topics are picked by picking N words/topic
uniformly from W.


Each document d is generated as


Pick
k
d

binomially from [0,T]


Repeat
k
d

times


Pick topic t


Pick L/
k
d

words from topic t.


Relevance


3 kinds of relevance


Relevance with respect to topic


Can be measured/known because document collection and
topics are synthetic


Used for evaluating the ranking function.


Relevance with respect to query


Actual relevance score of a document with respect to a
query


Used to rank documents


Observed relevance


Relevance of a document as judged by the user seeing only
the abstract.


Used to simulate user behavior.


User behavior parameters


Noise


Accuracy of user’s relevance estimate


Affects observed relevance. (
obsRel
)


obsRel

is drawn from an incomplete Beta distribution
where
α

gives noise level and
β

is selected so that
mode is at
rel
(
d,q
)


Threshold


User selectivity over results (
rT
)


Patience


Number of results user looks at before
giving up (
rP
)


Reformulation


How likely is the user to
reformulate query(
P
reform
)

User Behavior Model

While question T is unanswered

1.1 Generate query q (Let d1,d2..,dn be results for q)

1.2 Start with document 1
ie

i

= 1

1.3 while patience (
Rp
) > 0


1.3.1 if
obsRel
(
di,q
) >
rT



1.3.1.1 if
obsRel
(di+1, q) >
obsRel
(
di,q
) + c then




continue looking further in the list




1.3.1.2 else




di

is a good document, click on it.




If
rel
(
di,T
) is 1, user is DONE




Decrease patience
Rp
.



1.3.2 else



Decrease patience
Rp



Rp

=
Rp

-

(
rT



obsRel
(
di,q
))


1.3. 3 Set
i

=
i

+ 1

1.4 With probability (1


Preform
)

, user gives up.

User Preference Model


Based on the
clickthrough

log files, users’
preferences for documents given query q can
be found.


Clickthrough

logs generated by simulating
users.


From preference, features values are
calculated.

Feedback Strategies


Single Query Strategy


Click >
q
Skip Above


For query q, if document
d
i

is clicked,
d
i

is
preferred over all
d
j
, j <
i
.



Click 1
st

>
q

No
-
Click 2
nd


For query q, if document 1 is clicked, it is
preferred over the 2
nd

document in the list.








Feedback Strategies


2
-
Query Strategy 1


This strategy uses 2 queries in a query chain, but
document rankings only for the later query.


Given queries q' and q in a query chain


Click >
q'

Skip Above


For query q', if document
di

is clicked in query q,
di

is
preferred over all
dj
, j <
i



Click 1
st

>

q'

No
-
Click 2
nd


For query q', if document 1 is clicked, it is preferred
over the 2
nd

document in the list for q



Feedback Strategies


2
-
Query Strategy 2


This strategy uses 2 queries in a query chain, and
document rankings for both used.


Given queries q' and q in a query chain


Click >
q'

Skip Earlier Query


For query q', if document
di

is clicked in query q,
di

is
preferred over seen documents in query previous
query.


Click >

q'

Top two earlier Query


If no document clicked for query q', then
di

preferred
over top two in previous query.



Example


Q1

Q2

D1

D4

D2

D5

D3

D6

Preferences


D2 >
q1

D1


D4 >
q2

D5


D4 >
q1

D5


D4 >
q1

D1


D4 >
q1

D3

Features


Document
d
i

would be mapped to feature
vector with respect to query q.


2 types of features defined


Rank Features


Term/Document Features








q)

,

(di
Ø
q)

,

(di
Ø


q)

,

Ø(di
term
rank
q)

,

Ø(di
q)

,

(di
Ø
rank
q)

,

(di
Ø
term
Rank Features


Rank features allow representation of ranking
given by the existing static retrieval function.


Used a simple TFIDF weighted cosine similarity
metric (
rel
0
)


28 rank features used for ranks
1,2,..,10,15,20,…100.


Set to 1 if clicked document is at or above
specified rank.

Term Features


Allows representation of fine grained
relationship between query terms and
documents.


If for query
q,
document

d
is clicked, then for
each word ,


Forms a sparse feature vector, as only very few
words are included in query.

q
w

1


w)
,

(d
Ø
term

Learning


Retrieval Function
rel
(
d
i
, q)
defined as






where is the weight vector.




Intuitively, weight vector assigns weight to each
feature identified.


Task of learning a ranking function is reduced to
the task of learning an optimal weight vector.






q)

,

Ø(di

q)

rel(di,


w

w

How does affect ranking?


Points are ordered by
their projections onto


For the ordering will
be 1,2,3,4.


For the ordering will
be 2,3,1,4.


Weight vector needs
to be learnt that will
minimize
number of
discordant rankings.

w

w

1
w

2
w

w

Learning Problem

Learning problem can be formalized as follows


Find weight vector such that


maximum of following inequalities fulfilled.


such that


then


Without using slack variables, this is NP
-
hard
problem.

w

1
)
,
(
r
d
d
j
i


)
,
(
)
,
(
1
1
q
d
r
q
d
r
j
i

q)

,

Ø(d


q)

,

Ø(d
j
i



w
w


SVM Learning


Equivalent optimization problem would be


Minimize


Subject to

rearranging which we get constraint


and

and




ij
C
w
w
ij
2
1



ij

-

1


q)

,

Ø(d
w


q)

,

Ø(d
:
)
,
,
(
j
i








w
j
i
q
ij

-

1

q))

,

Ø(d

-

q)

,

Ø(d
(
:
)
,
,
(
j
i




w
j
i
q

0
:


ij
ij

01
.
0
:
]
28
,
1
[



i
w
i
Re
-
ranking using the learnt model


SVM
-
Light package is used.


Model provides values for all support
vectors.


User behavior is again simulated, this time using
the learnt ranking function.


How does
reranking

work?


First, a ranked list of documents is obtained using the
original ranking function.


This list is re
-
ordered, using the weights of each
feature obtained from the learnt model.

y


Experiments


Experiments done to study the behavior of the
search engine by varying parameters like


Noise in users’ relevance
judgement


Ambiguity of words in topics and queries


Threshold value which user considers good
document


Users’ trust in ranking


Users’ probability of reformulation of query.

Results
-

Noise

70
75
80
85
90
95
100
0
1
2
3
4
5
6
Expected Relevance

Learning Iterations

Ranking Function performance at various noise levels

Noise Low
Noise Medium
Noise High
Maximum Noise
Noise


My experiment


Did implementation for extracting preferences
and encoding them in features.


70
72
74
76
78
80
82
84
86
0
1
2
3
4
5
6
Expected Relevance

Learning Iterations

Ranking function performance at various noise levels

(My implementation)

Noise Low
Noise Medium
Noise High
Topic and Word Ambiguity

70
75
80
85
90
95
100
0
1
2
3
4
5
6
Expected Relevance

Learning Iterations

Ranking function performance at different levels of word ambiguity

No ambiguous words
Words somewhat ambiguouos
Words more ambiguous
Probability of user reformulating query

70
75
80
85
90
95
100
0
1
2
3
4
5
6
Expected Relevance

Learning Iterations

25% give up probability
50% Give up probabilty
75% Give up probability
100% Give up probability
Thank You