Ming Li Talk about Bioinformatics

powerfultennesseeBiotechnology

Oct 2, 2013 (3 years and 8 months ago)

72 views

Lecture 13: Adversary Arguments
continues …


This lecture, we continue with the adversary
argument by giving more examples.


Problem: given a list of n elements, find the
2nd largest.
[Reference: S. Baase, Computer Algorithms:
Introduction to Design & Analysis, Second Edition, Addison
-
Wesley,
1991, pp. 128
-
133.]

Example 4. Largest and Second Largest


Find 2
nd

largest element


Naive algorithm:


find the maximum element (n
-
1 comparisons)


remove it from the list find the maximum
among those remaining (n
-
2 comparisons)


total = 2n
-
3 comparisons

Tournament algorithm


Have players play in pairs, then winners play each other, etc.,
until a winner is determined. Then the "runner
-
up" (2nd largest)
is among those defeated by winner. Why? Because any other
player has at least TWO players superior to him/her.


10 6 3 5 12 4 7 11


\

/
\

/
\

/
\

/


\

/
\

/
\

/
\

/


10 5 12 11


\

/
\

/


\

/
\

/


10 12


\

/


\

/


12

Thus the total is n+logn
-
2

A matching lower bound for 2
nd

largest


We can prove that the algorithm just given is
optimal
,
by an
adversary argument

similar to that we saw
before. The result is due to S. S. Kislitsyn in 1964.


The idea. First, note that any algorithm to find the 2nd
largest element must "know" the largest element.
(For otherwise the 2nd largest might be the largest!)
To determine the largest element requires n
-
1
comparisons. The adversary argument we will give
shows that we can always force ceil(lg n) elements to
be compared with the maximum. This will prove what
we want, since the runner
-
up is the maximum of
these ceil(lg n) elements, requiring ceil(lg n)
-
1
comparisons to decide.


Adversary strategy:


All keys are assigned weights w[i]


Weights are all initialized to 1


Adversary replies are based on weights

When x is compared to y

Weights

Adversary Reply Weight Changes

w[x]>w[y]


x>y


w[x]:=w[x]+w[y]; w[y]:=0;

w[x]=w[y]>0


x>y


w[x]:=w[x]+w[y]; w[y]:=0;

w[y]>w[x]


y>x


w[y]:=w[y]+w[x]; w[x]:=0;

w[x]=w[y]=0


Consistent

None

Accumulation of weights


Solution of the problem requires all weights to
be accumulated to one key


All other keys must have weight zero


Since weight accumulates to the highest
weight, weight can at most double with each
comparison,
lg

n comparisons are required to
accumulate all weights to the highest weight.


And for these
lg

n comparisons, because the
losers have non
-
0 weights, they have not lost
before!

More formal argument


We now show that this adversary strategy forces ceil(lg n)
comparisons of max with elements that have never previously
lost a comparison.


Observe that w(k) ≤ 2 w(k
-
1) by the adversary strategy. It follows
that w(k) ≤ 2
k

w(0). But w(0) =1. Hence w(k) ≤ 2
k
.


We end at w(k) = n. Hence n ≤ 2
k
. Hence k ≥ ceil(lg n).


But k is the number of comparisons of max against elements
that have never previously lost (because they have non
-
zero
weights), so we have shown there must be at least ceil(lg n)
such comparisons. There must be at least n
-
1 comparisons to
find the maximum, and among the ceil(lg n) elements that lost
comparisons to max, we need to do at least ceil(lg n)
-
1
comparisons to find the 2nd largest.


Thus the lower bound is n+ lg n


2, for the worst
-
case.

Example 5. Lower bound for Median


The best upper bound is 3n + o(n)


Best lower bound is 2n + o(n) by Dor and Zwick (focs96).


We prove a 1.5n + O(1) worst
-
case lower bound here.


If x is the median, We must have the following situation:



* *


\

/ elements > median


* *


| /


----------------
x
--------------------
median
--------



/ |
\



* * * elements < median


|


*

Plan:


First, we call a comparison involving x
crucial

if it is the first
comparison where


(a) x < y and y ≤ median OR



(b) x > y and y ≥ median.


So the algorithm must make n
-
1
crucial

comparisons. We need
to show the algorithm also makes (n
-
1)/2 non crucial
comparisons. The idea is as follows: first, the adversary picks
some value (
not

some element) to be the median. Then it
assigns a label in {N, L, S} to each element, and also values, as
appropriate. Initially each element is assigned an "N" (= "Never
participated in any comparison"). "L" means Larger than the
median and "S" means Smaller than the median.

Adversary strategy


The adversary follows the following strategy:


Algorithm Adversary


compares responds


----------------------------------------------------------



(N,N) assign one element to be larger than


the median, one smaller; result is (L,S)


(S,N) or (N,S) assign the N
-
element to be larger than


the median; result is (S,L) or (L,S)


(L,N) or (N,L) assign the N
-
element to be smaller than


the median; result is (L,S) or (S,L)


(S,L) or (L,S) consistent with previously assigned


or (S,S) or (L,L) values

Proof continues …


This strategy continues until (n
-
1)/2 S's (or (n
-
1)/2 L's) have
been assigned. If at some point (n
-
1)/2 S's are assigned, then
the adversary assigns the remaining elements to be greater
than the median, except for one, which IS the median. A similar
thing is done if (n
-
1)/2 L's have been assigned.


The last element assigned is always the median.


This strategy will always force the algorithm to perform (n
-
1)/2
non
-
crucial comparisons. For any time an N
-
element is
compared, a non
-
crucial comparison is done (except at the very
end, when a crucial comparison may be done with the median
itself). The least number of comparisons with N
-
elements that
can be done is (n
-
1)/2. The TOTAL number of comparisons is
therefore n
-
1 + (n
-
1)/2 = (3/2)(n
-
1).


Summary


We have been extremely lucky with the
problems we have discussed.


For most of other problems you will see in
future, lower bounds, even the very weak
ones, are extremely hard to get, if we can get
them at all.


We learned two powerful methods:


Adversary argument


Incompressibility method.