Ming Li Talk about Bioinformatics


Oct 2, 2013 (3 years and 11 months ago)


Lecture 13: Adversary Arguments
continues …

This lecture, we continue with the adversary
argument by giving more examples.

Problem: given a list of n elements, find the
2nd largest.
[Reference: S. Baase, Computer Algorithms:
Introduction to Design & Analysis, Second Edition, Addison
1991, pp. 128

Example 4. Largest and Second Largest

Find 2

largest element

Naive algorithm:

find the maximum element (n
1 comparisons)

remove it from the list find the maximum
among those remaining (n
2 comparisons)

total = 2n
3 comparisons

Tournament algorithm

Have players play in pairs, then winners play each other, etc.,
until a winner is determined. Then the "runner
up" (2nd largest)
is among those defeated by winner. Why? Because any other
player has at least TWO players superior to him/her.

10 6 3 5 12 4 7 11











10 5 12 11







10 12






Thus the total is n+logn

A matching lower bound for 2


We can prove that the algorithm just given is
by an
adversary argument

similar to that we saw
before. The result is due to S. S. Kislitsyn in 1964.

The idea. First, note that any algorithm to find the 2nd
largest element must "know" the largest element.
(For otherwise the 2nd largest might be the largest!)
To determine the largest element requires n
comparisons. The adversary argument we will give
shows that we can always force ceil(lg n) elements to
be compared with the maximum. This will prove what
we want, since the runner
up is the maximum of
these ceil(lg n) elements, requiring ceil(lg n)
comparisons to decide.

Adversary strategy:

All keys are assigned weights w[i]

Weights are all initialized to 1

Adversary replies are based on weights

When x is compared to y


Adversary Reply Weight Changes



w[x]:=w[x]+w[y]; w[y]:=0;



w[x]:=w[x]+w[y]; w[y]:=0;



w[y]:=w[y]+w[x]; w[x]:=0;




Accumulation of weights

Solution of the problem requires all weights to
be accumulated to one key

All other keys must have weight zero

Since weight accumulates to the highest
weight, weight can at most double with each

n comparisons are required to
accumulate all weights to the highest weight.

And for these

n comparisons, because the
losers have non
0 weights, they have not lost

More formal argument

We now show that this adversary strategy forces ceil(lg n)
comparisons of max with elements that have never previously
lost a comparison.

Observe that w(k) ≤ 2 w(k
1) by the adversary strategy. It follows
that w(k) ≤ 2

w(0). But w(0) =1. Hence w(k) ≤ 2

We end at w(k) = n. Hence n ≤ 2
. Hence k ≥ ceil(lg n).

But k is the number of comparisons of max against elements
that have never previously lost (because they have non
weights), so we have shown there must be at least ceil(lg n)
such comparisons. There must be at least n
1 comparisons to
find the maximum, and among the ceil(lg n) elements that lost
comparisons to max, we need to do at least ceil(lg n)
comparisons to find the 2nd largest.

Thus the lower bound is n+ lg n

2, for the worst

Example 5. Lower bound for Median

The best upper bound is 3n + o(n)

Best lower bound is 2n + o(n) by Dor and Zwick (focs96).

We prove a 1.5n + O(1) worst
case lower bound here.

If x is the median, We must have the following situation:

* *


/ elements > median

* *

| /


/ |

* * * elements < median




First, we call a comparison involving x

if it is the first
comparison where

(a) x < y and y ≤ median OR

(b) x > y and y ≥ median.

So the algorithm must make n

comparisons. We need
to show the algorithm also makes (n
1)/2 non crucial
comparisons. The idea is as follows: first, the adversary picks
some value (

some element) to be the median. Then it
assigns a label in {N, L, S} to each element, and also values, as
appropriate. Initially each element is assigned an "N" (= "Never
participated in any comparison"). "L" means Larger than the
median and "S" means Smaller than the median.

Adversary strategy

The adversary follows the following strategy:

Algorithm Adversary

compares responds


(N,N) assign one element to be larger than

the median, one smaller; result is (L,S)

(S,N) or (N,S) assign the N
element to be larger than

the median; result is (S,L) or (L,S)

(L,N) or (N,L) assign the N
element to be smaller than

the median; result is (L,S) or (S,L)

(S,L) or (L,S) consistent with previously assigned

or (S,S) or (L,L) values

Proof continues …

This strategy continues until (n
1)/2 S's (or (n
1)/2 L's) have
been assigned. If at some point (n
1)/2 S's are assigned, then
the adversary assigns the remaining elements to be greater
than the median, except for one, which IS the median. A similar
thing is done if (n
1)/2 L's have been assigned.

The last element assigned is always the median.

This strategy will always force the algorithm to perform (n
crucial comparisons. For any time an N
element is
compared, a non
crucial comparison is done (except at the very
end, when a crucial comparison may be done with the median
itself). The least number of comparisons with N
elements that
can be done is (n
1)/2. The TOTAL number of comparisons is
therefore n
1 + (n
1)/2 = (3/2)(n


We have been extremely lucky with the
problems we have discussed.

For most of other problems you will see in
future, lower bounds, even the very weak
ones, are extremely hard to get, if we can get
them at all.

We learned two powerful methods:

Adversary argument

Incompressibility method.