Hyperplane

brasscoffeeAI and Robotics

Nov 17, 2013 (3 years and 7 months ago)

69 views

Chapter 16

A Perceptually Tuned Watermarking

Scheme for Digital Image Using Support

Vector Machines

-

Chin
-
Chen Chang and Iuon
-
Chang Lin
-


Jae
-
Heon JUNG

2004. 9.


http://mips.changwon.ac.kr

1

Outline

1. Introduction

2. Related Works

3. Support Vector Machines

4. The Watermarking Scheme Using Support Vector
Machines

5. Experimental Results

6. Discussions

7. Conclusions

http://mips.changwon.ac.kr

2

1. Introduction


SVM for the copyright protection of digital images


Widely Known about WM


The watermark can be embedded as either visible or
invisible data


Visible : the owner can be easily identified, but watermark is
usually not robust against image processing.


Invisible : the embedding locations and the modified values
are secret. Not only without secret key, no one gets to know
where the watermark is embedding but also resistant to the
most common signal processing.


http://mips.changwon.ac.kr

3


Digital watermarks possible to embed two domains


Spatial domain (ex: SVM)


Frequency domain



Digital watermarks must satisfy some requirements
(Barni et al. 1998)


Unobtrusiveness


Readily extraction


Inclusion of no original image


Robustness


http://mips.changwon.ac.kr

4


Classification tools: Support Vector Machines (SVM)


Based on statistical learning theory (Vapnik 1998)


Good generalization ability


Handwritten digital recognition


Face detection


Particle identification


Text categorization


ETC…


http://mips.changwon.ac.kr

5


The objective of SVM scheme


WM image visually indistinguishable from the original
and to enhance the security


Fully satisfy the requirements of unobtrusiveness


Readily extraction


No need original image


Robustness




http://mips.changwon.ac.kr

6

2. Related Works


Review some watermarking schemes


Replace the last significant bits (LSB) with the
watermark bits


Watermarking schemes on domain


A binary copyright image is transformed into a noise
-
like
image


WM embedded by modifying multiple bits in the blue channel
(less sensitive in the color domain) of a color image.


A block
-
oriented, modular
-
arithmetic
-
base WM scheme (Lin
2000, Lin 2001)

http://mips.changwon.ac.kr

7



Watermarking schemes on frequency domain


(FFT, DCT or Wavelet )



Used a
back
-
propagation neural network

(
BPN
)
(Hwang et al. 2000)


Minimize the empirical risk




Recently, a new technique [SVM]


Minimize the structural risk


To minimize the generalization error as to the unknown test
data


Produce a high quality WM and enhance the security

http://mips.changwon.ac.kr

8

3. Support Vector Machines


SVM (Cortes and Vapnik 1995)


based on Statistical Learning Theory


Maps the input vector into a high dimensional feature
space, optimal separating hyperplane is constructed
through some decision functions.



http://mips.changwon.ac.kr

9

3.1 Liner SVM for Two
-
Class sepatable Data

Figure 1. SVM optimal separating hyperplane


Margin

d1

d2

d2

d2

d1

d1

Class1

Class2

Optimal separating hyperplane

MAX margin d1 + d2

w
Support vector

Support vector

http://mips.changwon.ac.kr

10

hyperplane

optimal

hyperplane

Max
-

margin

hyperplane

Which hyperplane is the best?

information 1.

http://mips.changwon.ac.kr

11





-

Hyperplane :










distance between (0,0) and hyperplane :


-

2 Hyperplanes :





margin between H1 and H2 :






-

linear SVM






Subject to






0
'
:



b
x
w
y
H
Orthogonal to H

w
b
/
|
|
1
'
:
1



b
x
w
y
H
1
'
:
2




b
x
w
y
H
w
/
2
w
w
'
2
max
1
1
'



i
i
y
for
b
x
w
1
1
'





i
i
y
for
b
x
w
1
)
'
(


b
x
w
y
i
i
H1

H2

H

)
2
'
min
or
(
w
w
information 2.

http://mips.changwon.ac.kr

12


Training vector :


Belonging different classes :


Optimization problem :



with respect to


k
i
R
x
n
i
,...,
2
,
1
,


}
1
,
1
{


i
y

,
,
min
b
w
.
,...,
2
,
1
,
0
,
1
)
)
(
(
k
i
and
b
x
w
y
i
i
i
t
i









)
(
i
x

: Transformation function that is used to map

Into a high dimensional feature space.

i
x



k
i
i
t
C
w
w
1
2
1

0

C
: upper bound that is determined by the
tradeoff between the smoothness of the
decision function , generalization error

slack variable

If not linearly,
C = penalty

C
i



0
Max margin between two classes

Wolf duality theory

http://mips.changwon.ac.kr

13


Wolf duality theory :


Optimization problem



With respect to

i

min
,
2
1
i
t
i
t
i
e
Q




,
,...,
2
,
1
,
0
k
i
C
i




0

i
t
i
y

and

Q is a by positive matrix,

k
k
)
(
j
i
j
i
ij
x
x
K
y
y
Q

)
(
)
(
)
(
j
t
i
j
i
x
x
x
x
K



Is the kernel

)
)
,
(
(
)
(
0
b
x
x
K
y
sign
x
f
k
i
i
i
i






Optimal separating function is

http://mips.changwon.ac.kr

14

3.2 Liner SVM for Multi
-
Class sepatable Data


Constructs classifiers :

2
/
)
1
(

g
g

Each classifiers trains data from two different classes


(
ith

and
jth
)


Classification problem



with respect to



ij
ij
ij
b
w

,
,
min


t
t
ij
ij
t
ij
C
w
w
)
(
)
(
(
2
1
)

.
0
,
1
)
)
(
)
((
,
1
)
)
(
)
((
)
)








ij
t
ij
t
ij
t
t
ij
ij
t
ij
t
t
ij
b
x
w
b
x
w





If is in the
ith

class,

If is in the
jth

class, and


t
x
t
x
,
1
)
(
)
(
)
(
)



ij
t
t
ij
b
x
w
sign
x
f

If the decision function



is in the
ith

class, otherwise, is in the
jth

class.


x
x
http://mips.changwon.ac.kr

15

3.3 Non
-
Liner SVM

p
j
i
j
i
x
x
x
x
k
)
1
(
)
,
(



non
-
liner kernel function


Polynomial


Radial basis function

2
/
||
||
2
)
,
(

j
i
x
x
j
i
e
x
x
k


http://mips.changwon.ac.kr

16


The procedure of the proposed scheme


1. the location decision phase


2. the watermark embedding phase


3. the watermark extracting phse

4. The watermarking Scheme SVM

http://mips.changwon.ac.kr

17


Watermark image w, size (a * b)


2D array



w = w(0,0)

w(0,1)



w(0,b
-
1)



w(1,0)

w(1,1)



w(1,b
-
1)




|


|




|


w(a
-
1,0)

w(a
-
1,1)



w(a
-
1,b
-
1)




and
a
i
j
i
w
,
0
},
1
,
0
{
)
,
(



.
0
b
j



Original image O, 8bit per pixel


2D array



O = o(0,0)

o(0,1)



o(0,w
-
1)



o(1,0)

o(1,1)



o(1,w
-
1)




|


|




|


o(h
-
1,0)

o(h
-
1,1)



o(h
-
1,w
-
1)

h: height, w: width o(
I , j
) is from 0 to 255,

h
i


0
w
j


0
http://mips.changwon.ac.kr

18

4.1 Location Decision phase


For security purpose


A secure location decision algorothem


First
-

divide the original image O 3x3 non overlapping blocks


B is the block set

},
1
3
,...,
1
,
0
,
1
3
,...,
1
,
0
|
{
,





w
j
h
i
b
B
j
i

j
i
b
,


O(i*3, j*3) O(i*3, j*3+1) O(i*3, j*3+2)



O(i*3+1, j*3) O(i*3+1, j*3+1) O(i*3+1, j*3+2)



O(i*3+2, j*3) O(i*3, j*3+1) O(i*3, j*3+2)


The block is selected to embed the watermark in
performing the location decision procedure.

j
i
b
,
http://mips.changwon.ac.kr

19


Public key cryptosystem (Rabin 1978)


Owner choose large prime s and t, n = s* t


two secret keys


and are chosen to decide the coordinate




for block .

1
k
2
k
)
(
,
r
r
y
x
r
r
y
x
b
,

The initial location

)
(
0
,
0
y
x
,
mod
2
1
0
n
K
X

,
mod
2
2
0
n
K
Y

and
w
X
x
,
3
mod
0
0

.
3
mod
0
0
h
Y
y


The other a * b
-
1 locations…

,
mod
2
1
n
X
X
r
r


,
mod
2
1
n
Y
Y
r
r


and
w
X
x
r
r
,
3
mod

.
3
mod
h
Y
y
r
r

Where r = 1,2,…, a*b.

WM size

http://mips.changwon.ac.kr

20

4.2 Watermark Embedding phase

http://mips.changwon.ac.kr

21


After finding the a * b location blocks


Mark sequences W=


where

)
...
...
(
1
,
,
1
,
0


b
a
k
W
W
W
W
.
0
,
0
),
(
),
,
(
b
j
and
a
i
b
i
j
k
j
i
w
W
k









Support the training set , contains
m

data points.



is
ith

input pattern (if 3 x 3 block is a 9
-
dimensional data)
which is the 4 leftmost most significant bits (MSB) for each pixel
in a texture block.



is corresponding target value (assigned by the owner, must be
selected carefully by means of the HVS)

}
,
{
i
i
p
d
i
d
i
p
It decides the range of each pixel in the block to be modified, So is very important

i
p
http://mips.changwon.ac.kr

22


For each selected block , determine a flexible output
value


Intensity of nine pixels in the block

r
r
y
x
b
,
r
p
r
r
y
x
b
,
r








2
0
,
))
3
,
3
(
(
9
1
j
i
r
r
r
j
y
i
x
O


Average variation to be made in ,

r
C
r
r
y
x
b
,
r
r
r
p
2
mod




r
C
,
2
r
r
p


,
2
5
r
r
p


),
2
(
r
r
p



,
2
3
r
r
p


If

If

If

If


,
2
3
0
0
r
r
r
p
and
W




,
2
2
3
0
r
r
r
r
p
p
and
W




,
2
0
1
r
r
r
p
and
W




.
2
2
1
r
r
r
r
p
p
and
W




http://mips.changwon.ac.kr

23


Mark sequence is embedded by modifying each
pixel in block .


Finally, marked block becomes

r
W
r
r
y
x
b
,
r
r
y
x
b
,
'

r
r
y
x
b
,
'
r
y
x
C
O
r
r

)
3
,
3
(
r
y
x
C
O
r
r


)
1
3
,
3
(
r
y
x
C
O
r
r


)
2
3
,
3
(
r
y
x
C
O
r
r


)
3
,
1
3
(
r
y
x
C
O
r
r



)
1
3
,
1
3
(
r
y
x
C
O
r
r



)
2
3
,
1
3
(
r
y
x
C
O
r
r


)
3
,
2
3
(
r
y
x
C
O
r
r



)
1
3
,
2
3
(
r
y
x
C
O
r
r



)
2
3
,
2
3
(
http://mips.changwon.ac.kr

24

4.3 Watermark Extracting phase

http://mips.changwon.ac.kr

25


Secret keys and the trained SVM are provided by owner
(
easily found marked block
)



Make the 4 leftmost MSB for each pixel in the designated
block as the input pattern.


The output value easily obtained through SVM
computing


Mean intensity of the 9 pixels in the marked block
can be computed as




Finally, mark sequence can be extracted



Using the location decision procedure

r
p
'
r
'

r
r
y
x
b
,
'
'
))
3
,
3
(
(
9
1
'
2
0
,







j
i
r
r
r
j
y
i
x
O

r
r
r
p
2
mod
'
'




r
w
'
0, If

1, If

,
'
'
0
r
r
p



r
r
r
p
p
'
2
'
'



http://mips.changwon.ac.kr

26

5. Experimental Results


LIBSVM ( library of support vector machines)


50 patterns in the training set, 9 input and 1target value per pattern

3 selected kernels

http://mips.changwon.ac.kr

27

(a) original “Lena” (c) watermarked “Lena”

(b) WM logo (d) extracted WM from (c)

http://mips.changwon.ac.kr

28

(a) Blurred image with WM (c) sharpened image with WM

(b) Extracted WM from (a) (d) extracted WM from (c)

http://mips.changwon.ac.kr

29

(a) Reconstructed from


JPEG compressed

(b) Extracted WM from (a)

(c) Resized image from


shrunken with WM

(d) Extracted WM from (c)

Compression ratio 12:1

Shrunken 512x512 to 300x300

Recovered 300x300 to 512x512


http://mips.changwon.ac.kr

30

http://mips.changwon.ac.kr

31

6. Discussions


Location decision procedure


Rabin’s public key cryptosystem (Rabin 1978)


If leaks out , Infeasible to derive the secret keys


)
(
,
r
r
y
x
1
k
2
k

SVM no need normalization step



( BPN need normalization step )


SVM required memory less then BPN


SVM required training time less then BPN

http://mips.changwon.ac.kr

32

7. Conclusions


A perceptually tuned watermarking scheme

to enhance the
security and robustness


How? Modifying the intensity of each pixel in the texture blocks.


By trained SVM


WM quality is high


No need the original image for Extracting procedure


Robust against burring, sharpening, lossy compression,
and scaling attacks.


Most important thing is SVM to enhance the security and
robustness.