Dermoscopic Interest Point

companyscourgeAI and Robotics

Oct 19, 2013 (4 years and 21 days ago)

67 views

Dermoscopic Interest Point
Detector and Descriptor


Howard Zhou
1
, Mei Chen
2
, James M. Rehg
1


1
School of Interactive Computing, Georgia Tech

2
Intel Research Pittsburgh


2

2009
-
07
-
01

Skin cancer


Skin cancer : most common type of cancer ( > 1 million )



[ Top 5 categories of estimated annual cancer incidence for 2009 from National Cancer Institute ]

3

2009
-
07
-
01

Skin cancer


Skin cancer : most common type of cancer ( > 1 million )


forms in tissues of the skin


[ Image courtesy of “An Atlas of Surface Microscopy of Pigmented Skin Lesions: Dermoscopy” ]

Skin lesions

4

2009
-
07
-
01

Skin cancer


Skin cancer : most common type of cancer ( > 1 million )


forms in tissues of the skin

[ Image courtesy of “An Atlas of Surface Microscopy of Pigmented Skin Lesions:
Dermoscopy
” ]

Benign lesions

Skin cancer

5

2009
-
07
-
01

Skin cancer


Skin cancer : most common type of cancer ( > 1 million )


forms in tissues of the skin

[ Image courtesy of “An Atlas of Surface Microscopy of Pigmented Skin Lesions: Dermoscopy” ]

Benign lesions

Skin cancer

Squamous cell

carcinoma

Basal cell

carcinoma

Melanoma

6

2009
-
07
-
01

Dermoscopy


Non
-
invasive imaging technique


I
mprove diagnostic accuracy by 30%


[ Image courtesy of “An Atlas of Surface Microscopy of Pigmented Skin Lesions: Dermoscopy” ]

Skin cancer

Squamous cell

carcinoma

Basal cell

carcinoma

Melanoma

7

2009
-
07
-
01

Dermoscopy

[ Image courtesy of “An Atlas of Surface Microscopy of Pigmented Skin Lesions: Dermoscopy” ]

Clinical view


Non
-
invasive imaging technique


I
mprove diagnostic accuracy by 30%


8

2009
-
07
-
01

Dermoscopy

[ Image courtesy of “An Atlas of Surface Microscopy of Pigmented Skin Lesions: Dermoscopy” ]

D
ermatoscope


Non
-
invasive imaging technique


I
mprove diagnostic accuracy by 30%


Microscope + light + liquid medium



9

2009
-
07
-
01

Dermoscopy


Non
-
invasive imaging technique


I
mprove diagnostic accuracy by 30%


Microscope + light + liquid medium


Reveal pigmented structures


[ Image courtesy of “An Atlas of Surface Microscopy of Pigmented Skin Lesions: Dermoscopy” ]

D
ermatoscope

Dermoscopy view

10

2009
-
07
-
01

Dermoscopic features


Pigmented structures revealed by
dermoscopy

[ Image courtesy of “An Atlas of Surface Microscopy of Pigmented Skin Lesions: Dermoscopy” ]

Dermoscopy view

11

2009
-
07
-
01

Dermoscopic features


Pigmented structures revealed by
dermoscopy

[ Image courtesy of “An Atlas of Surface Microscopy of Pigmented Skin Lesions: Dermoscopy” ]

Dermoscopy view

Blue
-
white veil

12

2009
-
07
-
01

Dermoscopic features


Pigmented structures revealed by
dermoscopy

[ Image courtesy of “An Atlas of Surface Microscopy of Pigmented Skin Lesions: Dermoscopy” ]

Dermoscopy view

Blue
-
white veil

Scar
-
like
depigmentation

13

2009
-
07
-
01

Dermoscopic features


Pigmented structures revealed by
dermoscopy

[ Image courtesy of “An Atlas of Surface Microscopy of Pigmented Skin Lesions: Dermoscopy” ]

Dermoscopy view

Blue
-
white veil

Scar
-
like
depigmentation

Brown globules

14

2009
-
07
-
01

Dermoscopic features


Pigmented structures revealed by
dermoscopy



[ Image courtesy of “An Atlas of Surface Microscopy of Pigmented Skin Lesions: Dermoscopy” ]

Dermoscopy view

Blue
-
white veil

Scar
-
like
depigmentation

Brown globules

Negative network

15

2009
-
07
-
01

Dermoscopic features


Pigmented structures revealed by
dermoscopy


[
Betta

et al. 2006], [
Grana

et al. 2006], [
Iyatomi

et al. 2007],…

[ Image courtesy of “An Atlas of Surface Microscopy of Pigmented Skin Lesions: Dermoscopy” ]

Dermoscopy view

Blue
-
white veil

Scar
-
like
depigmentation

Brown globules

Negative network

16

2009
-
07
-
01

Dermoscopic features


Over 100
dermoscopic

features

[ Image courtesy of “An Atlas of Surface Microscopy of Pigmented Skin Lesions: Dermoscopy” ]

Dermoscopy view

Blue
-
white veil

Scar
-
like
depigmentation

Brown globules

Negative network



17

2009
-
07
-
01

Dermoscopic features

[ Image courtesy of “An Atlas of Surface Microscopy of Pigmented Skin Lesions: Dermoscopy” ]

Dermoscopy view

Blue
-
white veil

Scar
-
like
depigmentation

Brown globules

Negative network



BW classifier

SLD classifier

BG classifier

NN classifier




Over 100
dermoscopic

features


Multiple binary classifiers for each image

18

2009
-
07
-
01

Dermoscopic features

[ Image courtesy of “An Atlas of Surface Microscopy of Pigmented Skin Lesions: Dermoscopy” ]

Dermoscopy view

Blue
-
white veil

Scar
-
like
depigmentation

Brown globules

Negative network




General detector?

19

2009
-
07
-
01

Dermoscopic features

[ Image courtesy of “An Atlas of Surface Microscopy of Pigmented Skin Lesions: Dermoscopy” ]

Dermoscopy view

Blue
-
white veil

Scar
-
like
depigmentation

Brown globules

Negative network




General

detector?


Dermoscopic

features consist of low level image characteristics
(ridges, blobs, streaks,
pigmentation,…)

20

2009
-
07
-
01

Dermoscopic features

[ Image courtesy of “An Atlas of Surface Microscopy of Pigmented Skin Lesions: Dermoscopy” ]

Dermoscopy view

Blue
-
white veil

Scar
-
like
depigmentation

Brown globules

Negative network




General

detector?


Dermoscopic

features consist of low level image characteristics
(ridges, blobs, streaks, pigmentation,…)




interest points

21

2009
-
07
-
01

Dermoscopic

Interest Point (DIP)

[ Image courtesy of “An Atlas of Surface Microscopy of Pigmented Skin Lesions:
Dermoscopy
” ]

Dermoscopy view

Blue
-
white veil

Scar
-
like
depigmentation

Brown globules

Negative network




General

detector
: concentration/configuration of interest points


bag
-
of
-
visual
-
words approach


Dermoscopic

features consist of low level image characteristics
(ridges, blobs, streaks,
pigmentation,…)




interest points

22

2009
-
07
-
01

Dermoscopic

Interest Point (DIP)


Inspired by general interest point detector and descriptors
(SIFT & SURF)


We propose
Dermoscopic

Interest Point (DIP)


detector
-

to extract these low level building blocks


descriptor


for constructing a general visual vocabulary for
dermoscopic

features

23

2009
-
07
-
01

Dermoscopic

Interest Point (DIP)


Compared to the general interest point detector and
descriptors
(SIFT & SURF)


Same key
issues


Repeatable


Distinctive


Robust to
noise

and deformation (geometric and photometric)


Similar to SIFT & SURF


Corners and blobs


Scale and rotation invariant


In addition


Curvilinear
features (
fibrillar

pattern and radial streaming)


Color component


24

2009
-
07
-
01

Detector


Corners and blobs


Fast
-
Hessian
detector [Bay, et al. 2006]

Hessian matrix

25

2009
-
07
-
01

Detector


Corners and blobs


Fast
-
Hessian
detector [Bay, et al. 2006
]







Box filter approximation to replace Gaussian derivatives


Fast using Integral image

Hessian matrix

26

2009
-
07
-
01

Detector


Corners and blobs


Fast
-
Hessian
detector [Bay, et al. 2006
]







Curvilinear structures


Curvilinear detector [Steger, 1996]

Hessian matrix

27

2009
-
07
-
01

Detector


Corners and blobs


Fast
-
Hessian
detector [Bay, et al. 2006
]







Curvilinear structures


Curvilinear detector [Steger, 1996]

Hessian matrix

28

2009
-
07
-
01

Detector


Corners and blobs


Fast
-
Hessian
detector [Bay, et al. 2006
]







Curvilinear structures


Curvilinear detector [Steger, 1996]

Hessian matrix

29

2009
-
07
-
01

Detector


Corners and blobs


Fast
-
Hessian
detector [Bay, et al. 2006
]







Curvilinear structures


Curvilinear detector [Steger, 1996]

Hessian matrix

30

2009
-
07
-
01

Descriptor


Distinctiveness


Spatially localized information


Distribution of gradient
-
related features


Dermscopic
: color features


Invariance (Repeatability)


Relative strength to reduce the effect of photometric changes


Relative orientation for rotation invariance

31

2009
-
07
-
01

Descriptor


Distinctiveness


Spatially localized information


Distribution of gradient
-
related features


Dermscopic
: color features


Invariance (Repeatability)


Relative strength to reduce the effect of photometric changes


Relative orientation for rotation invariance


To construct


Reproducible orientation

32

2009
-
07
-
01

Descriptor


Distinctiveness


Spatially localized information


Distribution of gradient
-
related features


Dermscopic
: color features


Invariance (Repeatability)


Relative strength to reduce the effect of photometric changes


Relative orientation for rotation invariance


To construct


Reproducible orientation

33

2009
-
07
-
01

Descriptor


Distinctiveness


Spatially localized information


Distribution of gradient
-
related features


Dermscopic
: color features


Invariance (Repeatability)


Relative strength to reduce the effect of photometric changes


Relative orientation for rotation invariance


To construct


Reproducible orientation


Feature vector


34

2009
-
07
-
01

Descriptor


Orientation


For
rotation invariance



Haar
-
wavelet
responses in x and y
direction
(in a circular neighborhood)


35

2009
-
07
-
01

Descriptor


Orientation


For
rotation invariance



Haar
-
wavelet
responses in x and y
direction
(in a circular neighborhood)



Reponses represented as 2D vectors


dy

dx

36

2009
-
07
-
01

Descriptor


Orientation


For
rotation invariance



Haar
-
wavelet
responses in x and y
direction
(in a circular neighborhood)



Reponses represented as 2D vectors



Average responses in a sliding window
of 60 degree


dy

dx

37

2009
-
07
-
01

Descriptor


Orientation


For
rotation invariance



Haar
-
wavelet
responses in x and y
direction
(in a circular neighborhood)



Reponses represented as 2D vectors



Average responses in a sliding window
of 60 degree



The longest vector indicates the
orientation

dy

dx

38

2009
-
07
-
01

Descriptor


Descriptor components


Context of the descriptor: a square region oriented along the
orientation (centered around the interest point
)


Local statistics


Uniform 4 x 4
subregions



Intensity gradients (I): Sum of
Haar
-
wavelet
responses: dx, dy, |dx|,
|
dy|


Color
statistics (
C): Coarse
color histogram of
the region (alpha &
beta channels in L*a*b space)



[ Image courtesy of

Bay et al. 2006]

39

2009
-
07
-
01

Dermoscopy

Interest Point

40

2009
-
07
-
01

Dermoscopy specific


Common interest point descriptor
ignores
linear features

SURF

DIP

41

2009
-
07
-
01

Experiment

42

2009
-
07
-
01

Conclusion


A generalized framework for characterizing
dermoscopic

features using
Dermoscopic

Interest Point (DIP)


A
feature
detector and
a descriptor
specifically designed
for
this purpose


Initial experiments showed that our scheme achieves a
comparable level of invariance to lighting, scale, and rotation
changes

43

2009
-
07
-
01

Future work


Build a vocabulary of
dermoscopic

features using DIP


Explore the possibility of using DIP in skin CAD related
applications:


Dermoscopic

feature
extraction and classification


Dermoscopy

image registration


Dermoscopy

image search and retrieval via
dermoscopic

features



44

2009
-
07
-
01

Acknowledgement


Collaborators
(in alphabetical order)


Dr. Laura K. Ferris
M.D. Ph.D.
UPMC


Richard
Gass
,
Intel Research Pittsburgh


Casey
Helfrich
, Intel Research Pittsburgh


Many thanks to
our anonymous
reviewers for their
helpful comments and suggestion

45

2009
-
07
-
01

Thank you

Thank you
!

46

2009
-
07
-
01

Related publications


Interest pointer detector and descriptors


Distinctive image features from scale
-
invariant
keypoints


David G. Lowe

Intl. J. of Computer Vision (IJCV),

2004


Surf: Speeded up robust features

Herbert
Bay,
Tinne

tuytelaars
, and Luc Van
Gool
,

in
Eur. Conf. on Computer Vision (ECCV),

2006


An unbiased detector of curvilinear structures

Carsten

Steger
,

IEEE Trans. Pattern Anal. Machine
Intell
.(PAMI)

1996

47

2009
-
07
-
01

Outline


Introduction


Detector


Corners and blobs


Curvilinear structures


Descriptor


Orientation


Descriptor components


Validation


Conclusion

48

2009
-
07
-
01

Dermoscopic features


A Pigmented Skin Lesion (PSL) typically has several
dermoscopic

features


Over
100 of
these features

49

2009
-
07
-
01

Detecting line points

[ Steger 1998, ”An Unbiased Detector of Curvilinear Structures” ]

n(
x
)

Cross section

n(
x
)

L(
x
)

L

= 0

L’’
large


Curve

50

2009
-
07
-
01

Experiment