9 Image Processing on Diagnostic Workstations

ruralrompΛογισμικό & κατασκευή λογ/κού

2 Δεκ 2013 (πριν από 3 χρόνια και 8 μήνες)

90 εμφανίσεις

9

Image Processing on Diagnostic Workstations

B
ART
M.

TER
H
AAR
R
OMENY

Professor, Eindhoven University of Technology,
Department of Biomedical Engineering, Image
Analysis and Interpretation, PO

Box

513, WH 2.106,
5600

MB Eindhoven, The Netherlands


CONTENTS

9.1

Introduction

9.2

Hardware

9.3

Software

9.4

3D Visualizat
ion

9.5

Computer Aided Detection (CAD)

9.6

Atlases

9.7

CAD/CAM Design

9.8

Diffusion Tensor Imaging (DTI)
-

Tractography

9.9

Registration

9.10

RT Dose Planning

9.11

Quantitative Image Analysis

9.12

Workstations for Life Sciences

9.13

Computer
-
Aided Surgery (CAS)

9.14

New Developments

9.15

Outlook

References



Scientific terms marked with


are explained in
Wikipedia:
www.wikipedia.org

9.1

Introduction

Medical workstations have developed into the super
-
assistants of radiologists. The overwhelming produ
c-
tion of images, hardware that rapidly became chea
p-
er and powerful
3D visualization and quantit
a
tive
analysis software have all pushed the develo
p
ments
from simple PACS viewing into a really ve
r
satile
viewing environment. This chapter gives an ove
r-
view of these developments, aimed at rad
i
ologists’
readership. Many refe
r
en
ces and internet links


are given which discuss the topics in more depth
than is possible in this short paper. This paper is
necessarily incomplete.

Viewing stations are core business in a r
a-
diologist’s daily work. All big medical imaging
industries supply

professional and integrated env
i-
ronments (such as Philips ViewForum, Siemens
Syngo X, GE Advantage, etc.). There are ded
i
ca
t-
ed companies for viewing software (a.o. Merge
eFilm) or OEM solutions (a.o. Mercury Visage,
Barco). The application domain of wor
k
s
tations is
increasing. We now see them regularly employed
in PACS and teleradiology diagnostic review,
3D/3D
-
time (4D) visualization, co
m
puter
-
aided
detection (CAD), quantitative image analysis,
computer
-
assisted surgery (CAS), radi
o
therapy
treatment plann
ing, and pathology. Also the appl
i-
cations for medical image analysis in the life
-
sciences research are increasing, due to the i
n-
creased attention to small
-
animal scanning sy
s-
tems for molecular i
m
aging

, and the many types
of advanced microscopes (such as c
onfocal m
i-
croscopy


and two
-
photon laser scanning micr
o-
scopes

), all giving huge 3D datasets. The focus
of this chapter is on image processing (also termed
image analysis or computer vision) appl
i
cations.

9.2

Hardware

Early systems were based on expensive
hardware
platforms, called workstations, often based on the
UNIX


operating system

. Today, most systems are
based on readily available and affordable PC and
Mac hardware platforms (running MS
-
Windows or
Mac
-
OS respectively), which are still following
Moor
e’s law


of increasing perfor
m
ance (a doubling
every 24

months) at a st
a
ble price level.

The central processor unit (CPU

) is the core
of the system, running today at several G
i
gahertz,
and performance is expressed in Giga
-
FLOPS


(10
9

floating point operat
ions


per se
c
ond). Famous
CPUs are the Intel Pentium chip, and the AMD At
h-
lon processor. Today, we see the current 32

bit pr
o-
c
essors being replaced by 64

bit processors, which
are capable of processing more instructions simult
a-
n
e
ously and a
d
dressing a larg
er number of memory
elements (2
32

=

4.2
×

10
9
, so a 32

bit system cannot
have more than 4.2

GB of memory(10
9

=

Giga)).
There is also a trend to have more CPUs (‘dualcore’)
on the mothe
r
board

, paving the way to parallel
processing, which is currently still in its infancy.

The memory in th
e diagnostic workstation is
organized in a hierarchical fashion. From small to
large: the CPU has a so
-
called cache


on its chip, as
a local memory scratchpad for super
-
fast access, and
communicates with the main RAM (random access
memory

, today typically

1

4

GB) through the data
bus

, a

data highway in the computer. As the RAM
is fully electronic, access is fast (nanoseconds),
much faster than access to a local hard disk


(mill
i-
seconds). When the RAM is fully occupied, the CPU
starts communicating with th
e hard disk. This e
x-
plains why increasing the RAM of a slow co
m
pu
t-
er can mar
k
edly upgrade its performance. In a
PACS system, the disk storage is typically done
on a ‘r
e
dundant array of inexpensive disks’
(RAID

), where many disks in parallel prevent
loss o
f data in case of failure of a co
m
ponent.




Fig.

9.1a

c.

Brain aneurysm (
a
) and carotids (
b
):
examples of volume renderings

with a computer
game graphics card (3Mensio Inc) (
c
)

The speed of the network should be able to
accommodate the network traffic. Typically the
wor
k
station is part of a local area network (LAN

).
Today gigabit/second speeds are attained over wired
ne
t
work
s, wireless is slower (30

100

Mbit/s) but
convenient for laptops and ‘person digital assistants’
(PDAs). Many PACS install
a
tions can be serviced
remotely through LAN conne
c
tions to the supplier,
anywhere.

Networks are so fast nowadays that 3D vo
l-
ume render
ing can be distributed from a ce
n
tral
powerful computer to simple (and thus low cost)
viewing stations, called ‘thin clients’


(a.o. Te
r-
ar
e
con Aquarius). A

powerful dedicated graphics
board (in this case the VolumePro 1000) with ded
i-
cated hardware runs sev
eral 3D viewing applications
s
i
multaneously, and is remotely controlled by the
u
s
ers of the thin clients. Advantage is the capability
to handle huge datasets (e.g. >

3000 slices) easily,
but scalability (to e.g. do
z
ens of users) is limited.

Interestingly, the power of ‘graphical pr
o-
c
essing units’


(GPU

, the processor on the video
card


(or graphics accelerator card

) in the sy
s
tem)
has increased even faster than CPU power, mainly
due to the fact that GPUs form the core of the co
m-
puter game industry. The millions of systems needed
for this lucr
a
tive market and the high competition
betwe
en the market leaders NVIDIA and ATI have
created a huge performance/price ratio. A

GPU has a
50 times faster communication speed of the data
internally b
e
tween memory and processor, and has
dedicated hardware for rendering artificial enviro
n-
ments, such as

texture mapping

, pixel shaders


and
an intrinsic parallel design with pixel pip
e
lines

.
They have finally become fully programmable
(and can be i
n
structed by languages as DirectX


and OpenGL

) and are equipped with 1

1.5

gigabytes of local memory. These
‘games’
hardware boards are now increasingly used in 3D
medical visualization applications (a.o. 3Mensio
Medical Systems). There is also a community e
x-
ploring the use of GPUs for general processing
(D
I
COM undated a).

The viewing screens of diagnostic wor
k-
s
tations have to be of special diagnostic quality.
Excellent reviews of the important parameters
(resolution, contrast, brightness, 8, 10 or 12 bit
intensity range, hom
o
geneity, stability, viewing
angle, speed, etc. are available in the so
-
called
white pape
rs by a variety of ve
n
dors (a.o. Barco


B
ARCO
undated, Eizo


E
IZO
undated).

9.3

Software

The revolution in PACS (and teleradiology) vie
w-
ing stations was fired by the standard “Dig
i
tal
Imaging and Communications in Medicine”



(DICOM) standard (DICOM unda
ted a), 4000
pages). In the 1990s the ACR (American College
of Radiology) and NEMA (National Electrical
Manufacturers Association) formed a joint co
m-
mittee to develop this standard. The standard is
developed in liaison with other sta
n
dardization
organizati
ons including CEN TC251 in Europe
and JIRA in Japan, with review also by other o
r-
ganizations including IEEE, HL7 and ANSI in the
USA. It is now widely accepted. Convenient short
tutorials are available (
B
ARCO
undated). As the
scanners and viewing software
continue to d
e
ve
l-
op, new features have to be added to the sta
n
dard
continuously. Vendors are required to make avai
l
a-
ble their so
-
called conformity statements (see for
exa
m
ple
B
URRONI
et al. 2004), i.e. a specified list of
what conforms to the current versi
on of the sta
n
dard.

The second revolution was the standardiz
a-
tion of the internal procedural organization of med
i-
cal data handling in the ‘Health Level 7’


sta
n
dard
(HL7) (DICOM undated a).

The basic function of a viewing station is the
convenient viewing
of the data, with a patient sele
c-
tion section. The functions are grouped in a so
-
called
‘graphical user interface’


(GUI

). Versatile PC
based viewing packages are now widely available
(see RSNA 2006 for an extensive list), many also
offe
r
ing ‘extended ASC
I’


character sets for the
Chinese, Japanese and Korean ma
r
kets.

Basic functions of the GUI include admini
s-
trative functions as patient and study selection, r
e-
port viewing and generation, and visualization fun
c-
tions as cine loop, ‘maximum intensity pr
o
ject
ion’


(MIP), ‘multi
-
planar reformatting’


(MPR

) inclu
d-
ing oblique and curved reconstructions, cut planes,
measurement tools for distances and angles, magn
i-
fying glass, annotations, etc.

The development of computer vision alg
o-
rithms often follows a hierarc
hical pathway. The
design process (rapid prototyping) is done in high
-
level software (examples are Mathematica

, M
a-
ple

, Matlab

), where very powerful stat
e
ments and
algebraic functionality make up for very short code,
but his is difficult to extent to the

huge multi
-
dimensional medical images. When the formulas are
understood and stable, the implementation is made
into lower languages, like C, C++, Java. When
ult
i
mate speed (and limited var
i
ability) is required,
the code can be implemented in har
d
ware (GPU

,
field programmable gate array’s (FPGA

), ded
i-
cated chips, etc.). Many packages offer software
develo
p
ment kits for joint development (e.g.
MevisLab


by MEVIS, ‘Insight Segmentation and
Registr
a
tion Toolkit’ (ITK

) by NLM, etc.).

9.4

3D Visualization

The

first breakthrough in the use of workstations
has been by the invention of generating realistic
3D views from tomographic volume data in the
1980s. Now 3D volume rendering is fully intera
c-
tive, at high resol
u
tion and real
-
time speed, and
with a wide varie
ty of options, making it a non
-
trivial matter to use it.

Many dedicated companies are now esta
b-
lished (such as Vital Images with Vitrea, Mercury
Co
m
puter Systems with Amira, Barco with Voxar,
3Mensio with 3Vision, Terarecon with Aquarius,
etc.). Often a th
ird party 3D viewing application is
integrated in the PACS viewing application, and
supplied as a co
m
plete system by such an ‘original
equipment manufacturer’


(OEM

).

The principle of
ray tracing
(‘rendering’

)
(
N
OWINSKI
et al. 2005) is actually based on
mi
m-
icking the physics of light reflection with the
computer: the value of a pixel in a 2D image of a
3D view (also called a 2.5D view) is calculated
from the reflected amount of light from a virtual
light source, either bouncing on the surface of the
3D da
ta (this process is called ‘surface rende
r-
ing’

), or


as


the

summation of all contributions







Fig.

9.3.

Virtual colonoscopy with unfolding enables inspection of folds from all sides.

From
V
IL
A
NOVA ET AL
.

(
2003
)

from the inside of the 3D dataset along the line of
the ray in question, composed with a fo
r
mula that
takes into account the transparency (or the i
n-
verse: the opacity) of the voxels (this process is
called ‘volume re
n
dering’

).

The use
r

can change the opacity settings
by manipulating the so
-
called ‘transfer fun
c-
tion’

, this function giving the relation b
e
tween
the measured pixel value from the scanner and
the

opacity. As there is an infinite number of se
t-
tings possible, users often get co
n
fused, and a
standard set of settings is supplied, e.g. for lung
vessels, skull, abdominal va
s
cular, etc., or a set of
thumbnails is given with e
x
amples of presets,
from whic
h the user can choose. Attempts are
underway to extract the optimal se
t
tings from the
statistics of the data itself (
N
OWI
N
SKI
et al. 2005).

In
virtual endoscopy
(e.g. colonoscopy)
the camera is positioned inside the 3D dataset.
Cha
l
lenges for the computer
vision application
are the aut
o
matic calculation of the optimal path
for the fly
-
through through the center of the
win
d
ing c
o
lon, bronchus or vessel. Clever new
visualizations have been designed to screen the
foldings in the colon for polyps at both the fo
r-
ward as backward pass simultan
e
ously: unfolding
(
V
ILANOVA

2003) (see Fig.

9.3
) and viewing an
u
n
folded cube (
V
OS
et al. 2003) (see Fig.

9.4
).

Segmentation
is the process of dividing
the 3D dataset in meaningful entities, which are
then visua
l
ized separate
ly. It is essential for 3D
viewing by, e.g. cut
-
away views, and also, unfo
r-
tunately, one of the most difficult issues in co
m-
puter vision. It is discussed in more detail in
Sect.

9.5. When clear contrasts are available, such

Fig.

9.2.

Volume rendering of the heart and cor
o-
naries (Terarecon Inc)

as in contrast enhanced CT or MR

angiography
and bone structures in CT, the simple tec
h
niques
of thresholding and region growing can be e
m-
ployed, up to now the most often used segment
a-
tion technique for 3D volume visualization.


Fig.

9.4.

Unfolded cube projection in virtual
colonoscopy
. From
V
OS
et al. (2003)

This also explains the popularity of
ma
x-
imum intensity projection

, where pixels in the
2.5D view are determined from the maximum
along each ray from the viewing eye through the
dataset (such a diverging set of rays leads to a so
-
called ‘perspective rendering’

). As this may
ea
s
ily lead to depth ambiguities, often the more
appealing ‘closest vessel projection’


(CVP) is
applied, where the local max
i
mum values closest
to the viewer is taken. The sampled points of the
(oblique) rays
through the dataset are mostly l
o-
cated in between the regular pixels, and are calc
u-
lated by means of interp
o
lation

.

9.5

Computer Aided Dete
c-
tion (CAD)

One of the primary challenges of intelligent sof
t-
ware in modern workstations is to assist the h
u-
man expe
rt in recognition and classification of
disease processes by clever computer vision alg
o-
rithms. The often used term ‘computer
-
aided d
i-
agnosis’ may be an overstatement (better: ‘co
m-
puter
-
aided detection’), as the final judgement
will remain with the radiolo
gist. Typically, the
computer program marks a r
e
gion on a medical
image with an annotation, as an attention sign to
inspect the location or area in fu
r
ther detail. The
task for the software developer is to translate the
detection strategy of the expert int
o an efficient,
effective and robust computer vision algorithm.
Modern techniques are also based on (supe
r
vised
and unsupervised) ‘data mining’


of huge ima
g-
ing databases, to collect statistical appearances.
E.g. lear
n
ing the shape and texture properties o
f a
lung nodule from 1500 or more patients in a
PACS database is now within reach. Excellent
reviews exist on current CAD tec
h
niques and the
perspectives for CAD (
D
OI
2006;
G
ILBERT
and
L
EMKE
2005). The field has just begun, and some
first successes have be
en achieved. Ho
w
ever,
there is a huge amount of development still to be
done in years to come.

Some advances in CAD techniques that
have brought good progress are in the following
applic
a
tion areas.




Fig.

9.5a

c.

Virtual colonoscopy with surface smoothing.

a

Original dose (64

mAs);
b

6.25

mAs;
c

1.6

mAs. From
P
E
TERS
(2006b)


Mammography:

this has been the first
field where commercial applications found
ground, in part
icular due to the volume produ
c-
tion of the associated screening, the high resol
u-
tion of the modality and the specific search tasks.
Typical search tasks involve the automated dete
c-
tion of masses, micro
-
calcifications, stellate or
spiculated tumors, and the

location of the nipple.

How do such algorithms work? Let us
look in some detail to one example: stellate tumor
detection (
H
OFMAN
et al. 2006). As breast tissue
co
n
sists of tubular structures from the milk
-
glands to the nipple, tumor extensions may pre
f-
era
bly follow these tubular pathways. In a proje
c-
tion radi
o
graph this leads to a spiculated or star
-
shaped pa
t
tern. The computer will inspect the
contextual environment of each pixel (say 50

×

50

pixels) on the presence of lines with an orie
n-
tation pointing t
owards the rel
e
vant pixel. In this
way a total of 2500 ‘votes’ are collected for each
pixel. The pixels with a voting probability e
x-
ceeding some threshold are possible candidates
for further i
n
spection.

The location of the nipple is important as
a general
coordinate origin for localization refe
r-
ences with, e.g. previous recordings. The general
statistical ‘flow’ of line structures points towards
the nipple; the location can reasonably well be
found by
modelling

the apparent statistical line
structure with p
hys
i
cal flow models.

The role of MRI in breast screening is ri
s-
ing. As in regular anatomical scans, too many
false neg
a
tive detections are found, and current
attention now focuses on dynamic contrast e
n-
hanced MRI. The rationale is the high vascularity
of t
he neoplasm, leading to a faster uptake and
outwash over time of the contrast medium co
m-
pared to normal tissue. Current research focuses
on the understanding of this vascular flow pattern
(e.g. by two
-
compartment mode
l
ling) and the
optimal timing of the im
age s
e
quence.

Polyp detection in virtual colonography
:
polyps are characterized by a mushroom
-
like e
x-
trusion of the colon wall, and can be detected by
their shape: they exhibit higher local 3D curv
a-
ture


(‘Gaussian cu
r
vature’

) properties. These
can be det
ected with methods from ‘differential
geometry’


(the theory of shapes and how to
measure and characterize them), and highlighted
as, e.g. colored areas as attention foci for further
inspe
c
tion.

Methods have been developed to carry out
an electronic cleans
ing


of the colon wall when
co
n
trast medium is still present. An interesting
current target is possible to reduce strongly the
radi
a
tion dose of the CT scan, and still be able to
detect the polyp structures, despite the deterior
a-
tion of the detected colon
wall structures. Clever
shape smoot
h
ing techniques and edge
-
preserving
smoothing of the colon surface have indeed e
n
a-
bled a su
b
stantial dose reduction.

Thorax CAD
: here the focus is on the a
u-
tomated detection of nodules in the high res
o
l
u-
tion multi
-
slice C
T (MSCT) data, on the dete
c
tion
of pulmonary emboli, and of textural analysis by
classification of pixels, e.g. for the quantific
a
tion
of the extent of sarcoidosis. See
S
LUIMER
et al.
(2006) for a review.

Other CAD applications include
calcium
scoring
, use
d to detect and quantify calcified co
r-
onary and aorta plaques, analysis of
retinal fu
n-
dus images

for leaking blood vessels as an early
indic
a
tor for diabetes, and the inspection of skin
spots for melanoma (of particular attention in
Au
s
tralia).

9.6

Atlases

The use of interactive 3D atlases on medical
workstations is primarily focused on education
and surgery. As an example, K.
-
H. Höhne’s pi
o-
neering Voxel
-
Man series of atlases (
H
OFMAN
et
al. 2006) was initiated by the ‘visible human pr
o-
ject’

. Atlases for br
ain surgery (e.g. the Cerefy
Brain atlas family;
N
OWINSKI
et al. 2005) now
become probabilistic, based on a large number of
patient stu
d
ies.


Fig.

9.6.

The famous Voxel
-
Man atlas explored
many types of optimal educational visualization.
From
H
ÖHNE
(2004)


9.7

CAD/CAM Design

Workstations can also assist in the creation of
i
m
plants from the 3D scans of the patients. This
is a highly active area in ENT, dental surgery,
orthopedic surgery and cranio
-
maxillofacial su
r-
gery. Many design techniques have been dev
e
l-
oped to create the new shapes of the implants,
e.g. by mi
r
roring the healthy parts of the patient
of the opposite side of the body, 3D region gro
w-
ing of triangulated ‘f
i
nite element models’


in the
assigned space, etc. The ‘standard tesselation
language’


(STL

) is a common format to d
e-
scribe surfaces for 3D milling equipment for ra
p-
id prototyping

, such as stereolithography


sy
s-
tems, plastic droplets ditherers, five
-
axes co
m-
puterized milling machines, laser powder sinte
r-
ing systems, etc. Many dedicated r
apid pr
o
toty
p-
ing companies exist (e.g. Materialize Inc., see
also
www.cc.utah.edu/~asn8200/rapid.html
). In
the medical arena several large research inst
i
tutes
are active in this area (Ceasar, Berl
in; Co
-
Me,
Zürich).

9.8

Diffusion Tensor Imaging


⡄(I




Tractography


Three
-
dimensional (3D) visualization of fiber
tracts in axonal bundles in the brain and muscle
fiber bu
n
dles in heart and skeletal muscles can
now be done interactively. The images ar
e no
longer composed of scalar


(single) values in the
voxels, but a complete diffusion tensor


(a 3

×

3
symmetric matrix

) is measured in each voxel.

The three so
-
called eigenvectors


can be
calculated with methods from linear algebra

;
they span the ellipsoid of the Brownian motion


that the water molecules make at the location of
the voxels due t
o thermal diffusion. Complex
mathematical methods are being investigated to
group the fibers in meaningful bundles, to se
g-
ment and register the DTI data with anatomical
data, and find fiber crossings and endings aut
o-
matically. An interesting development is

the ph
o-
torealistic rendering of the tiny bundle stru
c
tures
(with specularities and shadows), based on the
physics of the rendering of hair.


Fig.

9.7.

Muscle fibers tracked in a high
-
resolution DTI MRI of a healthy mouse heart.
Lighting and sha
d
owing of lines combined with
color coding of helix angle (

h
). From
P
EETERS
et
al. (2006a)


9.9

Registration

Registration, or matching,
is a classical technique
in image analysis (
H
AJNAL
et al. 2001). It is e
m-
ployed to register anatomical to anatomical, or
anatomical to functional data, in any dimension.
Examples are MRI
-
CT, PET
-
CT, etc. The co
n-
struction of a PET and a CT gantry in a singl
e
system effectively solves the re
g
istration problem
for this modality.

The matching can be global (only transl
a-
tion, orientation and zooming of the image as a
whole) or local (with local deformation, also
called warping

). Registration can be done by
find
ing correspo
n
dence between (automatically
detected) lan
d
marks, or on the intensity landscape
itself (e.g. by correlation

). There is always an
entity (a so
-
called functional

) that has to be
mi
n
imized for the best match: e.g. the mean
squared distance betw
een the landmarks, a
Pie
r
son correlation coefficient, or others. In pa
r-
tic
u
lar, for multi
-
modality matching, the mutual
i
n
formation


(MI) has been found to be an effe
c-
tive minimizer. As an example, in MRI bone
voxels are black and in CT white; they show as

a
cluster in the joint probability hi
s
togram of the
MR vs CT intensities. The MI is a measure of
entropy (disorder) of this hist
o
gram.

9.10

RT Dose Planning

The accuracy of radiotherapy dose calculations,
based on the attenuation values of the CT scan of
the patient, needs to be very high to prevent u
n-
derexp
o
sure of the tumor and overexposure of the
healthy tissue. Typically the isodose surfaces are
calculated and viewed in 3D in the context of the
local anatomy. Increa
s
ingly the images made in
the linear
a
c
cellerator with the electronic portal
imaging device


(EPID) are used for precise l
o-
calization of the beam and repeat positioning of
the patient, by precise registration techniques.
The low contrast images (due to the high voltage
of the imaging beam) ca
n be enhanced by such
techniques as (ada
p
tive) histogram equalization

.

9.11

Quantitative Image

Anal
y
sis

This is the fastest growing application area of
med
i
cal workstations. The number of applications
is vast, every major vendor has research activities
i
n this area, and many research institutes are a
c-
tive. To quote from the scope of ‘Medical Image
Analysis’, one of the most influential scientific
journals in the field:

“The journal is interested in approaches
that utilize biomedical image datasets at all
sp
a-
tial scales, ranging from molecular/cellular ima
g-
ing to tissue / organ imaging. While not limited to
these alone, the typical biomedical image datasets
of interest include those acquired from: magnetic
res
o
nance, ultrasound, computed tomography,
nuclear

medicine, X
-
ray, optical and confocal
microscopy, video and range data images.

The types of papers accepted include
those that cover the development and impleme
n-
tation of algorithms and strategies based on the
use of var
i
ous models (geometrical, statistic
al,
physical, functional, etc.) to solve the following
types of problems, using biomedical image d
a-
t
a
sets:




Fig.

9.8.

Multimodality MRI of atherosclerotic plaque in the human carotid artery: (w1) T1
-
weighted 2D
TSE, (w2) ECG
-
gated proton de
n
sity
-
we
ighted TSE, (w3) T1
-
weighted 3D TFE, (w4) ECG
-
gated partial
T2
-
weighted TSE, (w5) ECG
-
gated T2
-
weighted TSE.
Middle: Feature space for cluster analysis. Right:
classification r
e
sult. From
H
OFMAN
et al.
(2006)

R
epresentation of pictorial data, visualiz
a-
ti
on, feature extraction, segmentation, inter
-
study
and inter
-
subject registration, longitudinal / te
m-
poral studies, image
-
guided surgery and interve
n-
tion, texture, shape and motion measurements,
spectral analysis, digital anatomical atlases, st
a-
ti
s
tical sha
pe analysis, computational anatomy
(mode
l
ling normal anatomy and its variations),
computational physiology (mode
l
l
ing organs and
living systems for image analysis, simulation and
training), virtual and augmented reality for the
r
a-
py planning and guidance, t
elemedicine with
medical images, tele
-
presence in medicine, tel
e-
surgery and i
m
age
-
guided medical robots, etc.”

See also the huge amount of toolkits for
computer vision:
http://www.cs.cmu.edu/~cil/v
-
s
ource.html
. Important conferences in the field
are MICCAI, CARS, IPMI, ISBI and SPIE MI. In
the following some often
-
used techniques are
shortly discussed. There are excellent tutorial
books (
M
O
LECULAR VISUALIZATIO
NS
undated;
Y
OO
2004) and review papers f
or the field.

Segmentation


is a basic necessity for
many subsequent viewing or analysis applic
a-
tions. Mostly thresholding and 2D/3D region
growing are applied, but these often do not give
the required result. Proper segmentation is not
o-
riously difficult.
There are dozens of techniques,
such as model
-
based segmentation, methods
based on statistical shape variations (‘active
shape mo
d
els’

), clustering methods in a high
-
dimensional feature space (e.g. for textures), hi
s-
togram
-
based methods, physical models o
f co
n-
tours (‘snakes’, level sets

), region
-
growing


methods, graph partitioning


methods, and multi
-
scale segment
a
tion

.

The current feeling is that fully automated
segmentation is a long way off, and a mix should
be made between some (limited, initial) us
er
-
interaction (semi
-
automatic segmentation).

Feature detection


is the finding of sp
e
ci
f-
ic landmarks in the image, such as edges, co
r
ners,
T
-
junctions, highest curvature points, etc. The
most often used mathematical technique is multi
-
scale diffe
r
ential g
eometry


(
T
ER
H
AAR
R
OMENY
2004). It is interesting that the early stages of the
human visual perception system seem to e
m
ploy
this strategy.

Image enhancement


is done by calcula
t-
ing specific properties which then stand out rel
a-
tive to the (often noisy) ba
ckground. Examples
are the likeness of voxels to a cylindrical stru
c-
ture by curvature rel
a
tions (‘vesselness’

), edge
preserving smoothing

, coherence enhancing

,
tensor voting

, etc. Comme
r
cial applications are,
e.g. MUSICA (‘Multi
-
Scale Image Contrast A
m-
plification’, by Agfa), and the Swedish Co
n-
textVision (
http://www.contextvision.se/
). E
n-
hancement is o
f
ten used to cancel the noise
-
increasing effects of substantially lowering the X
-
ray dose, such as in fluoro
scopy and CT scree
n-
ing for virtual colonoscopy.

Quantitative MRI
is possible by calcula
t-
ing the real T1 and T2 figures from the T1 and T2
weighted acquisitions, using the Bloch equation


of MRI physics. Multi
-
modal MRI scans can be
exploited for ti
s
sue cla
ssification: when different
MRI techniques are applied to the same volume,
each voxel is measured with a different physical
pro
p
erty, and a feature space can be constructed
with the physical units along the dimensional
axes: e.g. in the characterization of

tissue types in
atherosclerotic lesions with T1, T2 and pr
o
ton
density weighted acquisitions, fat pixels tend to
cluster, as do blood voxels, muscle voxels, calc
i-
fied voxels, etc.
Pattern recognition techniques like neural ne
t-
works


and Bayesian statisti
cs


may find the
proper cluster boundaries.

Shape
can be measured with differential
geometric properties, such as curvature

, saddle
points

, etc. It is often applied when, e.g. in the
automated search for (almost) occluded lung ve
s-
sels in pulmonary emboli
, polyps on the colon
vessel wall, measuring the stenotic index, spic
u-
lated lesions in mammography, etc. A

popular
method is based on ‘active shape models’

,
where the shape variation is established as so
-
called shape eigenmodes


by analyzing a large
set o
f variable shapes and performing a ‘principal
component analysis’

, a

well known mathemat
i-
cal technique. The first eigenmode gives the main
variation, the second the one but largest, etc. Fi
t-
ting an atlas or model
-
based shape on a patient’s
organ or segmen
ted structure becomes by this
means much more computationally efficient.

Temporal analysis
is used for bolus trac
k-
ing (time
-
density quantification), fun
c
tional maps
of local perfusion parameters (of heart and brain),
contrast
-
enhanced MRI of the breast, ca
rdiac ou
t-
put calcul
a
tions by measuring the volume of the
left ventricle over time, multiple sclerosis lesion
growth / shrinkage over time, regional ca
r
diac
wall thickness variations and local stress/strain
calcul
a
tions, and in fluoroscopy, e.g. the freezin
g
of the stent in the video by cancell
a
tion of the
motion of the cor
o
nary vessel.

9.12

Workstations for Life
Sciences

In life sciences research a huge variety of (high
d
i
mensional) images is generated, with many new
types of microscopy


(confocal

, two
-
pho
ton

,
cryogenic transmission electron microscopy

,
etc.) and dedicated (bio
-
) medical small animal
scanners (micro
-
CT, mini PET, mouse
-
MRI,
etc.). The r
e
search on molecular imaging and
molecular medicine is still primarily done in
small animal models.


F
ig.

9.9a,b.

Two
-
photon florescence microscopy
of collagen fibers of tissue
-
engineered heart
-
valve tissue.
a

Result of structure preserving d
e-
noising.
From
D
ANIELS
et al. 2006

There is great need for quantitative image
analysis. An example is, e.g. the mea
surement of
quant
i
tative parameters that determine the
strength of newly engineered heart valve tissue of
the p
a
tient’s own cell line, such as collagen fiber
thickness, local orientation variation and tortuo
s-
ity

. The source images are from two
-
photon
micr
o
s
copy, where the collagen is specifically
colored with a collagen specific molecular ima
g-
ing marker.

Another example is the detailed study of
the micro
-
vascular structure in the goat heart
from ultra
-
thin slices of a cryogenic microtome


(degree of branch
ing, vessel diameter, diffusion
and perfusion distances, etc.). Typical resolution
is 25

50

micron in all directions, with datasets of
2000
3
.


Fig.

9.10.

A 3D visualization of a microtome stack
(40×40×40


洩m潦o瑨攠m楣io
-
癡獣畬a瑵牥 潦oa goat
桥a牴

VAN
B
AVEL
et al. 2006) [
B
ENNINK

2006]

This research arena will benefit greatly in
the near future from the spectacular developments
in the diagnostic image analysis and visualization
workst
a
tions.

9.13

Computer
-
Aided

Su
r
gery (CAS)

In the world of CAS some ver
y advanced simul
a-
tion and training systems (KISMET, Voxel
-
Man)
have been created. Especially in dental implants,
craniofacial surgery and lapar
o
scopic surgery
there are many and highly advanced systems t
o-
day. Surgical navigation workstations are ro
u
tin
e-
ly
displaying the combination of the anatomy and
the position and orientation of the instr
u
ments in
the operating the
a
tre.


Fig.

9.11.

Virtual laparoscopy trainer (Origin:
Forschungszentrum Karlsruhe KISMET)

An interesting development is the use of
complex

fluid dynamics
modelling
, which e
n
a-
bles the prediction of rupture chances in a
b-
dominal aorta surgery, and selecting optimal
therapeutic proc
e
dures with bypass surgery in the
lower aorta.

In neurosurgery workstations can be e
m-
ployed in the calculation of a
n optimal (safest)
insert path for electrodes for deep brain stimul
a-
tion (DBS), based on a minimal costs path avoi
d-
ing blood ve
s
sels and ventricles, and starting in a
gyrus. Workstations assist in inter
-
operative vis
u-
alization by warping the pre
-
operative
imagery to
the real situation in the patient during the oper
a-
tion, by intra
-
operative MRI, or ultrasound.


Fig.

9.12a

c.

Abdominal aorta aneurysm:
a

color
coding of displacement (mm);
b

Von Mises strain;
c

Von Mises stress (kPa).
From
DE
P
UTTER
et al.
(2
005)


9.14

New Developments

The visual perception of depth (when viewing
3D) data is helped enormously if the viewer can
move the data himself. There are many depth
cues (stereo, depth from motion, depth from pe
r-
spective), but depth from motion is the str
ongest.
That is why maximum intensity projections
(MIP) are preferably viewed d
y
namically. By
self
-
tracking also the muscle’s proprioceptors are
giving fee
d
back to the brain, adding to the visual
sensation. The combination with human’s superb
eye
-
hand coor
dination has led to the concept of
the Dextroscope (
www.dextroscope.com
), where
a (computer generated) view or object can be
manipulated under a half
-
transparent mirror,
through which the viewer sees the display.

Di
s-
plays can also be equipped with haptic (tactile)
feedback systems, which are now commercially
available.


Fig.

9.13.

Stereo viewing and manipulation with
haptic feedback

Super
-
large screens, and touch screens are
becoming
popular; a new trend is the multi
-
touch
screen (
http://cs.nyu.edu/~jhan/ftirtouch/

with
movie), where multiple positions to interact s
i
m-
ultaneously make more complex transform
a
tions
possible, such as zoom
ing, multiple simu
l
taneous
objects interactions, etc.

9.15

Outlook

We have actually just started with exploiting the
huge power these super assistants can add, in any
of the fields discussed above


hardware, sof
t-
ware and integration. Image processing play
s an
essential role, be it for visualization, segment
a-
tion, computer
-
aided detection, navigation, regi
s-
tration, or quantitative analysis. There will be an
ever greater need for clever and robust alg
o-
rithms: it is the conviction of the author that the
study

of human brain mechanism for the inspir
a-
tion for such algorithms has a bright future to
come (
TER
H
AAR
R
OMENY
2004). The radiol
o-
gi
sts will benefit from these sup
er
-
assistants, and
finally: the p
a
tient has the best benefit of all.

References

Barco (2007
) W
hite Papers Barco (screens, DICOM):
http://www.barco.com/medical/en/downloads/whit
epapers.asp?dltype=15

Bovik AC (ed) (2000) Handbook of image and video pr
o-
c
essing (commun
ications, networking and mult
i-
m
e
dia). Academic Press

Burroni M et al. (2004) Melanoma computer
-
aided diagn
o-
sis: reliability and feasibility study. Clin Cancer
Res 10:1881

1886

Daniels F, ter Haar Romeny BM, Rubbens MP, van Assen
HC (2006) Quantification of

collagen orientation
in 3D engineered tissue. In: Ibrahim F (ed) Proc
Int Conf on Biomedical Engineering BioMed
2006, Kuala Lu
m
pur, Malaysia, pp 344

348

de Putter S, Breeuwer M, Kosea U, Laffarguec F, Rouet J,
Hoogeveen R, van den Bosch H, Buthe J, van de

Vosse F,
Gerritsen FA (2005) Automatic determ
i-
nation of the dynamic geometry of abdominal ao
r-
tic aneurysm from MR with application to wall
stress simulations. Proc
CARS, International Co
n-
gress Series 1281, pp 339


344

DICOM (2007
a) standard:
http://medical.nema.org/

DICOM (2007

b) conformance statement (example):


https://www.merge.com/RESOURCES/pdf/dcs/dc
s_efilm21.pdf

Doi K (2006) Diagnostic

imaging over the last 50 years:
research and development in medical imaging sc
i-
ence and technology. Phys Med
Biol 51:R5

R27

Eizo (undated) White Papers Eizo (screens):
http://www.eizo.com/support/wp
/index.asp

Gilbert FJ, Lemke H (eds) (2005) Computer
-
aided diagn
o-
sis. Special issue of British Journal of Radiology,
vol. 78, British Inst
i
tute of Radiology

GPGPU (undated) General
-
purpose computation using
grap
h
ics hardware,
http://www.gpgpu.org/

Hajnal JV, Hill DLG, Hawkes DJ (eds) (2001) Medical
image registration. CRC

Health Level 7 (undated) Health Level 7,

http://www.hl7.org/
.
See also regional sites
:

(Au
s
tralia
http://www.hl7.org.au/
,


UK
http://www.hl7.org.uk/
,


Canada
http://www.cihi.ca/hl7
, etc.)

Hofman JMA, Branderhorst WJ, ten Eikelder HMM, Ca
p-
pendijk V
C, Heeneman S, Kooi ME, Hilbers PAJ,
ter Haar Romeny BM (2006) Quantification of
atherosclerotic plaque components using in
-
vivo
MRI and supervised classifiers”, Magn Reson Med
55:790

799

Höhne K
-
H (2004) VOXEL
-
MAN 3D
-
Navigator” (CD
-
ROM).
Springer,

Berlin

Heidelberg New York

Karssemeijer N (1995) Detection of stellate distortions in
mammograms using scale
-
space operators. Proc.
Information Processing in Medical Imaging, pp
335

346

Kaufman, Müller K (2005) Overview of volume rendering.
The visualization hand
book. Elsevier

Molecular visualizations (undated) Molecular visualiz
a-
tions:
http://molvis.sdsc.edu/visres/index.html

Nowinski W, Thirunavuukarasuu S, Benabid G (2005) The
Cerefy clinical brain atlas
. Enhanced edition with
su
r
gical planning and intraoperative support (CD
-
ROM). Thieme

Peeters THJM, Vilanova A, Strijkers GJ, ter Haar Romeny
RBM (2006b) Visualization of the fibrous stru
c-
ture of the heart. Proc. 11th workshop on Vision,
Mo
d
elling

and Visu
alization. Aachen, Germany

Peeters THJM, Vilanova A, ter Haar Romeny RBM
(2006a) Visualization of DTI fibers using hair
-
rendering techniques.
In: Lelieveldt BPF,
Haverkort B, de Laat CTAM, Heijnsdijk JWJ (eds)
Proc ASCI 2006. Lommel, Belgium, pp 66

73

Pete
rs JF, Grigorescu SE, Truyen R, Gerritsen FA, de Vries
AH, van Gelder RE, Rogella P (2005) Robust
automasted polyp detection for low
-
dose and
normal
-
dise virtual colonoscopy.
Proc
CARS
2005, International Congress Series 1281, Be
r
lin,
Ge
r
many, pp 1146

1150

RSNA 2006 Exhibitor list:

http://rsna2006.rsna.org/rsna2006/V2006/exhibitor
_list/home.cfm

Schaefer
-
Prokop CM, van Delden OM, Bouma H, Sonn
e-
mans JJ, Gerritsen FA, Lameris JS (
2006) To a
s-
sess the added value of a prototype computer
-
aided
detection (CAD) system for pulmonary embolism
(PE) in co
n
trast
-
enhanced multi
-
detector computed
tomography (CT) images. Proc Eur
Conf of Rad
i-
ology, Vienna, Austria, EPOS Poster

Sereda P, Vilanov
a A, Gerritsen FA (2006) Automating
transfer function design for volume rendering u
s-
ing hierarchical clustering of material boundaries.
In: Sousa Santos B, Ertl T, Joy K (eds) E
u-
rograp
h
ics/IEEE VGTC Symposium on Visualiz
a-
tion (Eur
o
Vis),Lisboa, Portugal, pp

243

250

Siggraph (undated) Ray tracing tutorial (SIGGRAPH):
http://www.siggraph.org/education/materials/Hype
rGraph/raytrace/rtrace0.htm

Sluimer IC, Schilham AMR,
Prokop M, van Ginneken B
(2006) Computer analysis of computed tomogr
a-
phy scans of the lung: a survey. IEEE Transactions
on Medical Imaging, vol. 25, pp. 385

405

Sonka M, Michael Fitzpatrick J (eds) (2000) Handbook of
medical imaging


vol. 2. Medical image

pr
o-
ces
s
ing and analysis. SPIE, Belligham, WA

ter Haar Romeny BM (2004) Front
-
end vision and multi
-
scale image analysis.
Springer,

Berlin

Heidelberg
New York

ter Haar Romeny BM (2004) Front
-
end vision and multi
-
scale image analysis.
Springer,

Berlin

He
idelberg
New York

van Bavel E, Bakker EN, Pistea A, Sorop O, Spaan JA
(2006) Mechanics of microvascular
remodelling
.
A review. Clin Hemo
r
heol Microcirc 34(1/2):35

41

Vilanova A, Gröller E (2003) Geometric modelling for
virtual colon unfolding. In: Brunnett
, Harmann,
Müller, Lisen (eds) Geometric modeling for scie
n-
tific visualization.
Springer,

Berlin

Heidelberg
New York
, pp 453

468

Vos FM, van Gelder RE, Serlie IWO, Florie J, Nio CY,
Glas AS, Post FH, Truyen R, Gerritsen FA, Stoker
J (2003) Three
-
dimensiona
l display modes for CT
colonography: conventional 3D virtual colon
o
sc
o-
py versus unfolded cube proje
c
tion. Radiology
228:878

885

Voxel Man surgical workstation:
htt
p://www.uke.uni
-
hamburg.de/medizinische
-
fakultaet/voxel
-
man/index_ENG.php

Whitby J (2006) The DICOM Standard”, White paper Ba
r-
co Inc, 2006. URL:
http://www.barco.com/barcoview/d
ownloads/Whit
ePaper_DICOM.pdf


Yoo TS (2004) Insight into images: principles and practice
for segmentation, registration, and image analysis.
AK Peters