Medical Image Processing Andrew Todd-Pokropek July 2009 ...

paradepetAI and Robotics

Nov 5, 2013 (4 years and 6 months ago)


Medical Image Processing July 2009
Andrew Todd-Pokropek ©UCL
UKRC Manchester June 2007
SSIP July 2009
Medical Image Processing: An overview.
From Raw Data to Quantitative Information
The links and special differences between
General Image Processing and
Medical Image Processing
Andrew Todd-Pokropek
University College London,
[INSERM U678 Paris]
• What can Physics and Computing do in medicine.
• Quantification
• Some clinical objectives, diagnosis and therapy
• Segmentation, …
• Registration, fusion, tracking change
• Into therapy
• Modelling and multi-scale imaging
• Image Science group UCL (CS and Medphys)
• Questions of validation
• IRC (UCL, KCL, Oxford, Manchester, IC)
• INSERM, Yale, Mayo, UCLA, Geneva, Cedar Sinai,
and many others
Medical Image and Signal (MIAS) Research Consortium
A road map
3 4
IIn ns sttr ru um me en ntta attiio on n
Can Physics and Computing change
Preprocessing Knowledge
Data Image
Acquisition Manipulation
• Change can be beneficial (or not) and be:
Interpretation Decision Evaluation
Complementary Data
– New ways of doing things
– The provision of different information
– Including information from other sources (e.g. fusion)
– Altering patient management
Help for image manipulation and intelligent useage
– Altering costs and convenience
– Standardisation
– Evaluation and validation
The Imaging Chain
– etc
5 6
Dripping Nuclear Medicine
SSIP 2009 ©UCL 1Medical Image Processing July 2009
Andrew Todd-Pokropek ©UCL
An example from CT
Two major domains of research
• Spiral CT (trivial, easy, but very significant)
• Multi-slice (Mostly obvious, not trivial to develop)
• Methodology, including:
• Photon counting CT (Can’t be done on real systems
– Physics
yet) – Mathematics and Statistics
– Informatics
– (Engineering)
• Applications , including:
– Cancer
– Neuroscience
– Cardiovascular
– Infection
– (the –omics)
7 8
Medical Physics and Biomedical Engineering
Some History
Bohr, Franck, von Hevesy
9 Goedel Enstein 10
Equipment An important annex issue-
Discharge tube
50 kV
• What is the place of high
Vacuum pump
technology in Global
• Can high tech. be replaced
(20 Hz)
by low tech. or alternative
Bank of the Nile in Sudan
Lead battery
11 12
SSIP 2009 ©UCL 2Medical Image Processing July 2009
Andrew Todd-Pokropek ©UCL
Kelvin and Quantification
• "In physical science the first essential step in the direction of
So- Why do we want Quantify?
learning any subject is to find principles of numerical
reckoning and practicable methods for measuring some
quality connected with it. I often say that when you can
measure what you are speaking about, and
express it in numbers, you know something • A description without numbers is a very poor things [Lord
about it; but when you cannot measure it, when
• To determine limits e.g. normal/ abnormal
you cannot express it in numbers, your
• To determine progress e.g. increasing/decreasing
knowledge is of a meagre and unsatisfactory
kind; it may be the beginning of knowledge, but you have • For research e.g. new classes/ phenomena
scarcely in your thoughts advanced to the state of Science,
whatever the matter may be." [PLA, 1883-05-03]
• Note difference between absolute and relative quantitation
– Different regions/ times
• "If you can not measure it, you can not improve it."
– MBq /ml
• But it is only one step in the process
B But ut w we e ha hav ve e be bee en n w wa aiit tiin ng g f for or a a llon ong g t tiim me e
13 14
Basic Image Processing
Image/ Signal processing in general
Type of quantification
Histogram Equalization
• Aims
• Relative
– One region with respect to another
– One time with respect to another
– One patient with respect to another (or a group/
• Many different types of data
– MR, CT, NM, US
• Absolute
....Microscopy, Visual,
– Therapy and Patient
– Size/ uptake of tracer
– Research (small animal systems -
– Physiological parameters
drug discovery)
15 16
– Model fitting
Some Examples
Some Tools
• Microcalcifications Processing
• Tumour staging
Physical, Mathematical
– Detection
– Detection
• matched filter
• segmentation
– Measurement
– Measurement
• how many
• volume
• shape
– Description
– Description
• staging
• benign/ malignant
• diagnostic strategy
Manipulation of
Evaluation Interpretation
Associated Data
17 18
SSIP 2009 ©UCL 3Medical Image Processing July 2009
Andrew Todd-Pokropek ©UCL
Classical Image Processing
M odelsinImageAnalysis
• Classic ‘edge detection’ methods
– Gradient (Sobel etc), zero crossings of Laplacian recoveryof quantitativeinformationfromimages.
– Canny
– Marr Hildreth
• Phase congruency
im age Boundary
• Model based
– Medial axis MREP
– Active shape
• Clustering
– Split merge
– K-Means
Ack Taylor
– Affinity
• etc
19 20
Featurem ap model
Integrated Segmentation via Game Theory
(Chakraborty & Duncan) Sub-voxel operations
P = X
(classified pixels)
1 2
Image P P
• Label on sub-pixel are
P = p
defined from higher
resolution image
Boundary Finding
Reaction curvefor player 1
• Higher resolution image
1 resolution downgraded to
F : constant
SPECT/PT pixel size
level curves
1 1 2 1 1 2
F (P ;P ) = f (P )+ ëf (P ;P )
1 21
• PVE defined if no. of
F : constant
2 1 2 2 1 2
F (P ;P ) = f (P )+ ì f (P ;P )
compartments (labels) is
2 ß 12
level curves
player 2
21 22
Equilibrium p
Endoscopic surgery
Colon segmentation
Ack Sorantin
Key question: how much
does the tumour invade
The colonic wall.
23 24
SSIP 2009 ©UCL 4Medical Image Processing July 2009
Andrew Todd-Pokropek ©UCL
Myocardial functional imaging
Cerebral Vascular Accident
Left Ventricle Right ventricle
Imaging of
contraction - perfusion
Automatic Estimation of Estimation of contraction
regions of interest from (Vx) intra-myocardial
1st passage in NMR NMR velocity estimation
Markovian approach
Hibernation - Stunning
25 26
Modelling and Atlases
• Non-rigid registration
– change detection
Ack IRC/ Rueckert
– voxel- based
– segmentation
H Ho ow w t to o e es st ta abl bliis sh h
Pre-contrast Post-contrast Subtract Subtract NRR
27 28
t the he da dat ta aba bas se es s
A Ac ck k.. T Th ho om mp ps so on n
Quantitation in PET/CT
M Mo od de elllliin ng g C Ch ha an ng ge es s iin n B Brra aiin n M Mo orrp ph ho ollo og gy y::
B Brra aiin n A Attrro op ph hy y iin n A Allz zh he eiim me err’’s s D Diise sea ase se
Brain Image Simulated Atrophy
Brain Image Simulated Atrophy
Crum, Smith, Hill (KCL)
Crum, Smith, Hill (KCL)
T Tha hac cke kerr,, B Br rom omi il le ey y ( (M Mc ch h))
R Ros oss sor or,, F Fox ox (IoN/UCL)
29 30
Ack Ell and Schulthess
SSIP 2009 ©UCL 590mm
Medical Image Processing July 2009
Andrew Todd-Pokropek ©UCL
3D + Time (+ channel)
Cardiac acquisition (and registration)
Tracking of Change
• Over time for example CT lung nodules
• To monitor interventional therapy
• Response to therapy for example MS lesions
• Drug trials to observe targeting and physiological
response as opposed to outcome
Bandwidth in terms of
temporal resolution
Ack U494/U678
31 32
Preclinical Systems Drug discovery and small animal scanners
Rat bone scan
Used as the basis for one of the
Micro SPECT systems
• Rat: 400g, 65x250mm
• 5mCi Tc99m -MDP
• 30min. scan, 1.5h p.i.
• Apt3, Ø=2.0mm
• scan range 250mm
zoom of rat spine 400g rat 30g mouse
33 34
Ack Schramm
Registration and Fusion
Courtesy of: Liselotte Højgaard,MD DMSc, Annika Eigtved, MD ph.d., Anne Kiil Berthelsen, MD.
PET & Cyclotron Unit, Dept. Nuclear Medicine, Rigshospitalet, University of Copenhagen.
Malignant melanoma with normal liver CT & US
Resection and radiation therapy of glioblastoma
Image fusion FDG-PET and MRI: complete remission
35 36
Ack P Slomka
SSIP 2009 ©UCL 6
250mmMedical Image Processing July 2009
Andrew Todd-Pokropek ©UCL
Tracking Change
Ack Taylor
Resection and radiation therapy of astrocytoma II-III°, secondary
changes in MRI
Recurrence shown by image fusion (FDG-PET and MRI)
37 38
Ack P Slomka
Functional images and surgery
Before surgery
A Project –
Ack Unité 494 /678 INSERM, LENA UPR 640 CNRS, Unité 483 INSERM,
Centre MEG, Service de Neuroradiologie, Hôpital Pitié-Salpêtrière
Integrating statistical
Pre- Pre-operative
Cortical stimulation
operative multi-modality
fMRI visualisation
Ativation maps Association maps
Establishing a
* H Ho ow w t to o m ma ak ke e iit t
clincal interface
39 40
pra prac ct tiic ca all// r rou out tiine ne After surgery
•in collaboration with existing projects
Ack U494/u678
Image Guided Radiotherapy
PET/CT improving cancer treatment
[A different type of quantification]
Tracers- F18- FDG tumour cell volume
Courtesy of
Holy Name Hospital
IUDR tumour growth MISO hypoxia
Dizendorf, Univ. of Zurich: Diagnostic Imaging – PET/CT Fusion Proves Its Worth
Dizendorf , Univ. of Zurich: Diagnostic Imaging – PET/CT Fusion Proves Its Worth
41 42
SSIP 2009 ©UCL 7Medical Image Processing July 2009
Andrew Todd-Pokropek ©UCL
Motion modelling in lung radiotherapy
Blackall et al WCOMP 2003, McClelland et al SPIE Med Imag 2005, ESTRO 2005 Interventional Radiology
• Radiologist are becoming
• Increasingly important
• Some example
– Angioplasty
– Biopsy (breast etc)
– Computer Aided Surgery
– Micro-video capsules
43 44
Edwards et al, Imperial College London
MAGI system in the Operating Room:
Overlay of 3D preoperative image data
on stereo field of view of
binocular operating microscope
(Edwards et al
IEEE- Trans Med Imag. 2000)
45 46
RF Ablation of Lung Tumours
Robotically Assisted Lung
Robotically Assisted Lung
Radiotherapy Using Optical
Under CT Fluoroscopy
(Accuray’s Synchrony)
Images Courtesy of Bill Lees
Kevin Cleary et al Georgetown University, Washington
47 48
SSIP 2009 ©UCL 8Medical Image Processing July 2009
Andrew Todd-Pokropek ©UCL
Temperature buildup during sonication
Virtual Human
3DAnatomical Human
1.1 seconds 4.5 seconds 7.9 seconds 11.3 seconds
Physiological Human
Ack Dov Maor. Insightec
49 50
3DAH Multiscale Imaging
• Model relates underlying structure to clinical image
Optical Microscopy
51 52
Multiple Spatial and Temporal Scales
Multi-scale Imaging
The Challenge
For optimal product design
Texture analysis
which spatial and temporal Microstructural information
scales should be resolved?
53 54
SSIP 2009 ©UCL 9Medical Image Processing July 2009
Andrew Todd-Pokropek ©UCL
Levels of Structural Organization Quantitative MRI for Bone
• Direct Methods • Indirect Methods
Trabecular Bone Level Bone Tissue Level BoneUltrastructural Level
Whole Bone Level
-Trabecular architecture -Structure is merely -The structure is determined by
-The structural
Direct method is to measure the
determined by the
organization is - Length scale of the organization of the apatite
microstructure directly from the
determined by the trabecular thickness porosity of the bone and collagen that form the
100 μ m due to Haversian
bone external and constituents of the bone tissue, images
cannals , lacunae, and
internal geometry, (figure taken from MarottiG 1996,
Indirect method is to measure
the bone density canliculi Ital. J. Anat. Embryol. 101 25-79
the relaxation time of T *
distribution and the
bone anisotropy Ack, Bert van Rietbergen, Finite Element Modeling,
The Physical Measurement of Bone, 475- 510
55 56
Diffusion MRI Ultrashort TE (UTE)
Trabecular bone (IR 500/0.08 minus 4.5/200msec at
T weighted image of an axial slice of a portion of bovine
1.5T, resolution 512 512, FOV 14 cm). Difference
epiphysis extracted by a bovine femur covered by a layer
UTE image of the skull. Trabecular bone can be seen
of fat
between the inner and outer tables.
MD and FA maps of an axial slice of a portion of
C. Rossi et al. / Magnetic Resonance Imaging 23 (2005) 245–248
epiphysis covered by a layer of fat
57 58
Ack Bydder
Finite Element Methods Structure of Articular Cartilage
• Zonal orientation of collagen
Volumetric Spatial Decomposition
spatially decomposed trabecular bone structures
into rods and plates
Image Guided Failure Assessment of Bone
59 60
FE Model
AckR. Muller et al
SSIP 2009 ©UCL 10Medical Image Processing July 2009
Andrew Todd-Pokropek ©UCL
Articular Cartilage MRI of Cartilage
• Imaging cartilage is challenging
• Imaging cartilage
• Thin (<4mm), leads to significant partial volume effect
– Thickness and curvature
• Osteoarthritis (OA) common cartilage disease, significant
area of current research interest – Imaging ECM constituents
• Early detection of OA – changes to structure
• Monitor potential treatments
– Disease slowing drugs?
– Disease altering drugs?
• Intervention
– Knee Replacement
– Graft
61 62
Burstein 2003
Adaptive Multiscale Modeling
Adaptive Multiscale Modeling
The Vision of Einstein
E Eiin nst ste eiin n:: “ “T Th he e m mo od de ell u use sed d s sh ho ou ulld d b be e
tth he e si sim mp plle est st o on ne e p po os ssi sib blle e,, b bu utt n no ott
simpler.” • Model transition schemes
– Pollution errors at the interface for continuous-
Adaptive Multiscale Modeling: “Start with a
Adaptive Multiscale Modeling: “Start with a
continuous and continuous-discrete transitions
s siim mp plle er r m mo od de ell,, b ba ase sed d o on n a a si sin ng glle e sca scalle e a an nd d
– Mathematically consistent discrete-continuum transition
u un nco cou up plle ed d p ph hy ys siic ca all p pr ro oce ces sse ses, s, a an nd d tth he en n a ad da ap pttiive velly y
Is stochastic modeling required?
introduce additional scales to permit coupled
introduce additional scales to permit coupled
• Probability error estimators
m mu ullttiisca scalle e--m mu ullttiip ph hy ys siics cs co con nsi sid de erra attiio on ns, s, w wh he en ne eve verr
a an nd d w wh he er re eve ver r tth he es se e a ar re e n ne ee ed de ed d,, u un nttiill tth he e s siim mp plle es stt • Multiscale sensitivity analysis
possible model is obtained.”
possible model is obtained.”
63 64
Stochastic Nature of Multiscale Problem
Other related topics
•Physical uncertainties (loads, domain, material properties)
•Statistical uncertainties (amount of data available,
probability fields such as correlations)
• Brain autoregulation modelling
• Cardiac Modelling
•Model uncertainties (mathematical modeling of physical
• 3D Anatomical Human
• Physiological Human
65 66
SSIP 2009 ©UCL 11Medical Image Processing July 2009
Andrew Todd-Pokropek ©UCL
Autoregulation in brain: multiscale modelling
Imaging of blood filled tubes in intralipid by
photoacoustic imaging (P. Beard UCL)
6 62 2µ µm m
3 30 00 0µ µm m
PaCO activation
chemistry transport
14mm x 14mm
• Excitation: 800nm (6.7mJ/cm ); pulse duration=8ns
67 • Lateral scanning step-size: 140µm 68
Cardiac Modelling
X-ray Phase Imaging (Ian Robinson)
F Pfeiffer & Christian David,
Paul Scherrer Institut
Jun Li et al., J. Anat 202, 463
F Pfeiffer (2 & 00 C 3h )ristian David,
Ack Noble/ Hunter
Ack Harvard Ack INRIA
Paul Scherrer Institut
69 70
How do you evaluate-
How to evaluate / validate
measures and statistics?
• True positive and false positive rates
• Testing on simulated data
• Probabilistic distances
(in silico)
– Hausdorf distance (largest difference)
• Results on phantoms
• Volume and volume overlap
• Results on clinical data
• Interclass correlation coefficient
(clinical trials)
– Williams (modified) index
• ??? How to measure false negatives???
• Valmet software
71 72
SSIP 2009 ©UCL 12Medical Image Processing July 2009
Andrew Todd-Pokropek ©UCL
But: How do you evaluate? What is optimal
• True positive and false positive rates
• It normally depends on the task
• Probabilistic distances
– Hausdorf distance (largest difference)
• Volume and volume overlap
• Interclass correlation coefficient
• Hence use of (very time consuming) techniques
– Williams (modified) index
• ??? How to measure false negatives???
such as Receiver operating characteristic (ROC)
• Valmet software
73 74
• Medical image/ signal
The Future of Medicine?
analysis is a very useful
tool for studying
• Preventive medicine physiological processes
– Including environment in man (and animal
• Personalised Medicine models)
– Instrumentation development
– Using the –omics (genomics/ proteomics …)
– Development of methodology
• Keyhole/ robotic surgery
– Applied to relevant problems
– Implanted devices
– Well engineered
• Nanotechnology – Widely used (technology
– Biolab
– Lots of exciting new
– MEMS (micro electro-mechanical systems)
– Lab-on-a-chip micro-arrays and diagnosis
– Including
– Drug production
• Complex Systems
– Mathematical biology
– Modelling and simulation
75 76
The End?
In conclusion
Working in multidisciplinary teams
77 78
SSIP 2009 ©UCL 13