towards a zoomable cell

natureplaygroundAI and Robotics

Nov 14, 2013 (3 years and 9 months ago)

81 views

towards a
zoomable

cell

QuickTime™ and a
TIFF (LZW) decompressor
are needed to see this picture.
abstract cell

natural
coordinate

system

Data

QuickTime™ and a
TIFF (LZW) decompressor
are needed to see this picture.
>48.000 3D Protein

Structures from PDB

QuickTime™ and a
TIFF (LZW) decompressor
are needed to see this picture.
QuickTime™ and a
TIFF (LZW) decompressor
are needed to see this picture.
?

A

I

H

G

F

B

C

D

E

>200.000 Images from

scientific publications

1

Computer Graphics

and


Visualization

TECHNISCHE

UNIVERSITÄT

DRESDEN

Zoomable Cell

Stefan Gumhold Michael Schröder


Norbert Blenn


Anne Tuukkanen



Marcel Spehr


Matthias Reimann


Computer Graphics

and


Visualization

Zoomable Cell, SPP 1335, Kickoff Meeting, 9.12.08,
Dagstuhl

3

Goals


Data analysis


Natural coordinate
system (NCS)


Mapping of images from
literature to NCS


3D models of complexes
in NCS


Visualization


aggregation of images,
volumes and 3D models


Rendering across scale
from 10

m to 1Å


Natural adjustment of
visualization parameters
with dynamic labeling


HCI


support for Virtual
Reality environments


speech control and input
device development


flexible navigation


community support
through web integration


Impact


Interface life scientists
„from different scales“


data aggregation and
analysis platform


production of illustrative
materials

Computer Graphics

and


Visualization

Zoomable Cell, SPP 1335, Kickoff Meeting, 9.12.08,
Dagstuhl

4

Human Cells

New Problems

Several
different

instances of the same
t
y
pe

each instance is
flexible

cells are treated badly
before imaging

very different imaging
modalities are used


Deformation Framework

Computer Graphics

and


Visualization

Zoomable Cell, SPP 1335, Kickoff Meeting, 9.12.08,
Dagstuhl

5

Various Data Types


cell


nucleus


pore


complexes


proteins

primitives, smooth surfaces

implicit surfaces

height fields

images: 2D, 3D, perspective

images: 2D, 3D, perspective

Computer Graphics

and


Visualization

Zoomable Cell, SPP 1335, Kickoff Meeting, 9.12.08,
Dagstuhl

6

Data Augmentation


define reference models


for each dataset


scale


imaging modality


features


points


curves


regions


labeling of features


for pairs of datasets


feature mapping


additional alignment
information

nucleolus

envelop

pore

Computer Graphics

and


Visualization

Zoomable Cell, SPP 1335, Kickoff Meeting, 9.12.08,
Dagstuhl

7

Integration of Datasets

Segmentation

Feature

Detection

Labeling

non
-
rigid

Registration

Computer Graphics

and


Visualization

Zoomable Cell, SPP 1335, Kickoff Meeting, 9.12.08,
Dagstuhl

8

Deformation

reference model

Computer Graphics

and


Visualization

Zoomable Cell, SPP 1335, Kickoff Meeting, 9.12.08,
Dagstuhl

9

Plan to a Solution


start with fully interactive tools

add automation step by step with full
interactivity for corrections

find features that persist over different scales

develop learning based segmentation
approaches

exploit mutual information to register datasets
of different dimension and modality

Computer Graphics

and


Visualization

Zoomable Cell, SPP 1335, Kickoff Meeting, 9.12.08,
Dagstuhl

10

Visualization Engine


protein structures


primitive splatting


tubes, surfaces


deferred shading


sorting based
transparency


3d surface models


LOD based rendering


depth peeling based
transparency


Images & Volumes


volume rendering


compression


transfer functions

Computer Graphics

and


Visualization

Example Images

Computer Graphics

and


Visualization

Quer
y
Based

Exploration
of

Images

Available image information


Expert labeled text
(categorical)


Unstructured information of
related text (textual)


Inherent image features
(abstract description of
image appearance)

More reliable and
structured

Less reliable and
structured

Navigation/Exploration


Around 100.000 images currently available to us


Even with automatic analysis one needs supporting browsing techniques


If

we have features that measure appropriate image similarities:


Hierarchical Browsing


Fish
-
Eye View

Hierarchical Browsing

Fish
-
Eye View

Methods to structure image
data set


By hand


Automatic analysis (off
-
the
-
shelf methods)


Unsupervised (Clustering)


Supervised (Multiclass Support Vector machines)


Need for appropriate problem oriented feature set

Image Feature Definition


Vast numbers of image descriptors are available


Need for general purpose image descriptors because of wide variety of
image origins


Standardized Multimedia content description (MPEG
-
7)





Class information from Image
Features

1.
Definition of semantic classes (assisted and manually, Gene Ontology
labels)

2.
Relation of abstract image descriptors to semantic classes (training,
learning)

3.
Evaluation of generalization ability

GoImage




Semantic Image Search


Comprehensive protein
-
interaction mapping
projects underway


What is the cost of completing an interactome
map and what is the best strategy for
minimizing the cost?


How can quality and coverage of interaction
data be maximized?

GoImage




Semantic Image Search

GoImage




Semantic Image Search

Refinement of a search for
membranes through selecting
nuclear envelope p.a.