Computer vision for non-visual

unclesamnorweiganΤεχνίτη Νοημοσύνη και Ρομποτική

18 Οκτ 2013 (πριν από 4 χρόνια και 8 μήνες)

111 εμφανίσεις

Computer vision for non
spectral regimes and non
traditional applications

Rama Chellappa


Opening remarks

visual sensors

Sensors in visual regimes are only a small % of
the possible sensors

Infrared, hyperspectral, LADAR, SAR, FOPEN
SAR, mmWave, polarimetric

Provide thermal signatures, material properties,
3D, all weather/all day coverage, looking behind
trees, walls, …

Sensor physics is important.

Low signal to clutter ratio.

Characterization of image statistics driven by
sensor physics

Problems addressed using
hyperspectral images

Sensor design

Material classification/segmentation

Anomaly detection

More than two decades of work

ARO MURI (2002
2007) on the science of land
based target

DARPA DDB program, NGA

Glenn Healey: pioneer in HSI and computer vision

Larry Wolf, Equinox Corporation

Unmixing of pixels (Remote sensing)

Compressive sensing possibilities (SPC)

Numerous books, papers, conferences…

Estimating object reflectance


Radio transfer function

irradiance to the camera

reflectance at location (x,y)

main light source (e.g. sun light)

ambient light source coming from all directions,
assuming constant intensity for all directions

fraction of unblocked sky from (x,y) view

geometry factor

Nguyen and Chellappa,

CVPR 2010 Workshop on

Beyond visible spectrum


Tracking radiance

Reflectance tracking

Robust against illumination, abrupt motion

Capable of recovering after losing track


Detection of land mines

Statistical models of clutter (non

Optimal detection methods

pixel detection methods

Detection of disturbed earth

Detection of mass graves in the Balkans

Broadwater and Chellappa, PAMI 2007, SP, 2010

Radar images

Synthetic aperture radar (SAR), foliage
penetrating SAR (FOPEN SAR), …

Speckle noise
(random fluctuations in the return signal from an object that is no
bigger than a single image
processing element)

Shape from shading

radar clinometry (USGS, Frankot, Chellappa, AI
Journal, 1990)

3D from interferometric SAR (Zebker and Goldstein), stereo SAR

Object detection, indexing recognition (DARPA programs MSTAR, SAIP,

MSTAR program (feature extraction, indexing, prediction, recognition)

object recognition framework

Features can correspond to scatterers (supported by physics)


Low signal to clutter ratio (tree trunks produce stronger returns than objects)

Symmetric alpha
stable noise

Global hawk, TESAR sensors

R.T. Frankot and R. Chellappa, Estimation of Surface Topography from SAR Imagery Using Shape

from Shading Techniques, in Physics
Based Vision: Shape Recovery, (eds.), L.B. Wolff, S.A. Shafer and

G.E. Healey, Jones and Bartlett Publishers, Boston, MA, pp. 62
101, 1992.

LADAR images

Took off in the mid eighties

From machine vision to outdoor conditions

Feature (step, crease edges, surfaces,) extraction,
matching and recognition (Hypothesis

Fundamental forms I and II (Besl and Jain)

Doppler LADAR (2km) for ATR

Better resolution at longer ranges


R. Chellappa, S. Der and E.J.M. Rignot, Statistical Characterization of FLIR, LADAR

and SAR Imagery, in Statistics and Images, K.V Mardia, (ed.), Carfax publishers,

Oxfordshire, U.K., pp. 273
312, 1994

Opening remarks


Road following, lane tracking and other
automotive applications

Dickmanns, Pomereleu, DOT, FHWA

Computer vision for the blind

Navigation, face/expression recognition

Analysis of Schlieren images

Detection of oblique structures such as shock waves
and shear layers

Understanding bee dances

Industrial inspection

Vision for Schlieren data reduction

Schlieren Imaging

Aerodynamic visualization technique in
wind tunnels
> long established

Capture density gradients in supersonic

Shock waves, shear layers and turbulent

High speed imaging
> data deluge

Images complete the picture

Offer what surface measurements may

Are we taking advantage of the data

Can vision extend analysis

Additional insight to flow unsteadiness

Desire for automation

Removal of human subjectivity

Recast as a segmentation/feature
extraction problem

Extraction of oblique structures

Oblique flow structures ubiquitous

Physically meaningful

Bilateral filter
> isoperimetric cut

Shock wave and shear

Canny & Hough transform


Length & quantifiable bounds

Location enforced from labeling

Scale implementation for

Hough transform binning

Convergence from two Hough grids

Success of automation 92

Outer shock motion history

Recommended for Publication
in AIAA Journal

Vision: viable analysis tool

Waggle dance

Orientation of waggle axis

of Food source.(with respect to sun).

Intensity of waggle dance

Sweetness of food source.

Frequency of waggle

Distance of
food source.

Parameters of interest in the waggle

Waggle Axis : Average orientation
of Thorax during Waggle.

Duration of Waggle : Number of
frames of waggle in each segment
of the dance.

Anatomy/behavior modeling


Three major body parts; each body part
modeled as an ellipse.

Anatomical modeling ensures

Physical limits of body parts are
consistent; accounts for structural
limitations and correlation among
orientation of body parts

Insects move in the direction of their head.

Veeraraghavan, Chellappa and Srinivasn, IEEE TPAMI, March 2008

Insects display very specific


Modeling behavior explicitly

Tracking performance

Behavioral understanding

Position tracking and
behavioral analysis in a
unified framework.


Detect Frames of Waggle Dance by
looking at

Rate of change of Abdomen

Average absolute motion of center of
abdomen in the direction
perpendicular to the axis of the bee.

Parameters of Interest in the Waggle

Waggle Axis : Average orientation of
Thorax during Waggle.

Duration of Waggle : Number of
frames of waggle in each segment of
the dance.

Looking for a screw amid screws

My advice to the young ones

Look for collaborations outside EE, CS

Helps with multidisciplinary credentials

Will help with winning MURIs: best source of

There are top Transactions and journals that
accept these papers!

Computer vision presents immense
opportunities outside traditional areas.