Initiative X: Adaptive Sensing and Sensory Biomimetics

skoptsytruculentAI and Robotics

Nov 15, 2013 (3 years and 6 months ago)


Initiative X: Adaptive Sensing and Sensory Biomimetics

The Goal

We are approaching an exciting era in which dysfunctional human sensory systems will
be routinely augmented, repaired or replaced using advanced technological solutions. The
requisite technol
ogies will have the potential for improving the lives of millions of
individuals worldwide. Related technologies are also on the horizon for extending
human sensory capabilities beyond their normal range. Augmented sensing systems in the
form of wearable
devices could potentially enhance the range and acuity of the basic
senses, as well as provide completely novel sensing capabilities from the human
perspective, such as infrared, ultrasonic or electric field sensing. Finally, as computers
and other human e
ngineered devices, such as nano

and micro
robots, become more
responsive to their environments, these devices will also require increasingly
sophisticated sensing capabilities.
All of these systems will need to function in the real
world, and hence face d
esign and performance constraints that are similar to those that
evolution has been grappling with for millions of years. One of the key insights from
biology is that sensory systems are adaptive. They constantly tune their own processing
properties over
multiple time scales in order to maintain and enhance the flow of useful
information to brain.

Research and development efforts in this area are necessarily cross
involving the interplay of basic sensory neurobiology, biomedical engineering,

electrical engineering, among others. On the biological side, it is necessary to develop an
generalized understanding of the adaptive information processing strategies used across
multiple sensory systems. This involves the careful and systematic rev
engineering of
biological systems, with the aim of developing a set of guiding principles for the forward
engineering of artificial sensory devices. On the engineering side, there are multiple
challenges to be addressed, from establishing long
reliable interfaces between nerve
cells and silicon devices, tissue biocompatibility, low power signal processing and real
time computation. The Beckman Institute provides a unique opportunity for developing
and testing the next generation of adaptive sens
ing devices. In addition, several potential
synergies exist with other BI strategic initiatives, including Healthy Minds (e.g.
overcoming sensory deficits that accompany aging), Human and Machine Vision (visual
prosthetics, adaptive machine vision), and 3D

Micro and Nano Assemblies (sensing and
control architectures for autonomous, mobile micro devices).

Challenges and Opportunities

Knowledge of the auditory system has led (at Illinois) to a highly successful adaptive
(intelligent) hearing aid concept. Ef
forts are now redirected to a more challenging
problem, that of enhancing the spatial (and thus frequency) resolution of cochlear
implants; principles derived will be applicable to other prosthetic, sensory and motor
substitution devices, such as the visua
l system, deep brain stimulation, and brain machine
interfaces. … Knowledge of the … system provides principles for the development of an
autofocusing camera. … Electric fish … There is significant value in developing the in
vivo and in vitro model syst
ems that provide fundamental understanding of our baseline
science and technology.

Specific Goals

Formulate a systematic, clearly articulated set of general design principles for adaptive
sensing systems based on reverse
engineering analysis of biologic
al systems. Develop a
simulation environment for testing these principles on artificial agents in virtual
Adaptive sensory processing involves not only the neural circuitry of the
brain, but also the dynamic couplings between brain, body, an
d environment. Non
stationary statistical properties of natural sensory scenes shape the need for adaptive
sensory processing. General principles can best be tested and refine using a modeling
and simulation environment in which the dynamic structure and

complexity of the
environment can be systematically controlled.

Current status:
Several NeuroTech faculty are actively involved in ‘reverse
engineering’ biological sensory systems across several modalities. Two
computational neuroscientists within the gr
oup have extensive neural modeling and
simulation experience. The group’s work has elucidated several important
mechanisms and principles of adaptive sensing, including insights into adaptive noise
suppression and gain control, source localization, and mul
tisensory integration. A
comprehensive analysis of information processing demands across multiple active
sensing modalities has recently been completed.

year goals:
To formalize a set of general (modality
independent) design
principles for adaptive
sensing systems as well as modality
specific extensions. To
develop an interactive modeling and simulation environment for testing these
principles on adaptive sensing agents in virtual environments. To promote the transfer
and testing of these principles
to sensory prostheses and intelligent robots operating in
real world environments.



Who’s responsible:

NeuroTech group

Develop the science and technology required to create high
resolution biomolecular
lear implants (utilizing spatially and temporally targeted drug delivery).

Current status:
Great strides have been made to screen and identify signaling
molecules responsible for neuron survival and neurite outgrowth
in vitro
Considerable work (3D cultu
in vivo

validation of potency of molecules and
guidance of neurite outgrowth) remains to realize our goal.

year goals:
Within 5 years, the goals are to develop novel drug delivery
techniques and to validate the potency of several signaling molecu
les in promotion
and guidance of neurite outgrowth
in vivo
. Also to develop model systems, both
animal and 2D/3D in vitro, for evaluation and understanding of localization of and
cellular response mechanisms to the biomolecules. Further, to have attracted
significant federal funding for this effort.


The Beckman Institute and the UIUC campus have the capital equipment
needed for this work, including many of the molecular biological instruments and
techniques, and neural
interface fabrication tech
niques. A key faculty hire in neural
prosthetic engineering would greatly enhance the project. The short
term critical need
is pilot project support so that well
trained students and staff may work to meet the
five year goals.

Who’s responsible:

euroTech group, Carle Hospital, OTHER BECKMAN GROUPS?

Develop a biologically inspired intelligent sensing device with multiple adaptive sensing

[extension of Tom’s self
aiming camera system; need input on this section]


Current status:


input from Tom’s self
aiming camera and Mark’s biorobotics efforts

year goals:

need input


need input

Who’s responsible:

NeuroTech and Artificial Intelligence groups, OTHER BECKMAN GROUPS?