Physical Fuctuomatics (Tohoku
University)
1
Physical
Fluctuomatics
1st Review of probabilistic information processing
Kazuyuki Tanaka
Graduate School of Information Sciences
kazu@smapip.is.tohoku.ac.jp
http://www.smapip.is.tohoku.ac.jp/~kazu/
Webpage:
http://www.smapip.is.tohoku.ac.jp/~
kazu/PhysicalFluctuomatics/2013/
Physical Fuctuomatics (Tohoku
University)
2
Textbooks
Kazuyuki Tanaka:
Introduction
of Image
Processing by Probabilistic Models, Morikita
Publishing Co., Ltd., 2006 (in Japanese) .
Kazuyuki Tanaka: Mathematics of Statistical
Inference by Bayesian Network, Corona
Publishing Co., Ltd., 2009 (in Japanese).
Physical Fuctuomatics (Tohoku
University)
3
References of the present lecture
K. Tanaka: Statistical

mechanical approach to image
processing (Topical Review), Journal of Physics A:
Mathematical and General, vol.35, no.37, pp.R81

R150, 2002.
Y. Kabashima and D. Saad: Statistical mechanics of
low

density parity

check codes (Topical Review), J. Phys. A, vol.37,
no.6, pp.R1

R43, 2004.
H. Nishimori: Statistical Physics of Spin Glasses and
Information Processing,

An Introduction, Oxford University
Press, 2001.
M. Opper and D. Saad D (eds): Advanced Mean Field Methods

Theory and Practice, MIT Press, 2001.
C. M. Bishop: Pattern Recognition and Machine Learning,
Springer, 2006.
M. J. Wainwright and M. I. Jordan: Graphical Models,
Exponential Families, and Variational Inference, now
Publishing Inc, 2008.
M. Mezard, A. Montanari: Information, Physics, and
Computation, Oxford University Press, 2009.
Physical Fuctuomatics (Tohoku
University)
4
Benefit of Information &
Communications Technology
Ubiquitous Computing
Ubiquitous Internet
Benefit of Information & Communications Technology
Demand for Intelligence
It cannot be satisfied only with it being only cheap
and being quick.
Physical Fuctuomatics (Tohoku
University)
5
Field of Information Processing
Information processing according to theories
Inference from propositions
Realization by progress of computational processing capacity
Information processing in real world
Diversity of reason in phenomenon
Compete data is not necessarily obtained.
It is difficult to extract and select some important
information from a lot of data.
Uncertainty caused by the gap of knowing simply and
understanding actually.
We hope to deal successfully with such uncertainty.
Information processing for numerical calculations
Definite Procedure has been given for each calculation.
Physical Fuctuomatics (Tohoku
University)
6
Computer for next generations
Required Capacity
Capability to sympathize with a user
（
Kn潷汥d来)
Capability to put failure and experience to account in
the next chance
（
Le慲n楮g
）
How should we deal successfully with the uncertainty caused
by the gap of knowing simply and understanding actually?
Formulation of knowledge and uncertainty
Realization of information processing data with
uncertainty
Physical Fuctuomatics (Tohoku
University)
7
Computational model for information
processing in data with uncertainty
Probabilistic Inference
Probabilistic model
with graphical structure
（
B慹es楡ietw潲k
）
Medical diagnosis
Failure diagnosis
Risk Management
Probabilistic information processing can give us unexpected
capacity in
a system constructed from many cooperating
elements with randomness
.
Inference system for data with uncertainty
modeling
Node is random variable.
Arrow is conditional probability.
Mathematical expression of uncertainty
=>Probability and Statistics
Graph with cycles
Important aspect
Physical Fuctuomatics (Tohoku
University)
8
Computational Model
for Probabilistic Information Processing
Probabilistic
Information Processing
Probabilistic Model
Bayes Formula
Algorithm
Monte Carlo Method
Markov Chain Monte Carlo Method
Randomized Algorithm
Genetic Algorithm
Approximate Method
Belief Propagation
Mean Field Method
Randomness and
Approximation
Physical Fuctuomatics (Tohoku
University)
9
Probabilistic Image Processing
Noise Reduction by
Probabilistic Image
Processing
K. Tanaka: J. Phys. A, vol.35, 2002.
A. S. Willsky: Proceedings of IEEE, vol.90, 2002.
192
202
190
202
219
120
100
218
110
192
202
190
202
173
120
100
218
110
Modeling of Probabilistic Image Processing based on Conventional Filters
Markov
Random
Field
Model
Probabilistic Image
Processing
The elements of such a
digital array are called pixels.
At each point, the intensity
of light is represented as an
integer number or a real
number in the digital image
data.
Algorithm
Conventional Filter
Physical Fuctuomatics (Tohoku
University)
10
Probabilistic Image Processing
Degraded Image (Gaussian Noise
）
P牯扡扩lis瑩c Imag攠P牯捥csi湧
Lowpass Filter
Wiener Filter
Median Filter
MSE:520
MSE: 2137
MSE:860
MSE:767
MSE:1040
K. Tanaka: J. Phys. A, vol.35, 2002.
A. S. Willsky: Proceedings of IEEE, vol.90, 2002.
Physical Fuctuomatics (Tohoku
University)
11
Error Correcting Code
Y. Kabashima and D. Saad: J. Phys. A, vol.37, 2004.
High Performance Decoding Algorithm
010
000001111100000
00
1
001
0
1110000
1
0 1 0
code
010
error
decode
Parity Check Code
Turbo Code, Low Density Parity Check (LDPC) Code
majority rule
Error Correcting Codes
14 January, 2010
Hokkaido University GCOE Tutorial
(Sapporo
）
12
Error Correcting Codes and
Belief Propagation
1
1
0
1
0
0
Received Word
Code Word
Binary Symmetric
Channel
14 January, 2010
Hokkaido University GCOE Tutorial
(Sapporo
）
13
Error Correcting Codes and
Belief Propagation
Fundamental Concept for Turbo Codes and LDPC Codes
Physical Fuctuomatics (Tohoku
University)
14
CDMA Multiuser Detectors in Mobile Phone Communication
Relationship between mobile phone communication
and spin glass theory
T. Tanaka, IEEE Trans. on Information Theory, vol.48, 2002
Signals of
User A
Spreading Code
Sequence
Wireless
Communication
Received Data
Decode
Spreading Code
Sequence
Probabilistic model for
decoding can be
expressed in terms of a
physical model for spin
glass phenomena
Noise
Coded Signals
of Other Users
Coded
Signals of
User A
Physical Fuctuomatics (Tohoku
University)
15
Artificial Intelligence
Bayesian Network
Probabilistic inference system
Practical algorithms
by means of belief
propagation
J. Pearl: Probabilistic Reasoning in Intelligent Systems: Networks of
Plausible Inference (Morgan Kaufmann, 1988).
Physical Fuctuomatics (Tohoku
University)
16
Main Interests
Information Processing:
Data
Physics:
Material,
Natural Phenomena
System of a lot of elements with mutual relation
Common Concept between Information Sciences and Physics
Material
Molecule
Materials are constructed from a lot of molecules.
Molecules have interactions of each other.
０
,1
101101
110001
01001110111010
10001111100001
10000101000000
11101010111010
1010
Bit
Data
Data is constructed from many bits
A sequence is formed by deciding the arrangement of bits.
A lot of elements have mutual relation of each other
Some physical concepts
in Physical models are
useful for the design of
computational models in
probabilistic
information processing.
Physical Fuctuomatics (Tohoku
University)
17
Horizon of Computation
in Probabilistic Information Processing
Compensation of expressing uncertainty
using probability and statistics
It must be calculated by taking account of both events
with high probability and events with low probability.
Computational Complexity
It is expected to break throw the computational
complexity by introducing approximation algorithms.
Physical Fuctuomatics (Tohoku
University)
18
What is an important point in
computational complexity?
How should we treat the
calculation of the
summation over 2
N
configuration?
N
fold loops
If it takes 1 second in the case of
N
=10, it takes 17 minutes in
N
=20, 12 days in
N
=30 and 34
years in
N
=40.
Physical Fuctuomatics (Tohoku
University)
19
Why is a physical viewpoint effective in probabilistic
information processing?
Matrials are constructed from a lot of molecules.
(10
23
molecules exist in 1 mol.)
Molecules have intermolecular forces
of each other
Theoretical physicists always have to
treat such multiple summation.
Development of Approximate Methods
Probabilistic information processing is also usually reduced
to multiple summations or integrations.
Application of physical approximate methods
to probabilistic information processing
Physical Fuctuomatics (Tohoku
University)
20
Academic
Circulation
Academic
Circulation
Academic Circulation between
Physics and Information Sciences
Physics
Information Sciences
Understanding and prediction
of properties of materials and
natural phenomena
Extraction and processing of
information in data
Common Concept
Statistical
Mechanical
Informatics
Probabilistic
Information
Processing
Statistical
Sciences
Physical Fuctuomatics (Tohoku
University)
21
Summary of the present lecture
Probabilistic information processing
Examples of probabilistic information processing
Common concept in physics and information sciences
Application of physical modeling and approximations
Future Lectures
Fundamental theory of probability and statistics
Linear model
Graphical model
.
.
.
Comments 0
Log in to post a comment