Physical Fluctuomatics 1st Review of probabilistic information processing

odecrackAI and Robotics

Oct 29, 2013 (3 years and 9 months ago)

59 views

Physical Fuctuomatics (Tohoku
University)

1

Physical
Fluctuomatics

1st Review of probabilistic information processing

Kazuyuki Tanaka

Graduate School of Information Sciences

kazu@smapip.is.tohoku.ac.jp

http://www.smapip.is.tohoku.ac.jp/~kazu/

Webpage:

http://www.smapip.is.tohoku.ac.jp/~
kazu/PhysicalFluctuomatics/2013/

Physical Fuctuomatics (Tohoku
University)

2

Textbooks

Kazuyuki Tanaka:
Introduction

of Image
Processing by Probabilistic Models, Morikita
Publishing Co., Ltd., 2006 (in Japanese) .

Kazuyuki Tanaka: Mathematics of Statistical
Inference by Bayesian Network, Corona
Publishing Co., Ltd., 2009 (in Japanese).

Physical Fuctuomatics (Tohoku
University)

3

References of the present lecture

K. Tanaka: Statistical
-
mechanical approach to image
processing (Topical Review), Journal of Physics A:
Mathematical and General, vol.35, no.37, pp.R81
-
R150, 2002.

Y. Kabashima and D. Saad: Statistical mechanics of

low
-
density parity
-
check codes (Topical Review), J. Phys. A, vol.37,
no.6, pp.R1
-
R43, 2004.

H. Nishimori: Statistical Physics of Spin Glasses and
Information Processing,
---
An Introduction, Oxford University
Press, 2001.

M. Opper and D. Saad D (eds): Advanced Mean Field Methods
---

Theory and Practice, MIT Press, 2001.

C. M. Bishop: Pattern Recognition and Machine Learning,
Springer, 2006.

M. J. Wainwright and M. I. Jordan: Graphical Models,
Exponential Families, and Variational Inference, now
Publishing Inc, 2008.

M. Mezard, A. Montanari: Information, Physics, and
Computation, Oxford University Press, 2009.

Physical Fuctuomatics (Tohoku
University)

4

Benefit of Information &
Communications Technology

Ubiquitous Computing

Ubiquitous Internet

Benefit of Information & Communications Technology

Demand for Intelligence

It cannot be satisfied only with it being only cheap
and being quick.

Physical Fuctuomatics (Tohoku
University)

5

Field of Information Processing

Information processing according to theories

Inference from propositions

Realization by progress of computational processing capacity

Information processing in real world

Diversity of reason in phenomenon

Compete data is not necessarily obtained.

It is difficult to extract and select some important
information from a lot of data.

Uncertainty caused by the gap of knowing simply and
understanding actually.

We hope to deal successfully with such uncertainty.

Information processing for numerical calculations

Definite Procedure has been given for each calculation.

Physical Fuctuomatics (Tohoku
University)

6

Computer for next generations

Required Capacity

Capability to sympathize with a user

Kn潷汥d来)

Capability to put failure and experience to account in
the next chance

Le慲n楮g


How should we deal successfully with the uncertainty caused
by the gap of knowing simply and understanding actually?

Formulation of knowledge and uncertainty

Realization of information processing data with
uncertainty

Physical Fuctuomatics (Tohoku
University)

7

Computational model for information
processing in data with uncertainty

Probabilistic Inference

Probabilistic model

with graphical structure



B慹es楡ietw潲k


Medical diagnosis

Failure diagnosis


Risk Management

Probabilistic information processing can give us unexpected
capacity in
a system constructed from many cooperating
elements with randomness
.


Inference system for data with uncertainty

modeling

Node is random variable.

Arrow is conditional probability.

Mathematical expression of uncertainty

=>Probability and Statistics

Graph with cycles

Important aspect

Physical Fuctuomatics (Tohoku
University)

8

Computational Model

for Probabilistic Information Processing

Probabilistic
Information Processing

Probabilistic Model

Bayes Formula

Algorithm

Monte Carlo Method

Markov Chain Monte Carlo Method

Randomized Algorithm

Genetic Algorithm

Approximate Method

Belief Propagation

Mean Field Method

Randomness and
Approximation

Physical Fuctuomatics (Tohoku
University)

9

Probabilistic Image Processing

Noise Reduction by
Probabilistic Image
Processing

K. Tanaka: J. Phys. A, vol.35, 2002.

A. S. Willsky: Proceedings of IEEE, vol.90, 2002.

192

202

190

202

219

120

100

218

110

192

202

190

202

173

120

100

218

110

Modeling of Probabilistic Image Processing based on Conventional Filters

Markov
Random
Field
Model

Probabilistic Image
Processing

The elements of such a
digital array are called pixels.

At each point, the intensity
of light is represented as an
integer number or a real
number in the digital image
data.

Algorithm

Conventional Filter

Physical Fuctuomatics (Tohoku
University)

10

Probabilistic Image Processing

Degraded Image (Gaussian Noise


P牯扡扩lis瑩c Imag攠P牯捥csi湧

Lowpass Filter

Wiener Filter

Median Filter

MSE:520

MSE: 2137

MSE:860

MSE:767

MSE:1040

K. Tanaka: J. Phys. A, vol.35, 2002.

A. S. Willsky: Proceedings of IEEE, vol.90, 2002.

Physical Fuctuomatics (Tohoku
University)

11

Error Correcting Code

Y. Kabashima and D. Saad: J. Phys. A, vol.37, 2004.

High Performance Decoding Algorithm

010

000001111100000

00
1
001
0
1110000
1

0 1 0

code

010

error

decode

Parity Check Code

Turbo Code, Low Density Parity Check (LDPC) Code

majority rule

Error Correcting Codes

14 January, 2010

Hokkaido University GCOE Tutorial
(Sapporo


12

Error Correcting Codes and

Belief Propagation

1

1

0

1

0

0

Received Word

Code Word

Binary Symmetric

Channel

14 January, 2010

Hokkaido University GCOE Tutorial
(Sapporo


13

Error Correcting Codes and

Belief Propagation

Fundamental Concept for Turbo Codes and LDPC Codes

Physical Fuctuomatics (Tohoku
University)

14

CDMA Multiuser Detectors in Mobile Phone Communication

Relationship between mobile phone communication
and spin glass theory

T. Tanaka, IEEE Trans. on Information Theory, vol.48, 2002



Signals of

User A



Spreading Code
Sequence

Wireless
Communication




Received Data

Decode



Spreading Code
Sequence

Probabilistic model for
decoding can be
expressed in terms of a
physical model for spin
glass phenomena

Noise

Coded Signals

of Other Users

Coded
Signals of
User A

Physical Fuctuomatics (Tohoku
University)

15

Artificial Intelligence

Bayesian Network

Probabilistic inference system

Practical algorithms

by means of belief
propagation

J. Pearl: Probabilistic Reasoning in Intelligent Systems: Networks of
Plausible Inference (Morgan Kaufmann, 1988).

Physical Fuctuomatics (Tohoku
University)

16

Main Interests

Information Processing:


Data

Physics:


Material,


Natural Phenomena

System of a lot of elements with mutual relation

Common Concept between Information Sciences and Physics

Material

Molecule

Materials are constructed from a lot of molecules.

Molecules have interactions of each other.


,1

101101
110001

01001110111010
10001111100001
10000101000000
11101010111010
1010

Bit

Data

Data is constructed from many bits

A sequence is formed by deciding the arrangement of bits.

A lot of elements have mutual relation of each other

Some physical concepts
in Physical models are
useful for the design of
computational models in
probabilistic
information processing.

Physical Fuctuomatics (Tohoku
University)

17

Horizon of Computation

in Probabilistic Information Processing

Compensation of expressing uncertainty

using probability and statistics


It must be calculated by taking account of both events
with high probability and events with low probability.

Computational Complexity

It is expected to break throw the computational
complexity by introducing approximation algorithms.

Physical Fuctuomatics (Tohoku
University)

18

What is an important point in
computational complexity?

How should we treat the
calculation of the
summation over 2
N

configuration?

N

fold loops

If it takes 1 second in the case of
N
=10, it takes 17 minutes in
N
=20, 12 days in

N
=30 and 34
years in
N
=40.

Physical Fuctuomatics (Tohoku
University)

19

Why is a physical viewpoint effective in probabilistic
information processing?

Matrials are constructed from a lot of molecules.

(10
23

molecules exist in 1 mol.)

Molecules have intermolecular forces

of each other

Theoretical physicists always have to
treat such multiple summation.

Development of Approximate Methods

Probabilistic information processing is also usually reduced
to multiple summations or integrations.

Application of physical approximate methods

to probabilistic information processing

Physical Fuctuomatics (Tohoku
University)

20

Academic
Circulation


Academic
Circulation


Academic Circulation between
Physics and Information Sciences

Physics

Information Sciences

Understanding and prediction
of properties of materials and
natural phenomena

Extraction and processing of
information in data

Common Concept

Statistical
Mechanical
Informatics

Probabilistic
Information
Processing

Statistical
Sciences

Physical Fuctuomatics (Tohoku
University)

21

Summary of the present lecture

Probabilistic information processing

Examples of probabilistic information processing

Common concept in physics and information sciences

Application of physical modeling and approximations

Future Lectures

Fundamental theory of probability and statistics

Linear model

Graphical model

.

.

.