glibdoadingΤεχνίτη Νοημοσύνη και Ρομποτική

20 Οκτ 2013 (πριν από 3 χρόνια και 7 μήνες)

78 εμφανίσεις


A.V. Gavrilov, V.M. Kangler

Novosibirsk state technical university, Novosibirsk, Russia, Email:


The principles of applicat
ion of Hopfield neural network for Data Analysis is proposed in this
report. Here the artificial neural networks are used for realization of associative memory representing
information contents of a relational database (DB). This approach is used for solve

the task of detection of
associative interrelations between of fields of Data Base and forecasting of values of its.


Critical competitive struggle, aspiration of the companies to receive profit from the investments in
information technologi
es, growth of number of the employees accepting the decisions, stimulated active
development of new area of computer science

detection of knowledge in databases (KDD

Discovery in Databases). Its basic assignment is automated search earlier un
known laws in databases
storing the information (for example, about activity of the companies), and use of the extracted knowledge
during acceptance of the decisions. With the help of methods KDD it is possible to reveal, for example,
structure of the cons
umers of the given goods to prevent frauds with credit cards or to predict change of a
situation in the financial market. The basic task of systems KDD is the realization of the analysis of the
data contained in databases, with the purpose of detection of
the latent or unobvious laws. Allocate five
standard types of laws, which allow to reveal automatic intellectual methods of the analysis and
representation of the data: association, sequence, classification, clustering and forecasting. The means of
on of knowledge in databases basically use five methods: an induction of associative rules; trees of
the decisions; K

nearest neighbours; neural networks; genetic algorithms. Their combination is
sometimes applied. The report is devoted to the decision o
f two tasks KDD, detection of associations and
forecasting, by artificial neural networks (ANNs).

The analysis of Data Base by Hopfield neural network.

Data Analysis in the given work is understood as a complex of the measures directed on
revealing of lat
ent laws in databases. Use of the artificial neural networks for the decision of a task is
considered which within the framework of tasks of the automated intellectual analysis of the data can be
treated as follows.


The qualitative solving of a task of fi
nd of values of fields of DB, associated to other fields with
known values.


The solving of a task of forecasting of values of a set of fields of DB with the given values of other
set of fields.

Having system similar to associative memory, and also formal

rules of carry of information
contents of an analyzed database in this memory, it is possible to solve this task. Thus is used the
association nature of such system and on values of some factor as "key" are taken is associative close
values of other facto
rs. One of possible ways to realize of such associative memory is to construct the
distributed dynamic system or network of discrete elements with attractors as the typical patterns
contained in DB [1]. Each such pattern will have the area of an at
traction, and any entry condition
representing any allowable pattorn, is obliged to get in one of its areas of an attraction. With current of
time during evolution this initial structure is transformed to the closest of structures

attractors, stored in
emory, namely in that, it belonged to area of which attraction. Hence, submitting on an input as the
entry condition for such distributed system some structure, we shall carry out its automatic recognition,
which will be parallel. In a role of such distrib
uted dynamic system it is offered to use artificial neural

The decision, offered us, in a general view looks as follows. Artificial neural network is trained on
records of an analyzed relational database. During training neural network becomes gn
oseological model
of the DB. The received thus model the user can use for forecasting and research of associative
connections latent in a database. As such model it is offered to use a network representing particular
realization of dynamic system with asso
ciative memory, proposed by Hopfield [2

The primary preparation given for the decision of the put task consists in representation of set of
the factors as the contents of a relational database. The structural relations are thus set and the task is
uced to definition of semantic affinity of contents of fields of the given structure. As binary artificial
neural network operate with binary vectors, the additional processing initial given is required with the
purpose of reduction to a kind represented b
y such vector.

The vector needed for artificial neural network turns out the concatination of binary subvectors
representing the fields of a relational database. Each field of a relational database can be referred to one
of two types: numerical or symboli
cal. The value of a symbolical field is brought in the dictionary,
appropriate each field. To raise efficiency of procedures of search of value in the dictionary and their
coding, contents of the dictionaries are ranging, the repeating values are excluded.

The value of
numerical fields are represented differently. A range of value of each of numerical fields (to numerical
fields those concern also, which contain dates) is broken into intervals, and each single value is
represented by an interval, by which i
t belongs. The point on a numerical axis can be presented by a
degeneration interval. The replacement of numerical value with intervals provides large flexibility of
representation. Analogue of the dictionaries for numerical fields is the list of intervals
. After the
dictionaries and sets of intervals were generated, they are exposed to coding. That is each value in the
dictionary or interval in a set the binary code representing subsequently this value or an interval in neural
network is put in conformity.

The dimension of a code (length of a code) for each dictionary and set (in
other words for each field) is defined by capacity of set of various values in the dictionary or set, and also
way of coding. The various variants of coding can essentially have an

effect for quality of work of
associative memory submitted by Hopfield neural network. The examples to that can be served by
experimental data received as a result of model modeling during preparation of the given work. The
formation of the dictionaries a
nd their coding is a preparatory grade level of neural network. Training is
carried out according to a Hebb rule [5].

Each record of a database is represented as a binary vector and participates in formation of
weight factors of connections between formal

neurons of a network. It is desirable to remove from training
repeating records of a database. As the repeated record of a vector in associative memory submitted by
neural network adverse has an effect for a power landscape of this network. Namely, the de
pth of a
power minimum appropriate to such vector is doubled. The deeper minimum usually has wider area of an
attraction, and it reduces areas of an attraction of the next minima and deforms the appropriate drawing
vectors. In a limiting case it can result

that in system there will be only one very deep minimum of energy,
the area of which attraction grasps all possible spin configurations. At system the persuasive idea " is
formed as though ", that, certainly, it is extremely undesirable.

There are variou
s versions of Hopfield networks with similar structure, but some differences are
in operation of them. Here are considered only binary condition neuron: + 1 and
1) Hopfield networks
with discrete functioning in time. Specifity uniting such network is asyn
chronous of operation of them.
That is as against synchronous functioning, with which the condition all neurons of a network is defined
simultaneously, the asynchronous functioning means in each particular moment of time an opportunity of
switching only on
e neuron. For modeling we deal with four algorithms of operation of Hopfield's neural


The determined algorithm;


Stochastic algorithm;


The determined algorithm with annealing imitation;


Stochastic algorithm with annealing imitation.

The proces
s of operation of neural network is not depended from mentioning algorithms is
multiiterative. Each of iterations includes two steps: a choice of the neuron
candidate, formation of a
condition of the chosen neuron
candidate. The difference of stochastic fu
nctioning from determined
consists in a method of a choice of the neuron
candidate. With stochastic

the candidate for switching is
neuron chosen casually with the help of the random
number generator. Thus, such situations are
possible, that during operat
ion the condition of any neurons may be never analyzed. With the determined
operation neurons become the candidates for change of the condition by way of following numbers. In
algorithms with annealing imitation the change of a condition of the neuron
idate carries probabilistic
character. If as a result of change of a condition on opposite the complete energy of a network will be
lowered, the condition of neuron varies on opposite. Differently neuron's condition varies on opposite with
the certain prob
ability dependent on parameter, named in "temperature" of a network simulating a level of
thermal noise. On the first iteration of operation of the neural network the initial value of temperature
(maximal value) is established, then gradually through some
of iterations the value of temperature
decreases. The iterative process stops, when temperature reaches final value (minimal value). If to
compare among themselves algorithms without and with annealing imitation, at the first set of advantages
before secon
d (it is easier, require smaller amount of iterations, have greater accuracy

if the power
landscape near to a global minimum is not cut up by local minima), but if the power landscape contains
set of local minima, the algorithms with annealing imitation
behave better (from the point of view of
criterion of quality of functioning). The confirmation of last statement can be served by the data of model
modeling. If to consider operation of system as a whole, it includes the following stages:


the formation o
f inquiry,


the preparation of an input vector of a network,


the cycles of functioning of the neural network,


the formation of a target vector.

Let's consider each stage.

The Formation of inquiry
. At this stage the user chooses a part of fields DB and
gives them some
values (using the dictionaries, appropriate to this fields, of values and sets of intervals). During functioning
the system will predict values of other fields (distinct from set by the user) similarly to associative

The preparatio
n of an input vector of a neural network.

At this stage the input vector is formed for
the neural network. It "is going" from subvectors, appropriate to the fields of Data Base. With formation of
a vector the zero categories subvectors are replaced on (
. Those subvectors, whose value sets the
user, are marked as "frozen". The values of other subvectors are formed with use of the random

The cycles of operation of a neural network
. This stage includes a number of cycles of operation
of a

neural network. Before each cycle on the neural network moves an input vector generated at the
previous stage of operation. Each cycle of operation of neural network consists of repeated iterations,
thus each iteration in turn includes two above described

stages (choice of the neuron
formation of a condition of the chosen neuron
candidate). It is necessary to notice neurons that
appropriate "frozen" subvectors, cannot be chosen as the neuron
candidate, and, hence, they do not
change the conditio
n during all functioning. On end of the first cycle received by a neural network a vector
and the value of energy of a network is remembered in the special buffer. At the end of each subsequent
cycle the value of energy of a network is compared with value
stored in buffer. If the current value of
energy is less then in the buffer, a current condition of a network (the condition of a network represents a
vector in quality which component the condition neurons act) replaces in the buffer earlier stored vector

The formation of a target vector.

It is a formal stage, when a vector kept in the buffer is given out
by system as the vector

answer restored by associative memory by a fragment, given by the user.


To the basic advantages of the approach,

proposed here, it is possible to relate the following.

Organic association of the decisions of such tasks as forecasting and detection of associations
within the framework of the uniform approach.

The training of neural network with the purpose of the s
ubsequent modeling requires only two
continuous passes on a simulated database. While with the traditional approach, namely with
accumulation statistics about combinations of values contained in DB, the multiiterative complex
procedures of search are requi

The decision of a task in those situations, when the traditional approach appears statistically
insolvent. For example, when analyzed DB contains small number of similar records.

Universality of Hopfield network, used as a basis of construction of a
ssociative memory.

Simplicity of program realization of Hopfield model.

This approach was realized in the program of Data Base Analysis (AnalDB). There the SQL
query is the source of data for learning of the neural network in this program. This program i
s realized on
Delphi 3.0 and Cbuilder 3.0. It may be used for find of most probable values of any fields in context
determined by other fields, and for forecasting of fields of also unexisting record (if the records are the
patterns of process in time). Th
is program is used for research of the charge of water in river Ob for last
104 years with the purpose of forecasting inflow of water and research of a demographic situation in one
district of Novosibirsk. This work is supported by the program "Universitie
s of Russia

researches" and grant of Russian Federation on a direction "Computer Science and cybernetics".


[1] Haken H. Synergetic computers for pattern recognition and associative memory. In Computational

Natural and Ar
tificial. Ed. H. Haken. Springer

Verlag, Berlin, 1987.

[2] Hopfield J. J. Neural networks and physical systems with emergent collective computational abilities //
Proc. Nat. Acad. Sci. USA


[3] Ackley D. H., Hinton G. E., Sejnowski T. J. A learn
ing algorithm for Boltzmann machines // Cognit Sci.


[4] Vedenov A. A., Ezhov A. A., Kamchatnow A. M. A study of Hopfield model of associative memory.
Institute of atomic energy.



[5] Hebb D. O. The Organization of Behavio

New York: Wiley, 1949