Fuzzy Logic & Neural Network

glassesbeepingAI and Robotics

Oct 20, 2013 (3 years and 11 months ago)

107 views

Fuzzy Logic

&

Neural Network


Introduction


Fuzzy logic has rapidly become one of the
most successful of today's technologies for
developing sophisticated control systems. The
reason for which is very simple. Fuzzy logic
addresses such applications perfectly as it
resembles human decision making wit
h an ability
to generate precise solutions from certain or
approximate information. It fills an important gap
in engineering design methods left vacant by purely
mathematical approaches (e.g. linear control
design), and purely logic
-
based approaches (e.g.
expert systems) in system design.




While other approaches require accurate equations to model real
-
world behaviors, fuzzy
design can accommodate the ambiguities of real
-
world human language and logic. It provides
both an intuitive method for
describing systems in human terms and automates the conversion of
those system specifications into effective models.

What does it offer?


The first applications of fuzzy theory were prima
ri
ly industrial, such as process control
for cement kilns. However, a
s the technology was further embraced, fuzzy logic was used in
more useful applications. In 1987, the first fuzzy logic
-
controlled subway was opened in Sendai
in northern Japan. Here, fuzzy
-
logic controllers make subway journeys more comfortable with
smoot
h braking and acceleration. Best of all, all the driver has to do is push the start button!
Fuzzy logic was also put to work in elevators to reduce waiting time. Since then, the applications
of Fuzzy Logic technology have virtually exploded, affecting thin
gs we use everyday.

Take for example, the
fuzzy washing machine
. A load of clothes in it and press start, and the
machine begins to churn, automatically choosing the best cycle. The fuzzy microwave, Place
chili, potatoes, or etc in a
fuzzy microwave
and p
ush single button, and it cooks for the right time
at the proper temperature. The
fuzzy car
,
maneuvers

itself by following simple verbal
instructions from its driver. It can even stop itself when there is an obstacle
immediately

ahead
using sensors. But, p
ractically the most exciting thing about it, is the simplicity involved in
operating it.



Background




The success of modern mathematics is largely due to the efforts of Aristotle and the
philosophers who preceded him. In their efforts to devise a concise theory of logic, and later
mathematics, the so
-
called "Laws of Thought" were invented. One of these, th
e "Law of the
Excluded Middle," stated that every proposition must either be True or False, A or not
-
A, either
this or not this. For example, a typical rose is either red
or
not red. It cannot be red
and
not red.
Every statement or sentence is true or fals
e or has the truth value 1 or 0. Even then, there already
were strong and immediate objections to this proposal.


In fact, almost two centuries earlier, Buddha saw the world filled with contradictions, that
things could be a certain degree true and a
certain degree false at the same time. However, it was
Plato who laid the foundation for what would become fuzzy logic, indicating that there was a
third region (beyond True and False) where these opposites "tumbled about.

The world's getting fuzzier

In th
e early 1900s, Lukasiewicz came and proposed a
systematic alternative to the bi
-
valued logic(
bivalence
) of
Aristotle. He described a three
-
valued logic, which can best be
translated as the term `possible', and assigned it a numeric
value between True and F
alse. Knuth, a former student of
Lukasiewicz proposed a three
-
valued logic apparently missed
by Lukasiewicz, which used an integral range [
-
1, 0 +1] rather
than [0, 1, 2]. Nonetheless, this alternative failed to gain
acceptance, and has passed into relativ
e obscurity.


It's heeere

It was not until relatively recently that the theory of fuzzy logic was discovered. Lotfi A. Zadeh,
a professor of UC Berkeley in California, soon to be known as the founder of fuzzy logic
observed that conventional computer lo
gic was incapable of manipulating data representing
subjective or vague human ideas such as "an
attractive

person" or "pretty hot". Fuzzy logic,
hence was designed to allow computers to determine the distinctions among data with shades of
gray, similar to
the process of human reasoning. In 1965, Zadeh published his seminal work
"Fuzzy Sets" which described the mathematics of fuzzy set theory, and by extension fuzzy logic.
This theory proposed making the membership function (or the values False and True) ope
rate
over the range of real numbers [0.0, 1.0]. Fuzzy logic was now introduced to the world.

Although, the technology was introduced in the United States, the scientist and researchers there,
ignored it mainly because of its unconventional name. They refus
ed to take something which
sounded so child
-
like seriously. Some mathematicians argued that fuzzy logic was merely
probability in disguise. Only stubborn scientists or ones who worked in discrete continued
researching it.

While the US and certain parts of

Europe ignored it, fuzzy logic was
accepted

with open arms in Japan, China and most Oriental countries. It
may be
surprising

to some that the world's largest
numbers of fuzzy
researchers are

in China with over 10,000 scientists. Japan, though
currently po
sitioned at the leading edge of fuzzy studies falls second in
manpower, followed by Europe and the USA. Hence, it can

be said that the popularity of fuzzy logic in the Orient reflects the fact that Oriental thinking
more easily accepts the concept of "fu
zziness". And because of this, the US, by some estimates,
trail Japan by at least ten years in this forefront of modern technology.


What do you

mean
fuzzy
??!!

Before illustrating the mechanisms which make fuzzy logic machines work, it is important to
realize what fuzzy logic actually is. Fuzzy logic is a superset of conventional(Boolean) logic that
has been extended to handle the concept of partial truth
-

truth
values between "completely true"
and "completely false". As its name suggests, it is the logic underlying modes of reasoning
which are approximate rather than exact. The importance of fuzzy logic derives from the fact that
most modes of human reasoning and

especially common sense reasoning are approximate in
nature.

The essential characteristics of fuzzy logic as founded by Zader Lotfi are as follows.



In fuzzy logic, exact reasoning is viewed as a limiting case of approximate reasoning.



In fuzzy logic every
thing is a matter of degree.



Any logical system can be fuzzified



In fuzzy logic, knowledge is interpreted as a collection of elastic or, equivalently , fuzzy
constraint on a collection of variables



Inference is viewed as a process of propagation of elastic

constraints.

The third
statements hence, define

Boolean logic as a subset of Fuzzy logic.



Fuzzy Sets

Fuzzy Set Theory was
formalized

by Professor Lofti Zadeh at the University of
California in 1965. What Zadeh proposed is very much a paradigm shift that

first gained
acceptance in the Far East and its successful application has ensured its adoption around the
world.

A paradigm is a set of rules and regulations which defines boundaries and tells us what to
do to be successful in solving problems within th
ese boundaries. For example the use of
transistors instead of vacuum tubes is a paradigm shift
-

likewise the development of Fuzzy Set
Theory from conventional bivalent set theory is a paradigm shift.

Bivalent Set Theory can be somewhat limiting if we wis
h to describe a 'humanistic'
problem mathematically. For example, Fig 1 below illustrates bivalent sets to
characterize

the
temperature of a room.



The most obvious limiting feature of bivalent sets that can be seen clearly from the
diagram is that they are mutually exclusive
-

it is not possible to have membership of more than
one set
(opinion

would widely vary as to whether 50 degrees Fahrenheit is
'cold' or 'cool' hence
the expert knowledge we need to define our system is mathematically at odds with the
humanistic world). Clearly, it is not accurate to define a
transition

from a quantity such as 'warm'
to 'hot' by the application of one degree Fahre
nheit of heat. In the real world a smooth
(unnoticeable) drift from warm to hot would occur.




Fuzzy Rules

Human beings make
decisions

based on rules. Although, we may not be aware of it, all
the
decisions

we make are all based on computer like if
-
then statements. If the weather is fine,
then we may decide to go out. If the forecast says the weather will be bad today, but fine
tomorrow
, then we make a
decision

not to go today, and postpone it till
tomorrow
.

Rules
associate ideas and relate one event to another.

Fuzzy machines, which always tend to mimic the
behavior

of man, work the same way.
However, the
decision

and the means of choosing that
decision

are replaced by fuzzy sets and the
rules are replaced b
y fuzzy rules. Fuzzy rules also operate using a series of if
-
then statements.

For instance, if X then A, if y then b, where A and B are all sets of X and Y. Fuzzy rules
define fuzzy
patches
, which is the key idea in fuzzy logic.

A machine is made smarter

using a concept designed by Bart Kosko called the Fuzzy
Approximation
Theorem (
FAT). The FAT theorem generally states a finite number of patches
can cover a curve as seen in the figure below. If the patches are large, then the rules are sloppy.
If the pat
ches are small then the rules are fine.


Fuzzy Patches

In a fuzzy system this simply means that all our rules can be seen as patches and the input
and output of the machine can be associated together using these patches. Graphically, if the
rule
patches

shrink, our fuzzy subset
triangles get

narrower. Simple
enough? Yes,
because

even
novices can build control systems that beat the best math models of control theory. Naturally, it
is
math
-
free

system.


Fuzzy Control

Fuzzy control, which directly uses fuzzy
rules,

is the most important application in fuzzy
theory. Using a procedure originated by Ebrahim Mamdani in the late 70s, three steps are taken
to create a fuzzy controlled machine:

1)Fuzzification(Using membership

functions to graphically describe a situation)

2)Rule evaluation(Application of fuzzy rules)

3)Defuzzification(Obtaining the crisp or actual results)




How far can fuzzy logic go???


Just from the examples given previously, it is clear
that fuzzy logic can be used in numerous applications. It can
appear almost
any place

where computers and modern
control theory are overly precise as well as in tasks requiring
delicate human intuition a
nd experience
-
based knowledge.
What does the future hold?
Consider

the example below
which is currently undergoing intensive research in OMRON
Research Center, Japan.


It may seem obvious that babies nowadays don't drink the way it is described in child
care books. They may drink a little or a lot depending on their physical condition, mood, and
other factors. But if a fuzzy
-
logic program can be created that would recomm
end how much to
feed the baby, mothers would have an easier time raising the child. The basis of the research is to
develop a program to determine the appropriate amount of milk to feed the child according to a
knowledge base that includes the child's pers
onality, physical condition, and some environmental
factors. This can prevent the child from being fed unnecessarily. Now although adapting fuzzy
logic to babies may seem silly, one can easily imagine using it to control the feeding of animals
in captivity
, for instance.

Want to see more. Well, here are some of the future fuzzy uses as predicted by Professor
Bart Kosko from UC Southern California:

Vast expert decision
makers,

theoretically able to distill the wisdom of every document ever
written
.

Sex ro
bots with a humanlike repertoire of behavior,

Computers that understand and respond to normal human language.

Machines that write interesting novels and screenplays in a selected style , such as Hemingway's.

Molecule
-
sized soldiers of health that will r
oam the blood
-
stream, killing cancer cells and slowing

the
aging process.

Hence, it can be seen that with the enormous research currently being done in Japan and
many other countries whose eyes have opened, the future of fuzzy logic is undetermined. There
is no limit to where it can go.

The future is bright. The future is
fuzzy
.




Introduction

to Neural networks

I have been interested in artificial intelligence and artificial life for years and I read most
of the popular books printed on the subject. I developed a grasp of most of the topics yet neural
networks always seemed to elude me. Sure, I could explain thei
r architecture but as to how they
actually worked and how they were implemented… well that was a complete mystery to me, as
much magic as science. I bought several books on the subject but every single one attacked the
subject from a very mathematical and
academic viewpoint and very few even gave any practical
uses or examples. So for a long long time I scratched my head and hoped that one day I would be
able to understand enough to experiment with them myself.

That day arrived some time later when
-

sat in

a tent in the highlands of Scotland reading
a book
-

I had a sudden blast of insight. It was one of those fantastic “eureka” moments and
although Scotland is a beautiful place I couldn’t wait to get to a computer so I could try out what
I’d just learnt. T
o my surprise the first neural net I programmed worked perfectly and I haven’t
looked back since. I still have a great deal to learn, neural nets are a huge subject, but I hope I
can share enough knowledge and enthusiasm to get you started on your own litt
le projects. In
many ways the fields of AI and A
-
Life are very exciting to work in. I think of these subjects as
the explorers of old must have looked at all those vast empty spaces on the maps. There is so
much to learn and discover.

I’ll start off by de
scribing what a neural net actually is and what it’s architecture is, then
I’ll do a little theory on how we get it to perform for us but I’ll try to use as little maths as
possible. (Having some understanding of mathematics is impossible to avoid however
and the
deeper you get into this topic the more mathematics you are going to have to learn). Finally,
we’ll get to the fun bit. I’ll come up with a little project I will program and take you through one
step at a time. It will be in this last phase of the
tutorial where I hope you get the same “eureka”
feeling for neural nets as I did back in rainy old Scotland. Until then just sit back, absorb and be
patient.



So, what exactly is a Neural Network?

A neural network is mans crude way of trying to simulate
the brain electronically. So to
understand how a neural net works we first must have a look at how the old grey matter does its
business…

Our brains are made up of about 100 billion tiny units called
neurons
. Each neuron is
connected to thousands of other
neurons and communicates with them via electrochemical
signals. Signals coming into the neuron are received via junctions called
synapses
, these in turn
are located at the end of branches of the neuron cell called
dendrites
. The neuron continuously
receive
s signals from these inputs and then performs a little bit of magic. What the neuron does
(this is over simplified I might add) is sum up the inputs to itself in some way and then, if the end
result is greater than some threshold value, the neuron fires. I
t generates a voltage and outputs a
signal along something called an
axon
. Don't worry too much about remembering all these new
words as we won’t be using many of them from this moment onwards, just have a good look at
the illustration and try to picture w
hat is happening within this simple little cell.






Neural networks are made up of many artificial neurons. An artificial neuron is simply an
electronically modelled

biological neuron. How many neurons are used depends on the task at
hand. It could be as few as three or as many as several thousand. One optimistic researcher has
even hard wired 2 million neurons together in the hope he can come up with something as
int
elligent as a cat although most people in the AI community doubt he will be successful
(
Update: he wasn't!
). There are many different ways of connecting

artificial neurons together to
create a neural network but I shall be concentrating on the most common

which is called a
feedforward

network. So, I guess you are asking yourself,
what does an artificial neuron look
like?

Well here you go:








Each input into the neuron has its own weight associated with it illustrated by the red circle. A
weight is sim
ply a floating point number and it's these we adjust when we eventually come to
train the network. The weights in most neural nets can be both negative and positive, therefore
providing excitory or inhibitory influences to each input. As each input enters
the nucleus (blue
circle) it's multiplied by its weight. The nucleus then sums all these new input values which gives
us the
activation

(again a floating point number which can be negative or positive). If the
activation is greater than a threshold value
-

lets use the number 1 as an example
-

the neuron
outputs a signal. If the activation is less than 1 the neuron outputs zero. This is typically called a

step

function (take a peek at the following diagram and have a guess why).