Fuzzy Logic & Neural Network

glassesbeepingAI and Robotics

Oct 20, 2013 (3 years and 7 months ago)


Fuzzy Logic


Neural Network


Fuzzy logic has rapidly become one of the
most successful of today's technologies for
developing sophisticated control systems. The
reason for which is very simple. Fuzzy logic
addresses such applications perfectly as it
resembles human decision making wit
h an ability
to generate precise solutions from certain or
approximate information. It fills an important gap
in engineering design methods left vacant by purely
mathematical approaches (e.g. linear control
design), and purely logic
based approaches (e.g.
expert systems) in system design.

While other approaches require accurate equations to model real
world behaviors, fuzzy
design can accommodate the ambiguities of real
world human language and logic. It provides
both an intuitive method for
describing systems in human terms and automates the conversion of
those system specifications into effective models.

What does it offer?

The first applications of fuzzy theory were prima
ly industrial, such as process control
for cement kilns. However, a
s the technology was further embraced, fuzzy logic was used in
more useful applications. In 1987, the first fuzzy logic
controlled subway was opened in Sendai
in northern Japan. Here, fuzzy
logic controllers make subway journeys more comfortable with
h braking and acceleration. Best of all, all the driver has to do is push the start button!
Fuzzy logic was also put to work in elevators to reduce waiting time. Since then, the applications
of Fuzzy Logic technology have virtually exploded, affecting thin
gs we use everyday.

Take for example, the
fuzzy washing machine
. A load of clothes in it and press start, and the
machine begins to churn, automatically choosing the best cycle. The fuzzy microwave, Place
chili, potatoes, or etc in a
fuzzy microwave
and p
ush single button, and it cooks for the right time
at the proper temperature. The
fuzzy car

itself by following simple verbal
instructions from its driver. It can even stop itself when there is an obstacle

using sensors. But, p
ractically the most exciting thing about it, is the simplicity involved in
operating it.


The success of modern mathematics is largely due to the efforts of Aristotle and the
philosophers who preceded him. In their efforts to devise a concise theory of logic, and later
mathematics, the so
called "Laws of Thought" were invented. One of these, th
e "Law of the
Excluded Middle," stated that every proposition must either be True or False, A or not
A, either
this or not this. For example, a typical rose is either red
not red. It cannot be red
not red.
Every statement or sentence is true or fals
e or has the truth value 1 or 0. Even then, there already
were strong and immediate objections to this proposal.

In fact, almost two centuries earlier, Buddha saw the world filled with contradictions, that
things could be a certain degree true and a
certain degree false at the same time. However, it was
Plato who laid the foundation for what would become fuzzy logic, indicating that there was a
third region (beyond True and False) where these opposites "tumbled about.

The world's getting fuzzier

In th
e early 1900s, Lukasiewicz came and proposed a
systematic alternative to the bi
valued logic(
) of
Aristotle. He described a three
valued logic, which can best be
translated as the term `possible', and assigned it a numeric
value between True and F
alse. Knuth, a former student of
Lukasiewicz proposed a three
valued logic apparently missed
by Lukasiewicz, which used an integral range [
1, 0 +1] rather
than [0, 1, 2]. Nonetheless, this alternative failed to gain
acceptance, and has passed into relativ
e obscurity.

It's heeere

It was not until relatively recently that the theory of fuzzy logic was discovered. Lotfi A. Zadeh,
a professor of UC Berkeley in California, soon to be known as the founder of fuzzy logic
observed that conventional computer lo
gic was incapable of manipulating data representing
subjective or vague human ideas such as "an

person" or "pretty hot". Fuzzy logic,
hence was designed to allow computers to determine the distinctions among data with shades of
gray, similar to
the process of human reasoning. In 1965, Zadeh published his seminal work
"Fuzzy Sets" which described the mathematics of fuzzy set theory, and by extension fuzzy logic.
This theory proposed making the membership function (or the values False and True) ope
over the range of real numbers [0.0, 1.0]. Fuzzy logic was now introduced to the world.

Although, the technology was introduced in the United States, the scientist and researchers there,
ignored it mainly because of its unconventional name. They refus
ed to take something which
sounded so child
like seriously. Some mathematicians argued that fuzzy logic was merely
probability in disguise. Only stubborn scientists or ones who worked in discrete continued
researching it.

While the US and certain parts of

Europe ignored it, fuzzy logic was

with open arms in Japan, China and most Oriental countries. It
may be

to some that the world's largest
numbers of fuzzy
researchers are

in China with over 10,000 scientists. Japan, though
currently po
sitioned at the leading edge of fuzzy studies falls second in
manpower, followed by Europe and the USA. Hence, it can

be said that the popularity of fuzzy logic in the Orient reflects the fact that Oriental thinking
more easily accepts the concept of "fu
zziness". And because of this, the US, by some estimates,
trail Japan by at least ten years in this forefront of modern technology.

What do you


Before illustrating the mechanisms which make fuzzy logic machines work, it is important to
realize what fuzzy logic actually is. Fuzzy logic is a superset of conventional(Boolean) logic that
has been extended to handle the concept of partial truth

values between "completely true"
and "completely false". As its name suggests, it is the logic underlying modes of reasoning
which are approximate rather than exact. The importance of fuzzy logic derives from the fact that
most modes of human reasoning and

especially common sense reasoning are approximate in

The essential characteristics of fuzzy logic as founded by Zader Lotfi are as follows.

In fuzzy logic, exact reasoning is viewed as a limiting case of approximate reasoning.

In fuzzy logic every
thing is a matter of degree.

Any logical system can be fuzzified

In fuzzy logic, knowledge is interpreted as a collection of elastic or, equivalently , fuzzy
constraint on a collection of variables

Inference is viewed as a process of propagation of elastic


The third
statements hence, define

Boolean logic as a subset of Fuzzy logic.

Fuzzy Sets

Fuzzy Set Theory was

by Professor Lofti Zadeh at the University of
California in 1965. What Zadeh proposed is very much a paradigm shift that

first gained
acceptance in the Far East and its successful application has ensured its adoption around the

A paradigm is a set of rules and regulations which defines boundaries and tells us what to
do to be successful in solving problems within th
ese boundaries. For example the use of
transistors instead of vacuum tubes is a paradigm shift

likewise the development of Fuzzy Set
Theory from conventional bivalent set theory is a paradigm shift.

Bivalent Set Theory can be somewhat limiting if we wis
h to describe a 'humanistic'
problem mathematically. For example, Fig 1 below illustrates bivalent sets to

temperature of a room.

The most obvious limiting feature of bivalent sets that can be seen clearly from the
diagram is that they are mutually exclusive

it is not possible to have membership of more than
one set

would widely vary as to whether 50 degrees Fahrenheit is
'cold' or 'cool' hence
the expert knowledge we need to define our system is mathematically at odds with the
humanistic world). Clearly, it is not accurate to define a

from a quantity such as 'warm'
to 'hot' by the application of one degree Fahre
nheit of heat. In the real world a smooth
(unnoticeable) drift from warm to hot would occur.

Fuzzy Rules

Human beings make

based on rules. Although, we may not be aware of it, all

we make are all based on computer like if
then statements. If the weather is fine,
then we may decide to go out. If the forecast says the weather will be bad today, but fine
, then we make a

not to go today, and postpone it till

associate ideas and relate one event to another.

Fuzzy machines, which always tend to mimic the

of man, work the same way.
However, the

and the means of choosing that

are replaced by fuzzy sets and the
rules are replaced b
y fuzzy rules. Fuzzy rules also operate using a series of if
then statements.

For instance, if X then A, if y then b, where A and B are all sets of X and Y. Fuzzy rules
define fuzzy
, which is the key idea in fuzzy logic.

A machine is made smarter

using a concept designed by Bart Kosko called the Fuzzy
Theorem (
FAT). The FAT theorem generally states a finite number of patches
can cover a curve as seen in the figure below. If the patches are large, then the rules are sloppy.
If the pat
ches are small then the rules are fine.

Fuzzy Patches

In a fuzzy system this simply means that all our rules can be seen as patches and the input
and output of the machine can be associated together using these patches. Graphically, if the

shrink, our fuzzy subset
triangles get

narrower. Simple
enough? Yes,

novices can build control systems that beat the best math models of control theory. Naturally, it


Fuzzy Control

Fuzzy control, which directly uses fuzzy

is the most important application in fuzzy
theory. Using a procedure originated by Ebrahim Mamdani in the late 70s, three steps are taken
to create a fuzzy controlled machine:

1)Fuzzification(Using membership

functions to graphically describe a situation)

2)Rule evaluation(Application of fuzzy rules)

3)Defuzzification(Obtaining the crisp or actual results)

How far can fuzzy logic go???

Just from the examples given previously, it is clear
that fuzzy logic can be used in numerous applications. It can
appear almost
any place

where computers and modern
control theory are overly precise as well as in tasks requiring
delicate human intuition a
nd experience
based knowledge.
What does the future hold?

the example below
which is currently undergoing intensive research in OMRON
Research Center, Japan.

It may seem obvious that babies nowadays don't drink the way it is described in child
care books. They may drink a little or a lot depending on their physical condition, mood, and
other factors. But if a fuzzy
logic program can be created that would recomm
end how much to
feed the baby, mothers would have an easier time raising the child. The basis of the research is to
develop a program to determine the appropriate amount of milk to feed the child according to a
knowledge base that includes the child's pers
onality, physical condition, and some environmental
factors. This can prevent the child from being fed unnecessarily. Now although adapting fuzzy
logic to babies may seem silly, one can easily imagine using it to control the feeding of animals
in captivity
, for instance.

Want to see more. Well, here are some of the future fuzzy uses as predicted by Professor
Bart Kosko from UC Southern California:

Vast expert decision

theoretically able to distill the wisdom of every document ever

Sex ro
bots with a humanlike repertoire of behavior,

Computers that understand and respond to normal human language.

Machines that write interesting novels and screenplays in a selected style , such as Hemingway's.

sized soldiers of health that will r
oam the blood
stream, killing cancer cells and slowing

aging process.

Hence, it can be seen that with the enormous research currently being done in Japan and
many other countries whose eyes have opened, the future of fuzzy logic is undetermined. There
is no limit to where it can go.

The future is bright. The future is


to Neural networks

I have been interested in artificial intelligence and artificial life for years and I read most
of the popular books printed on the subject. I developed a grasp of most of the topics yet neural
networks always seemed to elude me. Sure, I could explain thei
r architecture but as to how they
actually worked and how they were implemented… well that was a complete mystery to me, as
much magic as science. I bought several books on the subject but every single one attacked the
subject from a very mathematical and
academic viewpoint and very few even gave any practical
uses or examples. So for a long long time I scratched my head and hoped that one day I would be
able to understand enough to experiment with them myself.

That day arrived some time later when

sat in

a tent in the highlands of Scotland reading
a book

I had a sudden blast of insight. It was one of those fantastic “eureka” moments and
although Scotland is a beautiful place I couldn’t wait to get to a computer so I could try out what
I’d just learnt. T
o my surprise the first neural net I programmed worked perfectly and I haven’t
looked back since. I still have a great deal to learn, neural nets are a huge subject, but I hope I
can share enough knowledge and enthusiasm to get you started on your own litt
le projects. In
many ways the fields of AI and A
Life are very exciting to work in. I think of these subjects as
the explorers of old must have looked at all those vast empty spaces on the maps. There is so
much to learn and discover.

I’ll start off by de
scribing what a neural net actually is and what it’s architecture is, then
I’ll do a little theory on how we get it to perform for us but I’ll try to use as little maths as
possible. (Having some understanding of mathematics is impossible to avoid however
and the
deeper you get into this topic the more mathematics you are going to have to learn). Finally,
we’ll get to the fun bit. I’ll come up with a little project I will program and take you through one
step at a time. It will be in this last phase of the
tutorial where I hope you get the same “eureka”
feeling for neural nets as I did back in rainy old Scotland. Until then just sit back, absorb and be

So, what exactly is a Neural Network?

A neural network is mans crude way of trying to simulate
the brain electronically. So to
understand how a neural net works we first must have a look at how the old grey matter does its

Our brains are made up of about 100 billion tiny units called
. Each neuron is
connected to thousands of other
neurons and communicates with them via electrochemical
signals. Signals coming into the neuron are received via junctions called
, these in turn
are located at the end of branches of the neuron cell called
. The neuron continuously
s signals from these inputs and then performs a little bit of magic. What the neuron does
(this is over simplified I might add) is sum up the inputs to itself in some way and then, if the end
result is greater than some threshold value, the neuron fires. I
t generates a voltage and outputs a
signal along something called an
. Don't worry too much about remembering all these new
words as we won’t be using many of them from this moment onwards, just have a good look at
the illustration and try to picture w
hat is happening within this simple little cell.

Neural networks are made up of many artificial neurons. An artificial neuron is simply an
electronically modelled

biological neuron. How many neurons are used depends on the task at
hand. It could be as few as three or as many as several thousand. One optimistic researcher has
even hard wired 2 million neurons together in the hope he can come up with something as
elligent as a cat although most people in the AI community doubt he will be successful
Update: he wasn't!
). There are many different ways of connecting

artificial neurons together to
create a neural network but I shall be concentrating on the most common

which is called a

network. So, I guess you are asking yourself,
what does an artificial neuron look

Well here you go:

Each input into the neuron has its own weight associated with it illustrated by the red circle. A
weight is sim
ply a floating point number and it's these we adjust when we eventually come to
train the network. The weights in most neural nets can be both negative and positive, therefore
providing excitory or inhibitory influences to each input. As each input enters
the nucleus (blue
circle) it's multiplied by its weight. The nucleus then sums all these new input values which gives
us the

(again a floating point number which can be negative or positive). If the
activation is greater than a threshold value

lets use the number 1 as an example

the neuron
outputs a signal. If the activation is less than 1 the neuron outputs zero. This is typically called a


function (take a peek at the following diagram and have a guess why).