THE IMPROVEMENTS OF PARALLEL COMPUTING ON COMPUTATION SPEED

shrewdnessfreedomΛογισμικό & κατασκευή λογ/κού

2 Δεκ 2013 (πριν από 3 χρόνια και 4 μήνες)

59 εμφανίσεις

Session C11

Paper#3201

University of Pittsburgh, Swanson School of Engineering 1


April 2,2013


THE IMPROVEMENTS OF PARALLEL COMPUTING ON COMPUTATION
SPEED


Stephen Klein (sjk83@pitt.edu Budny 4:00), Lunan Li (lul26@pitt.edu Luis 6:00)


Abstract

The

computer industry is slowly changing the
direction of their pioneering efforts to maximize computing
potential. The computer industry's efforts over the past
decades have been spent miniaturizing computer chips;
however, this trend cannot continue far int
o the future.
Therefore their attention is now being spent elsewhere, and
one area the industry is focusing on developing is parallel
computing. Parallel computing is a new method of
computing in which instructions are carried out by a
computer simultaneou
sly as opposed to
sequentially.

In our paper, we will talk about the challenges
associated with the realization of parallel computing on
consumer computers. This involves technical software
programming problems; that will be the focus of our paper.
Parallel computers use fundamentally di
fferent algorithms
than classical computers, and many times the algorithms can
be vastly more complicated. For this reason current
programming languages do not readily support parallel
computing. This lack of support necessitates the need for a
new program
ming language to be written. ParaSail is a
language written for this purpose; we will evaluate its
performance and potential to be popularly used by
programmers in the near future. We will also evaluate what
we believe the future of the computer industry w
ill look like,
and how computers of the future are likely to operate based
on
current industry
trends.
T
he development of parallel
computing
may

increase the burden

of the computer industry

on the environment. Due to
use of hazardous

materials,

like
mercur
y, arsenic, and radioactive barium,

manufacturing
and disposal of computers needs to be handled with more
care.

We will discuss these
environmental

issues in detail
and explore solutions to
the negative effects the computer
industry may cause
.

Parallel co
mputing will be very valuable in society. It is
a technology that will open the doorway for new innovations
in computer programming such as accurate speech
recognition and more accurate artificial intelligence. The
improvements it can bring to computer per
formance will
stimulate the growth of the computer industry and will
hopefully allow the exponential growth of computer speed to
continue.


Key words
-

computer language, multi
-
core, multi
-
threaded,
ParaSail, Parallel computing


THE MODERN ENVIRONMENT OF
COMPUTER ENGINEERING

With the development of the I
nformation Technology

industry, as well as the globalization of networking and
computers, computer engineering has become one of the
most popular fields in engineering. The development of both
computer hard
ware and software require computer
engineering to be cross
-
disciplinary between electrical
engineering and computer science. Parallel computing is a
form of computation in which many calculations are carried
out simultaneously, operating on the principle t
hat large
problems can often be divided into smaller ones, which are
then solved concurrently (in parallel)[1].

Although
computer
engineers paid attention to
miniaturizing computer chips

in the past decades,

this trend
cannot continue far into the future.
Firstly, the
manufacturing processes, and the chips themselves are
environmental hazards. The computer industry must needs to
focus on both improving the processor technology, and
minimizing the footprint the manufacturing process leaves
on the environment
.

Secondly,

Intel and Advanced Micro
Devices have been making transistors and memory cells
smaller and smaller to increase the amount of transistors that
can fit on a chip, and therefore increase the processor speed.
It is becoming increasingly difficult t
o continue to scale
transistors down because they are getting
too
small. The
manufacturing scale for the next generation of Intel chips is
twenty two nanometers. A theoretical limit for the smallest
unit size attainable is sixteen nanometers, at this limit

quantum effects interfere with the operation of transistors[2].
Because the limit
of

transistor size is nearly reached, the
computer industry's attention is being spent elsewhere, and
one area the industry is focusing on developing is parallel
computing.
Parallel computing is a wholly different process
than classical computing.


CLASSICAL VERSUS PARALLEL
COMPUTERS


Traditionally, computer software has been written for
serial computation. To solve a problem, an algorithm is
constructed and implemented as a
serial stream of
instructions. These instructions are executed by the central
processing unit (CPU) of a computer. Only one instruction
may execute

at a time

after that instruction is finished the
next is executed. Serial computers have distinct advantages

and disadvantages. The advantages of serial computers are
that they are composed of simpler hardware, and
programming a serial computer is a well
-
defined, simple
process. The weaknesses of serial computers are the fact that
some processes do not run optim
ally on their architecture,
Lunan Li

Stephen Klein



2


and they leave little room for improvement in terms of speed
of computations with the exception of packing more
transistors on a processor.

In contrast to traditional computer computation, parallel
computing executes multiple in
structions simultaneously.
The basic idea of parallel computing is divide and conquer, a
number of processors work on a task instead of just one.
Parallel computer’s high work efficiency is easy to
understand because a computer with multiple processors
whi
ch operate simultaneously can achieve results multiple
times faster. The maximum speed up of a parallel computing
is dictated by Amdahl’s law
, which says that the amount of
speedup gained by adding processors or processor cores is
dictated by amount of par
allelizable code, and there will be
diminishing returns as more processors are added[3].
Despite diminishing returns, parallel computing still has
potential to revolutionize the computer industry.

FIGURE 1

This is a diagram of Amdahl's law. It is easy to
see how as
more parallel code is added, the speedup due to parallel
computing increases[4].


WHY DO WE NEED PARALLEL
COMPUTING?


The need for parallel computing is much greater now
than in the past. The first reason for this is societies


pursuit
of efficient productivity. Because of the financial crisis that
happened in America in 2008, jobs have become scarce.
According to Trading Economics,
the u
nemployment
r
ate in
the United States increased to 7.90% in January of 2013
from 7.80% in De
cember of 2012[5]. Computer engineers
are very in demand, and further innovations, like parallel
computing, will drive the demand for computer engineers
even higher. Parallel computing will cause job creation in
the short term, however projections past the

next few years
cannot be made because its future is uncertain. According to
the Bureau of Labor Statistics, computer applications
software engineers and computer systems software engineers
are projected to be among the fastest growing occupations
from 200
8 to 2018. BLS reports an expected growth of 30%
for software developers from 2010 to 2020[6]. The
employment of computer hardware engineers is expected to
increase nine percent. From the report we can see that
software engineering is developing fast and r
equires many
new graduates to join the computer industry.
The adoption of
parallel computing in the computer industry will also likely
cause employment to rise because of the skilled
programmers that are need
ed

to write parallel code.

Parallel
co
mputing i
s a valuable process,
but most programmers
ignore it because it is much more complicated to write code
in parallel. As the need for parallel programs increases it
will take more work to create programs, and this increase in
work will create jobs for compu
ter software engineers.

Another reason parallel computing is important is the
impossibility of forever increasing computer processor clock
speeds. During the past three decades, the developments of
the CPU have been promoted by the development of
transistor technology and compute
r architecture. Moore's
law states that
the
speed
of
computer double
s

every 18 to 24
months. This simple law has driven the explosive growth of
the computer industry, and a lot of the technological
advancement in today's society. For example: smart

phones
today have more processing power than all of NASA's
computing capabilities in 1969, the year they sent Neil
Armstrong to the moon (a Sony Playstation has more
computing power than the fastest supercomputer in
1997)[7]. Because of the exponential growth of
computer
complexity, computer utility has risen as the price falls. If
this trend does not continue into the future, the computer
industry will slow down tremendously. There is no reason to
replace a three year old computer when the newer model is
only 20
percent faster. Especially when in the past, after
three years a consumer could get a 200% faster computer.
This stagnation in the computer industry will have drastic
economic effects if it is left unresolved. The growth of the
computer industry is marked
with growth of other industries
because of the way computers effect productivity across the
board. As computers get faster, and more highly automated,
they increase productivity in every area they are applied.

According to Michio

Kaku, if Moore's law ends

with no
replacement technology the global economy will decrease by
trillions
of dollars. He attributes this to the pervasiveness of
computers in every aspect of modern human life and the
scope of the computer industry on our global economy. He
states in h
is interview with Big Think that a number of
alternative technologies are in line to replace the void that
will be left when Moore's law collapses[9]. Although parallel
computing is not his first choice as a replacement technology
(he favors quantum comput
ing, a much more futuristic
technology), he recognizes the value of parallelism and says
that it can temporarily fill the void[9]. An economist at
Northwestern University, Robert Gordon, attributes Moore's
law to economic boom of the Clinton era up until t
oday[10].
Dr
.

Gordon believes that the importance of Moore's law is
often understated, and that it has more influence on the
global economy than any other single factor[10].

Lunan Li

Stephen Klein



3


It seems like today we take for granted the technology
that is now available. Only

thirty years ago the internet was
only used to transmit data
among

scientists, and now it has
billions of users
who

can connect to it with a device in our
pocket
s
.

T
hirty years ago, the world was shocked by a five
pound camera and a portable cassette pla
yer. These
examples of the drastic change in technology show how
important Moore's law is. In 1983 consumers had new
gadgets to buy because the technology was still progressing.
If Moore's law comes to an end there will no longer be the
drastic changes in
technology, and the age of economic
boom and technologically oriented consumerism will end.
Due to all of these factors, Moore's law has to be sustained
for as long as possible. The computer industry needs to take
seriously alternative technologies than mi
niaturizing
transistors; one of these technologies being parallel
computing[9].


PARALLEL COMPUTING
CAUSES
BURDEN
S

ON ENVIRONMENT


Environmental issues like industrial pollution and
global warming are very important topics in today's society.
More than eve
r before, guidelines are placed on
manufacturing companies and incentives are given for being
environmentally friendly. The development in the computer
industry poses a great burden on the environment because
hardware becomes obsolete and must be replaced

every
couple of years. In the consumer market people are expected
to buy a new computer every three to four years, and all of
the old technology simply makes its way to a landfill.
Although these new computers are more efficient and help
people a lot in
their daily work, they cause serious
environmental problems. The two biggest problems are the
proper disposal of computer chips and the harmful
manufacturing processes.


Chips in computers cannot be recycl
able

as soon as the
chip moves into a state of obso
lescence it is discarded. As
discussed above, parallel computing contributes to the
production of multi
-
core processors which are used in
parallel computers. Due to the highly efficient performance
of multi
-
core processors, serial computers are gradually
being replaced by parallel computers. The differences in
chip manufacturing techniques of multi
-
core computers and
serial computers require that all of the outdated serial
components be replaced by more modern parallel hardware.
Millions of tons of proces
sing chips are buried in a landfill
every year. This leads to environmental pollution because
some materials in chips contain harmful chemical residues
that cannot be recycled such as mercury, arsenic, and
barium
[
14
]
. Mercury and arsenic are toxic
elements, and
barium is radioactive. These toxins can cause serious
environmental harm if they are not disposed of properly, and
little is done to remedy this issue.

Another environmental issue with computer chips is the
industrial pollution cause by the
fabrication process. There
are four basic steps in microchip fabrication, which involve
sequences of layering, oxidation, patterning processing and
etching. In each step, microchips are bathed in a wide range
of chemical solvents in order to ensure their

purity, because
any small defect on a microchip surface can hinder its
function
[
14
]
. Chemical solvents used in chip fabrication
include: hydrochloric acid, hydrofluoric acid, arsenic,
benzene and hexavalent chromium
[
14
]
. Many of these
chemicals are known
to cause environmental problems and
have negative effects on human health. Chemicals used in
computer fabrication are of purity typically in the range of
99.995
-
99.9999%, compared with industrial grade purities of
90
-
99% and some of these ultrapure chemica
ls make their
way into the environment
[
14
].

Disposal of these chemicals
causes severe environmental problems.

Sustainability, or the capacity of something to endure,
will be hard to achieve in the computer industry. The three
to four year replacement cy
cle of most computer hardware
means that materials are used for a short time. The
manufacturing processes also ensure that no components can
be reused, so computers are not recyclable. This creates a
compounding negative effect on the environment that ma
kes
complete environmental sustainability a lofty goal.
Although it may be hard to become completely sustainable,
strides can be taken to improve the environmental footprint
of the computer industry.

One step in the right direction for the computer industry
is the implementation of computer waste retrieval services,
and a dedication to safer manufacturing processes. Many
countries have created incentives for manufactures to make
their computer system
more easily recyclable and
environmentally friendly. Both Taiwan and Japan, two
countries with large computer manufacturing industries,
have been making computers that are easily disassembled
into constituent components so that the toxic waste can be
safe
ly discarded and so certain parts can be recycled[25].
With programs like these implemented in more countries,
the computer industry would greatly reduce the negative
environmental impacts it causes.


PARALLEL COMPUTING HARDWARE


The first commercial mult
i
-
core processor was the
Power4 processor released by IBM in 2001[11]. In this
pioneering computer chip, two 64 bit one gigahertz (GHz)
PowerPC cores were integrated into a Power4 chip which
could execute 2 threads at once[11]. HP launched a similar
multi
-
core processor, the PA
-
RISC 8800, in 2003 where two
1 GHz PA
-
RISC 8700 processors were integrated. Sun
launched its own multi
-
core processor, the UltraSparc IV, in
2004 where two UltraSparc III cores were integrated. In
2005, Intel and AMD also launched th
eir respective
commercial dual
-
core processors, the Pentium D and the
Opteron[11]. These processor's architectures are very
Lunan Li

Stephen Klein



4


different t
o

multi
-
core chips of today. These early examples
of a young technology were literally two independent
processors on a single chip.

The chipmakers recognized an issue with their initial
designs: they found that competitions between cores for
computer resour
ces, liked shared cache memory, caused a lot
of transistors on the chips to remain unused because they
could not get instructions from the memory[12]. Past a
certain threshold the clock speed of these processors no
longer mattered because the bottleneck of

these early multi
-
core processors was memory management. In further
experiments with early parallel chips, peak speed of chips
with smaller and more numerous transistors were lower than
what they had been five years before. The main reason is
microprocess
ors on the chip did not fully ‘work’. Although
both cores on a microprocessor ran at the same time, the
instructions were handled more slowly. Intel improved the
flawed design of the Pentium D on their next generation of
processors
,

the Core2duo. The Core2
duo was two cores cast
on the same die with much greater integrated hardware
sharing capabilities. This chip was revolutionary because the
memory bottleneck was removed and both cores worked
effectively. The current i7 architecture is similar, but
optimize
d to run more efficiently. Although both of these
newer architectures are effective for the number of processor
cores now, they will become a bottleneck to performance
once the number of cores rises even higher. An architecture
of a future many
-
core proces
sor will likely have over 50
cores and memory caches that service one to two cores each;
these processors will likely run at much slower clock speeds
than today’s computers[13]. Although parallel computing
hardware has come very far and there is a very cle
ar road
map to the future of parallel computing in terms of
hardware, it is all meaningless unless software can utilize the
hardware through effective programming.



PROGRAMMING PROBLEMS IN
PARALLEL


Most

applications today are programmed without parallel
computing in mind. This means that applications are only
programmed to execute on
e

command at a time, and the
commands are executed sequentially. When a programmer
writes a code he is giving the computer a list of instructions
to execute. This is the estab
lished way of writing computer
code because parallelism in computer chips is a relatively
new innovation. The way programmers continue to code
must change to accommodate parallel computers.

A serial computer is bottlenecked by the number of
operations it c
an perform per second. Although most serial
computers of today can handle millions of instructions per
second, parallel computing will allow much greater
performance by multiplying the operations per second by the
amount of cores a computer processor has.
This performance
difference cannot be achieved without changing the way we
program our computers.

Although it is easy to see why we need to change the
way we program, it is not very clear how we should do it.
Some tasks are parallel by the nature of their
execution, but
some tasks seem impossible to make parallel. An analogy
would be if it takes a single worker nine hours to paint a
room, it would take three workers three hours to paint the
room; but if it takes a single chef 30 minutes to make a meal,
thre
e chefs could not do it in ten minutes because there is a
defined amount of time that each step ought to take.
Programming in parallel is the same way: some tasks must
be finished before others begin for many operations. This
means that splitting a chunk o
f code into “threads”
(individual components of a parallel program) is often not
any faster than sequential programming. This is because
many threads cannot begin calculations until other threads
finish, and that sequential order is the same as if the
algo
rithm was run on a serial computer. The only way to
achieve a performance boost through parallel computing is
to separate the code into independent threads that do not
depend upon each other for completion.


Adobe

Creative Suites
5


Applications that have
been written for parallel
computers can perform faster than their non parallelized
counterparts. This can be seen clearly in the processing
difference between Adobe CS4 and CS5. CS4 was not
optimized for multithreaded processors,
that

feature was
added in
CS5. This means that a direct comparison between
the two applications is a comparison between a non
-
parallel
applications and parallel applications. When the computation
times for certain operations within CS4 and CS5 are
calculated it is very easy to see
that CS5, the parallel
application, has the performance edge and out competes CS4
by a massive amount on any multi
-
core processor.

The group that compared the two programs used
computers from one to twelve cores to test how long it took
certain processes t
o run on each program. In CS4 the run
times for rendering a video in
A
dobe
A
fter

E
ffects (an
element of CS4 and CS5) between double, quad, and hexa
core processors were nearly identical. On CS5 however,
rendering the same video took less and less time with

each
core added[15]. In CS4 the render took about four and a half
minutes, but with two cores in CS5 the time was almost cut
in half at two minutes and twenty seconds. Although there
were diminishing returns for each core added, by the sixth
core the time

was down to one minute and one second[15].


COMPUTER LANGUAGES AND
PROGRAMS


Adobe CS5


Lunan Li

Stephen Klein



5


In the last section we compared the performance of
Adobe CS5 ( a parallel application) with its predecessor CS4
(a non parallel

application). Now we are going to analyze
from
where the performance boost comes. Adobe CS5 is an
image and video editing software package. Picture
processing is a very parallel process
.

F
or

example
;
w
e are
going to turn a colored image into a black and
white image
in CS4. The program would analyze the entire image pixel
by pixel, assigning each pixel a value for the shade of gray
that was calculated from the colors of the pixel. In Adobe
Photoshop CS5 however, the process would be broken up
into separate
ly executable threads. For the process of turning
an image into black and white, the image would be divided
up into sections; one section for every thread. Then the
multi
-
core processor would execute all of the threads at once
and then recombine each secti
on into the new image. It is
easy to see how parallelism can be applied to this
application
;
however
,
this is not always so easy to achieve.


OpenMP


Eventually what computer scientists are trying to achieve
is a simple way to write all programs in paralle
l. Today’s
computer languages do not easily facilitate parallel
computing. Fortran and C, popular computer languages, both
have an extension to make parallel computing possible: this
extension is called Open Multiprocessing or OpenMP.

OpenMP works by a pri
nciple of Fork
-
Join parallelism;
this is when a master thread creates subsidiary threads and
then runs those threads in parallel. Each thread has its own
set of code within the master thread that a programmer needs
to ensure is truly parallel. This extensi
on leaves it up to the
programmer to parallelize the code and do the problem
solving and that makes the programming process much
longer and more tedious.

A major drawback of OpenMP is memory and work
sharing. OpenMP uses global variables (variables that ar
e
stored in memory and accessible from anywhere in the
code), that can be accessed by multiple threads at once but
only one thread should edit the variable at a time. This
means that OpenMP opens multiple threads that can share
the same memory, and then al
l of the threads perform can
operations on the same variable. The best case scenario for
this error is it being recognized by the compiler and the
program is simply run sequentially so that the variable is not
changed by more than one thread at a time. In
the worst case
scenario this problem creates a race condition, a condition
where output is dependent on the timing of events, and
unexpected and unnoticed error results occur[16].

Another problem with OpenMP is the overhead created by
parallelization[16].
For loops that do not include a certain
number of operations, a parallel implementation of the code
will run less efficiently than a serial version. This is because
thread creation and data allocation in OpenMP is a time
costly process that serial programs

do not include. So if the
overhead of thread creation is greater than the performance
gained by parallelism than a serial version of the code would
be more efficient. The overhead in some cases greatly
reduces the benefits of OpenMP, but many times the be
nefit
of parallelism is much greater than the overhead[17].


General parallelism and maximum speed up:
Amdahl's law


OpenMP is an extension of serial languages, it is a
programming tool meant to extend the usability of other
languages. For that reason it c
omplicates programming in
any of those languages, and it is not an ideal solution to
general parallel programming. Serial languages are great for
the domain they were created in: serial computing. Parallel
computing however is a different method altogether

and is
not
entirely

compatible with existing methods of
programming. A new language needs to be created to make
parallel programming a simpler process because the way
parallel programs are currently written is inefficient.

Some constraints for a new langu
age are created by
Amdahl's law. This law is a mathematical formula to
calculate the speedup of computing with any number of
cores given a program that has semi
-
parallelized code. The
law shows that the speedup is greatly limited by the
percentage of code
that is sequential. In fact, speedup with
an infinite amount of cores simply approaches the limit of
the time taken for the execution of sequential code.



FIGURE 2

This graph shows how the increase in speed of an
algorithm is influenced by the number of cores, and the
percentage of parallelized code[18].


The importance of general parallel algorithms that allow
the percentage of parallelized code to increase is cle
arly seen
in this graph. With OpenMp programming in serial is many
times simpler than programming in parallel. Due to this,
programmers using the software are inclined to use more
serial code than parallel. However the serial operations
severely bottleneck

the performance of a parallel computer.
Lunan Li

Stephen Klein



6


This presents a problem; as more and more cores are added
to CPUs there will be smaller and smaller performance
boosts. The only way to overcome the problem is to
parallelize a very high percentage of the code. If o
ne
hundred percent of the code was parallelized, the speedup
according to Amdahl's law would simply be the number of
cores being used[3]. However, it is nearly impossible to
accomplish one hundred percent parallelization, but what
about 99.9%? A new comput
er language might make that
goal attainable.


PARASAIL


The need for a parallel computer language was
recognized by the software developer Adacore.

Adacore is
in the process of designing a parallel computing language
called
Parallel Specification and Implementation Language
or ParaSail. ParaSail is a language designed from the ground
up to run programs in parallel. Adacore made ParaSail so
that the rules of the language require that any process can be
broken up into really sm
all chunks called picothreads. These
pic
othreads are then sent to the
queue to be operated on by
each processor core in a computer. So if you have eight
cores and eight hundred picothreads, one hundred
picothreads are packaged as instructions and sent to e
ach of
the eight cores. The benefit of picothreading is that it will
easily scale up as more and more cores are added to a
processor, and cores can share picothreads if one core is
overloaded, so the work is more evenly distributed between
cores.

Parasail
allows for the creation of picothreads by
removing several things that are common with serial
computer languages. The first aspect they
removed

is global
variables[19]. Each variable in ParaSail only exists within
each operation performed in the program. T
his eliminates
the race conditions and serialization of parallel programs
with global variables. They also removed parameter aliasing
which is when a certain piece of data can be accessed by
more than one name. This is similar to

the global variable

condit
ion,

however these parameters can cause problems
within functions and loops as well. A third thing they
removed from conventional code to make ParaSail is explicit
threads[20]. Programming in parallel with Java or OpenMP
requires the programmer to declare
what processes will get
its

own thread, and when new threads are created and
terminated. Explicit threads would clearly interfere with the
picothreading process, and explicitly declaring threads is
what complicates the current programming of parallel
progr
ams[21].

ParaSail also created a different memory system t
o

most
current languages have[19]. In Java and C information is
stored in global dumps that are available to all parts of the
program. Para
S
ail removed this and instead created local
data dumps for
each thread. This optimizes cache read times,
and it removes all possibilities of race conditions in
concurrent threads (similar to the global variable race
conditions). They also changed their data storage method by
removing editable pointers. Pointers ar
e basically variables
whose value is the location of data from another variable. If
these were allowed it would once again mean that multiple
threads could operate on the same data locations, and this
could create unexpected errors and race conditions. Als
o,
pointers are used to create data trees with many levels of
information stored in one large structure. Because

of this,
pointers can cause sharing of memory locations within a
variable, which makes it hard to separate the data into the
chunks necessary to create picothreads[19].

ParaSail may be an excellent parallel programming tool
in the future, but because it is

in its early stages it is very
hard to validate its usefulness as a computer language.
According to Adacore in the segment of word counting
ParaSail code in figure 3


FIGURE 3

This image of ParaSail code shows a simple word counting
function[19].


the

Par
aSail compiler will create as many picothreads as
there are characters in the st
ring being evaluated[19]. Then
t
hese

picothreads can all be resolved at the same time on
many cores. In this case the problem does not seem
obviously parallel, but ParaSail eff
ectively turned this simple
code into a powerful general parallel algorithm. This
generalization means that this program would scale up to
processors with any number of cores and these threads were
not explicitly stated. Overhead in thread creation in para
sail
still exists, however it is not quite as drastic as OpenMP and
parallel programming in Java. Also the ParaSail compiler is
smart enough to write heuristics to detect whether the
overhead caused by picothread creation is greater than the
speedup due to

parallelization[20]. Using the heuristics it
will parallelize or serialize the code accordingly.


IMPACTS ON THE FUTURE OF
COMPUTERS

Lunan Li

Stephen Klein



7



The total scope of the application for parallel computers
is unknown. Parallel computing is a new innovation, so
researc
h and studies on the impacts it may have and the
applications it will have is still in the beginning stages.
Although there are many more examples of specific areas of
computing parallelism will impact, speech recognition and
artificial intelligence are tw
o very well
-
known examples.


Speech Recognition


Speech recognition is one process that is parallel by
nature. Before the advent of parallel computers, speech
recognition Word Error Rate (WER) accuracy was only at
about 60% and transcriptions from speech t
o computer took
much longer than the span of time it took to say the
sentence[22]. Now speech recognition accuracy is over 80%
and the transcription times are much faster. Much of this
boost in performance is because of parallel computing.

In an IEEE confe
rence paper, independent researchers
evaluated the scalability and performance of speech
recognition software on multiple core chips[23]. They tested
first with a sequential algorithm on a four core processor,

then they used the same decoding algorithm, bu
t parallelized
much of the code. The researchers tested using a four core
processor with a very high clock speed, and a thirty two core
processor with a relatively low clock speed. On the four core
processor, the parallelized algorithm took 3.4 times less
to
run than the sequential algorithm, and on the thirty two core
processor with the low clock speed the algorithm was run
10.5 times as fast as the sequential algorithm[24]. This
speedup clearly shows the improvement parallel computing
makes on speech reco
gnition, and it shows how speech
recognition software will scale up in speed as processor
cores increase.

Speech recognition is too inaccurate and slow to use
practically on a consistent basis. The WER is barely under
twenty percent, while human WER is fro
m one to four
percent. A major setback in the progress of speech
recognition technology is decoding times, but parallel
computing will remove this problem giving developers
motivation to now focus on improving accuracy. A computer
of the future might react

primarily to voice commands,
improving the efficiency of human
-
computer interfacing.


Artificial Intelligence


Although our feeble attempts at creating artificial
intelligence have come nowhere near full decision making
capabilities
,

the process of making intelligent decisions is a
parallel process[24]. Parallel computing could therefore aid
in the computation speed of artificial intelligence algorithms.
For each choice an artificial intelligence algorithm needs to
make a large variet
y of factors must be considered. In a
sequential program the algorithm would methodically
proceed through all the factors until arriving at a solution,
but in a parallel program all of the factors can be considered
at once. This is a lot more like how a br
ain or organic
intelligence system works, and that fact makes parallel
computing a much better fit for the development of artificial
intelligence than serial computers[25].

Artificial intelligence has the potential to change many
aspects of society. Most f
actory workers cannot be replaced
by robotic workers because if something goes wrong a robot
cannot make any decision about it, it just detects an error. If
the robot had artificial intelligence then it could easily do
many simple tasks that required decis
ion making abilities,
like sorting

recycling (a process done by assembly line
workers)[8]. Automating these tasks may take the place of
certain unskilled laborers, however it is impossible to make
conclusions as to whether or not artificial intelligence
technology will ultimately benefit th
e job market, or harm it.
Although research is being conducted into artificial
intelligence, we are many years away from its inception as a
common technology; for this reason it is hard to analyze the
effects it may have on society. Artificial intelligence

also has
the ability to take the driver's seat of your car in the
future[8]. Computerized systems could make decisions much
more rationally than human drivers, and they could also
communicate with other cars so that accidents are much
less
common
. In thes
e ways, parallel computing may bring about
change in our society.


CONCLUSION AND ANALYSIS


The implementation of generalized parallel computing
algorithms in an easy to use computer language is an
essential key to the overall development of parallel
compu
ting software. Parallel computing software is an
underdeveloped topic in computer engineering. Due to the
ten year head start multi
-
core processors have on parallel
computing software, developers need to begin taking greater
strides to see this technology
come to fruition.

The performance boosts that parallel computing can
provide are significant only if the computer industry will
change its standards in computer programming. Although
the work left to accomplish scalable parallel computing is
significant, P
araSail may be a stepping
-
stone to better
understanding methods by which scalable parallelism can be
accomplished.

Parallel computing is a very important innovation in the
future of the computer industry. It is likely to be the next big
step in the evolut
ion of the personal computer, and without it
the computer industry would begin to flounder when
Moore’s law comes to a halt. It will allow for the
advancement of other technologies, like accurate speech
recognition and better artificial intelligence capabi
lities.

Despite the benefits parallel computing will bring to
society, its negative impacts on the environment must also
be evaluated.

Although countries such as Japan
, Taiwan,

and
China are providing
retrieval services for hazardous
computer materials
, t
here is still a long way to go to make
Lunan Li

Stephen Klein



8


computer production environmentally friendly. Engineers
should
focus on using

new sustainable materials to substitute
mercury, barium
,

and arsenic.
In order to achieve
environmental sustainability, much more research
must be
done in developing clean alternatives to the harmful
practices of the computer industry. In spite of the harm the
computer industry may cause on the environment, parallel
computing is an important innovation to both the computer
industry and socie
ty as a whole.


REFERENCES


[1] Almasi, George S. (1990)
Highly Parallel Computing.
Redwood City, California: The Benjamin/Cummings
Publishing Company, Inc.

[2]J. Strickland. “How small can CPUs get?”

how stuff
works?
.(Online article).

HowStuffWorks "Transistors on the Nanoscale"

[3] “Amdahl’s Law”
Princeton
(online article).

http://www.pri
nceton.edu/~achaney/tmve/wiki100k/docs/A
mdahl_s_law.html

[4]
Tulane University

http://wiki.ccs.tulane.edu/images/4/40/Speedup_fac3.png

[5](February 2, 2013). “United States unemployment

Rate.”
Trading Economics.
(online article).

http://www.tradingeconomics.com/united
-
states/unemployment
-
rate

[6](July 18, 2012) “Software Developers.”
Occupational
Outlook Handbo
ok.

(online article).

http://www.bls.gov/ooh/Computer
-
and
-
Information
-
Technology/Software
-
developers.htm

[7]M. Kaku. (Mar. 20, 2011). “What happens when
comp
uter stop shrinking?”
SALON
. (online article).

http://www.salon.com/2011/03/19/moores_law_ends_excerp
t/

[8]B, Turner. “10 Ways Artificial Intelligence Will Affect
Our Lives”
Discovery
.
(Online article).

http://dsc.discovery.com/tv
-
shows/curiosity/topics/ways
-
artificial
-
intelligence
-
will
-
affect
-
our
-
lives.htm

[9]M. Kaku
. (March 7, 2012). “Computer's of the post
silicon era.”
BigThink
(online article).

http://bigthink.com/videos/tweaking
-
moores
-
law
-
computers
-
of
-
the
-
post
-
silicon
-
era
-
2

[10]C. C. Mann. (May 1, 2000). “The End of Moore’s
Law?”
MIT Technology Review
.(Featured st
ory).

The End of Moore's Law? | MIT Technology Review

[11]G. Chen, G. Sun, Y. Xu, B. Long. (2009). “Integrated
research of parallel computing : Status and future”
Chinese
ScienceBulletin
. (Journal Article).

http://link.springer.com/article/10.1007%2Fs11434
-
009
-
0261
-
9?LI=true#page
-
1

[12]A. Krishna, A. Samih, Y. Solihin. (Feb, 2012). “Data
Sharing in Multi
-
Threaded Applications and its Impact

on
Chip Design”
Dept. of Electrical and Computer
Engineering, North Carolina State University.
(Conference
paper).
www4.ncsu.edu/~aasamih/ispass.pdf

[13]M. Kanellos. (June 14, 2007). “Intel readies ma
ssive
multicore processors”
CNET
. (Online article).

http://news.cnet.com/Intel
-
readies
-
massive
-
multicore
-
processors/2100
-
1008_3
-
6190856.html

[14
]
M. Ada
mson, R. Hamilton, K. Hutchison, K.
Kazmierowski, J. Lau, D. Madejski, N. MacDonald.

(April, 2005).


Environmental Impact of Computer
Information Technology in an Institutional Setting:

A Case Study at the University of Guelph


(online article).

http://www.uoguelph.ca/isc/documents/050602environcs_00
0.pdf

[15]W. V. Winkle. (Nov. 1, 2010). “After Effects CS5”
tom’s
Hardware.
(Online article).

http://www.tomshardware.com/reviews/adobe
-
cs5
-
cuda
-
64
-
bit,2770
-
6.html

[16]C. Yang. (May, 2011). “Performance Comparison with
OpenMP Parallelization for Multicore Systems”
IEEE
.(Conference publications).

IEEE Xplore
-

Performance Comparison with Ope
nMP
Parallelization for Multi
-
core Systems

[17] T. Mattson. (Feb. 12, 2008). “Writing Parallel
Programs: a multi
-
language tutorial introduction”
Intel
Software
.(Online article).

http://software.intel.com/en
-
us/articles/writing
-
parallel
-
programs
-
a
-
multi
-
language
-
tutorial
-
introduction

[18](February 27, 2008).
wikimedia

http://commons.wikimedia.org/wiki/File:AmdahlsLaw.png

[19]D. Patterson. (July, 2010). “Less is more with Multi
-
core”

IEEE
. (online article).

http://www.embedded.com/design/programming
-
languages
-
and
-
tools/4375616/ParaSail
--
Less
-
is
-
more
-
with
-
multicore

[20]D. Selwood. (July 18, 2012). “Does the World Need a
New Programming Language”
EEJOURNAL
. (journal
article).

Does the World Need a New Programming Language?

[21]T. Taft. (Sept. 20, 2012). “
ParaSail: A Pointer
-
Free Path
to Object
-
Oriented Parallel Programming

Microsoft
research.
(online article).

ParaSail: A Pointer
-
Free Path to Object
-
Oriented Parallel
Programming
-

Microsoft Research

[22]. (2011). “
Rest in Peas: The Unrecognized Death of
Speech Recognition

ROBERT FORTNER

(online article).

http://robertfortner.posterous.com/the
-
unrecognized
-
death
-
of
-
speech
-
recognition

[23]K. You, J. Chong, Y. Yi, E. Gonina, C. Hughes, Y. Chen,
W. Sung, K. Keutzer. (Nov, 2009). “Parallel Scalability in
Speech Recognition”
IEEE.
(Magazine article)

http://pcl.intel
-
research.net/publications/ParallelSpeech.pdf

[24]S. Madan. “A Benchmark for Artificial Intelligence
Applications on Parallel Computers
-

BEAP”

IEEE

(Conference Publication).

http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=6
27117&tag=1

[25](Jan. 24, 2011). “
An intelligent approach to artificial
intelligence

softtalk blog

(blog).

Lunan Li

Stephen Klein



9


http://softtalkblog.com/2011/01/24/an
-
intelligent
-
approach
-
to
-
artificial
-
intelligence/


ACKNOWLEDGEMENTS


We could like to thank Coreen Klein for grammar a
dvice,
and Richard Gabrislon for information on the topic.