An Evaluation of GOAL and GAL Routing Algorithms: Aimed at Future Research

brrrclergymanNetworking and Communications

Jul 18, 2012 (5 years and 1 month ago)

272 views

An Evaluation of GOAL and GAL Routing
Algorithms: Aimed at Future Research


Christopher Lydick

Kansas State University

Department of Electrical and Computer Engineering


February 25, 2007


Abstract


Efficient routing algorithms are now becoming more and mo
re necessary for productive
Massively Parallel Processing (MPP) Computers, such as the Red Storm Supercomputer
located at Sandia National Laboratories, NM. Some current implementations of MPP
computers have used source
-
based routing, which during nodal fa
ilures, requires the
entire network to halt and recompute routing tables. It should be considered that
dynamic routing algorithms such as GOAL and GAL have performance characteristics
(throughput and latency), which rival source
-
based routing, but they la
ck data on how
well they perform when nodal failures exist. The work stemming from this research will
aim at evaluating GOAL and GAL routing algorithm performance in terms of nodal
failures. This paper serves as a basis for that research.




I.

Introduction



Not too long ago, focus for increasing performance within computing clusters was
placed on the processing speed. Now the focus has shifted into message passing and
routing algorithms that can keep up with the processing speeds of these large clusters.

The Cray Red Storm MPP (Massively Parallel Processing) computer at Sandia National
Laboratories, for example, uses source
-
based routing [5], which looks up all routes on a
precomputed table [4]. While this static routing implementation may work well unde
r
perfect conditions, the problem arises when a node or link fails. Large clusters such as
Red Storm with its 12,960 processing nodes [3] then must recalculate routes to and from
each node. This limits its availability and also hinders the productivity.



Focus for this paper will include introducing two potential candidates for a
dynamic routing implementation for large computing clusters such as Red Storm.
Hopefully with work obtained from this paper and in future work, it will be possible to
show an
implementation of routing which will prove to be stable, reliable, and more
productive than the source
-
based routing which is currently being implemented.


The rest of the paper will continue as follows: first, there is a discussion some of
the previous wo
rk in routing algorithms for the toroid
-
mesh networks in section II, then
move on to the GOAL (Globally Oblivious Adaptive Locally) routing algorithm in
section III. From there the paper will introduce the GAL (Globally Adaptive Load
-
Balance) routing algo
rithm in section IV, and finalize with an outline for future work and
conclusions in section V. References are listed in section IV.


II.

Previous Work



The previous work that this paper will focus on the work of [1] and [2]. These
two papers sum up a lot
of the work that has proven helpful within the realm of routing in
tori and mesh networks. This work seems particularly attractive to those currently
implementing static routing algorithms but may need more tangible data in order to
stimulate a change.


A lot of focus for these two routing algorithms has been to evaluate their
performance in terms of throughput and latency. I propose that this is not enough of a
benchmark when considering large MPP architectures. Nodal failures seem to be an area
where
previous work has not considered.


Before the introduction of the GOAL and GAL routing algorithms, CHAOS and
Minimal Adaptive routing algorithms were considered the best for dynamically routing
packets through a network [2]. Minimal Adaptive routing occu
rs by choosing the
minimum routes from source to destination using only information about the network
(such as output queues), making decisions at each hop [6]. It does not use any global
information to assess global congestion.


The Chaos routing algorit
hm uses an idea referred to as a deflection routing
scheme, by randomly granting contending packets access to a channel. Packets who lose
the allocation to the channel are misrouted to a free output port, which may or may not be
productive [6].


Accordi
ng to the work of [1] and [2], the GOAL and GAL routing algorithms
outperform CHAOS and Minimal Adaptive. Because of this assumption, this paper will
mainly focus on the GOAL and GAL routing algorithms.


III.

GOAL



The GOAL routing algorithm, Globally Oblivio
us Adaptive Locally, was
introduced in [1] and has been the basis for improving routing algorithms on networks
with high bisection bandwidth and high path diversity [1].


GOAL works by routing a packet from source
s
= {s
1
, s
2
, s
3
, …, s
n
} to destination
d
= {d
1
, d
2
, d
3
, …, d
n
} by obliviously choosing the direction to travel in each dimension
to as exactly balance channel load [1], sending the packet into the direction which is still
productive, but has the shortest queue. It differs from Minimal Adaptive
routing
algorithm by assigning particular quadrants of a network a computable probability based
on the
k
parameter (determined by the size of the network) as well as the distance
required in the
i
th
direction from source to destination [1].


For instance,
if there needs to be a packet sent from source node
s
= (1,2) to
destination
d
= (5,5) then there are multiple shortest paths. Depending on queue length at
each intermediate node, a decision is made to route the packet onto a particular interface.



Fig
ure 1. A section of a 16
-
ary 2
-
cube network graph

from source (1,2) to destination (5,5). Adapted from [1].



At the beginning of the path, at source
s
, it examines the channel queues for each
of its interfaces that would propagate the packet in the co
rrect direction. This is
designated as a (+1,+1), which means it needs to be routed in a positive
x
direction, and a
positive
y
direction (if considering the graph resides in a standard x,y plane). This is also
known as the (+1,+1) quadrant. After the no
de calculates which direction(s) it is able to
route in, keeping in mind the minimum path, it routes it only based upon its queue length.


If at source (1,2) the (+1, 0) queue (indicating the
x
direction queue) is shorter
than the (0,+1) queue (indicatin
g the
y
direction queue), then the next intermediate node
will be (2,2).


By using 3 virtual channels per unidirectional physical channel in a similar
configuration as the *
-
channels algorithm [1], GOAL is touted as being deadlock free.
As explained in [1
], there exist two types of channels * and non*. The packets will move
through the *
-
channels only when progressing in the most productive direction. The non
-
* channels are usable at any time and consist of *a and *b channels. The *a channel is
used if
the packet has not wrapped around, and *b is used if the packet has indeed
wrapped around the network (considering a fully connected torus network). Because of
this implementation, the channels are acyclic and are proven to be deadlock free. For the
spec
ific proof, see [1].


In terms of performance evaluation, the authors of [1] have compared the GOAL
algorithm against other well
-
known algorithms. These are: Valiant’s Algorithm, which
routes packets to a random node
q
anywhere in the network (phase 1)
and then to the
destination (phase 2) [1]; Dimension Order Routing which routes minimally in the x
-
dimension first, then the y
-
dimension, then any further dimensions similarly [1]; Two
Phase ROMM which routes to a random node
q
in the minimal quadrant, the
n to the
destination [1]; Randomized Local Balance which chooses a quadrant
Q
to route to in
according to a weighted probability distribution, then route within
Q
first to a random
intermediate node, then to the destination, randomizing the order of matchi
ng dimensions
[1];
Chaos Routing Algorithm as explained earlier; and finally the Minimal Adaptive
routing algorithm which always routes in the minimal quadrant, routing adaptively within
it [1], [6].


Using those other algorithms as benchmarks for perfo
rmance of the GOAL
algorithm, [1] evaluates their performance against adversarial traffic patterns. These
include Nearest Neighbor, Uniform Random, Bit Compliment, Transpose, Tornado, and
Worst
-
Case traffic patterns [1].


After assessing the traffic pat
terns and ranking GOAL against the other routing
algorithms, the authors concluded that GOAL indeed achieves high
-
throughput on the
adversarial traffic patterns. It met or exceeded the throughput of the other routing
algorithms [1].


IV.

GAL



Unlike the pr
evious discussion, the Globally Adaptive Load
-
Balanced (GAL)
routing algorithm uses global information to make routing decisions [2]. It senses global
congestion using segmented injection queues. By monitoring traffic within quadrants of
the network, GAL
first routes minimally while congestion is low, but when congestion
increases, routes non
-
minimally within quadrants with lower congestion.


As shown in Figure 2, the network can be broken up into multiple quadrants, four
in this case (for a 2
-
dimensional
graph). The minimal quadrant is quadrant I, meaning
that the minimal path(s) exist within this quadrant. But if congestion occurs and is noted
globally within this quadrant, then routing will actually take place non
-
minimally, to
quadrant II, III, or IV
.




Figure 2. Quadrants in a k
-
ary 2 cube

Torus network. Adapted from [1].




The quadrant to route in is chosen using two computations. First, it is evaluated
which quadrant is the minimal quadrant (in the case of Figure 2, this would be quadrant

I), and then, it measures whether or not that specific quadrant has an occupancy of traffic
below some threshold,
T
. If said quadrant is above the threshold, it is discarded as a
potential routing quadrant, and so the source chooses to route within an al
ternative
quadrant. From that point on, routing is done minimally within the quadrant selected.


This implementation also requires three virtual channels per unidirectional
physical channel, just like GOAL. By using the same *
-
channel configuration as
GOAL,
the proof is the same as proposed earlier in this paper, available in [1].


In terms of performance, [2] introduces a report card of how well throughput
factored into each of the cutting
-
edge routing algorithms. Table 1 below shows such
values.




CHAOS

MIN AD

GOAL

GAL

Θ
NN

A (4.0)

A (4.0)

C (2.33)

A (4.0)

Θ
TOR

C (0.36)

C (0.33)

A (0.53)

A (0.53)

Θ
avg

C (0.55)

B (0.63)

A (0.67)

A (0.73)

Latency

A (4.4)

A (4.4)

C (6.2)

A (4.4)

Stability

F (No)

P (Yes)

P (Yes)

P (Yes)


Table 1. Report Card of F
our Adaptive Routing Algorithms.


Θ
’ indicates throughput. Borrowed from [2].



Based upon the table and the authors’ evaluation of GAL’s performance, this
routing algorithm seems to perform very well under adversarial traffic patterns [2].
Because of th
e need for low latency and high throughput, it’s noted that GAL seems to be
a great competitor for networks that require these considerations.



V.

Future Work and Conclusions



The work from [1] and [2] has opened up many doors for research and
development
in the area of adaptive routing algorithms. In terms of national laboratories
such as Sandia and Lawrence Livermore (among others), this type of research proves
appealing because of the algorithms’ usability on tori and mesh networks. Considering
Red St
orm’s last known routing implementation of source
-
based routing [5], substantial
data is required in order to persuade a change in routing techniques to a more dynamic
and more reliable alternative.


The work stemming from this paper will include catering
more to the needs of
Sandia National Laboratories and Red Storm MPP (Massively Parallel Processing)
supercomputer by taking the information proposed by [1] and [2], and adding data
obtained from GOAL’s and GAL’s performance in terms of nodal failures. Bec
ause of
the large number of processing nodes, this is a real issue within large MPP computers.
Nodal failure probabilities may be low, but when considering over 10,000 processing
nodes, the failure of one or more nodes can really cripple computability whe
n source
-
based routing is used. The alternative, a dynamic and more reliable routing algorithm
may exist within or stemming from the GOAL and GAL routing algorithms. Future work
from this will expand on this thought.


In order to test the nodal failures
in terms of GOAL and GAL, the first thing to
consider would be simulating such an environment. By using simulations, the goal
would focus on benchmarking the GOAL and GAL routing algorithms compared to
Chaos, Minimal Adaptive, as well as the Source
-
based
routing in terms of availability,
throughput, and latency when random nodal failures exist.



VI.

References


[1] A. Singh, W. Dally, A. Gupta, B. Towles.
GOAL: A Load
-
Balanced Adaptive
Routing Algorithm for Torus Networks
. Proceedings of the 30
th
Annu
al International
ASYmposium of Computer Architecture, IEEE, 2003.


[2] A. Singh, W. Dally, B. Towles, A. Gupta.
Globally Adaptive Load
-
Balanced
Routing on Tori
. IEEE Computer Society, 2004.


[3] Sandia National Laboratories, Computation, Computer
s, Information and
Mathematics.
Red Storm
. <
http://www.cs.sandia.gov/platforms/RedStorm.html>.
Retrieved: 22 Feb, 2007. Last modified: 16 Feb, 2007.


[4] W. Dally, B. Towles.
Principles and Practices of Interconnection Networks
.
Chapter 11, Routi
ng Mechanics. © 2004.


[5] W. Camp.
Petascale Computing Architectural Requirements.
Sandia National
Laboratories, Computation, Computers, Information and Mathematics. SOS
-
10
Workshop, Maui, Hawaii. March 2006.


[6] W. Dally, B. Towles.
Principl
es and Practices of Interconnection Networks
.
Chapter 10, Adaptive Routing. © 2004.