ProposalV6x - University of Colorado Colorado Springs

gazecummingNetworking and Communications

Oct 26, 2013 (3 years and 10 months ago)

185 views

Master’s Pr
oject Proposal



Page
1

of
12



Investigating Serial Attached SCSI (SAS) over TCP (eSAS)

and benchmarking eSAS
prototype
against legacy SAS



Master’s Project
Proposal


b
y


Deepti
Reddy


As part of the requirements for the degree of


Master of
Science
in
Computer Science



University
of Colorado, Colorado Springs





C
ommittee Members and Signatures



Approved by






Date


______________________________



_______________


Project Advisor:

Dr. Edward Chow


______________________________



_______________


Member:

Dr.
Xiaobo Zhou



______________________________



_______________



Member:

Dr. Chua
n Yue










Master’s Pr
oject Proposal



Page
2

of
12

TABLE OF CONTENTS

1.

ABBREVIATIONS AND TE
RMS USED

2

2.

ABSTRACT

2

3.

SAS OVERVIEW

3

4.

SAS2 EXPAND
ER ZONING

5

5.

OUT OF BAND COMMUNIC
ATION IN STORAGE PRO
TOCOLS

6

6.

GOAL, MOTIVATION AND

CHALLENGES OF THE PR
OJECT

8

7.

PROJECT TIMELINE

10

8.

PROJECT DELIVERABLES

11

9.

REFERENCES

11


1.

Abbreviations

and Terms
used


SAS


Serial attached SCSI


SAN
-

Storage Area Network


SMP


Serial Management Protocol


SSP


Serial SCSI Protocol


HBA


Host Bus Adapter


HDD


Hard Disk

Drive

2.

Abstract


Serial Attached SCSI

[1
]
, the
successor

of SCSI is gaining
popularity

by leaps and
boun
ds in
enterprise
storage systems. SAS is reliable, che
aper, faster and more scalable

than its
predecessor SCSI
.

One of the limiting featu
res of SAS is
its

distan
ce
limitation
.
A
single
point to point SAS cable connection can cover only around 8 meters.

To scale
topologies to support a large number of devices beyond the native port

count, expanders
are used in SAS topologies

[2
]
.
With zonin
g

[2]

capabilities introduced in SAS2

Master’s Pr
oject Proposal



Page
3

of
12

expanders
, SAS is gaining popularity in Storage Area Networks.
With the growing
demand for SAS
in large topologies
arises the need to
investigate SAS over TCP

(eSAS)

to increase the distance and scalability of SAS.
The
iSCS
I protocol

[3
]

today provides
similar functionality where it sends SCSI commands over TCP. However, SAS drives
and SAS expanders can’t be used in a
n

iSCSI topology

such that

the iS
CSI HBA talks
directly in
-
band to SAS devices

making the iSCSI back
-
e
nd less

scalable and slower than
eSAS
.
T
he iSCSI specificatio
n is

leveraged heavily

for eSAS
.



The goal of this project is to p
rovide research results for future industry specification for
eSAS and iSCSI
. The project involves

investigating

send
ing a set of SAS c
ommands
and
responses over TCP/IP

(eSAS)

to address scalability and the distance limitations of
legacy SAS
.
A

client prototype application

will be implemented to send/receive

a small
set of commands
.

A
server
prototype
application will be implemented that receives
a set
of
eSAS commands
from the client
and sends eSAS responses.

The server application
mocks a simple eSAS Initiator while the client application mocks a simple eSAS target.
The performance of eSAS will be com
pared to legacy SAS

to determine the speed and
scalability of eSAS
.


3.

SAS

Overview


Serial attached SCSI

(SAS)

is a protocol

used to access storage devices
[1
]
. Storage
devices include tape drive, hard disk drives, solid state drives and disk systems.

SAS i
s the successor of SCSI

technology and is becoming wide
-
spread as performance
requirements and
addressability

exceeds well beyond what legacy SCSI supports.



In 2004, SAS interfaces were initially introduced at 3Gb/
s. Currently, supporting 6Gb/s
and
moving to 12Gbps by 2012, SAS interfaces have significantly increased

the available
bandwidth offere
d by legacy SCSI storage systems
.

Though fibre channel is more
scalable, it is a costly

solution for use in a SAN
.

Table 1 compares SCSI, SAS and Fibre
Chan
nel technologies.




SCSI

SAS

Fibre Channel

Topology

Parallel Bus

Full Duplex


Full Duplex

Speed


3.2 Gbps

3 Gbps, 6Gbps

Moving to 12 Gbps

2 Gbps

4 Gbps

Moving to 8 Gbps

Distance

1 to 12 meters

8

meters

10 km

Devices

SCSI only

SAS & SATA

Fibre Channel
only

Number of

Targets

14 devices

128 expanders

by 1 expander.

>16,000 with
cascaded expanders

127 devices in a
loop
. Switched
fabric can go to
millions of devices

Connectivity

Single
-
port

Dual
-
port

Dual
-
port

Drive Form Factor

3.5”

2.5”

3.5”

Master’s Pr
oject Proposal



Page
4

of
12

Cost

Low

Medium

High





Table 1



Comparing SCSI, SAS and Fibre Channel


An
initiator,

also called a Host Bus Adapter or Controller
,

is used

to send SAS commands
to SAS targets.
SAS controller devices have a limited number of ports.

A narrow Port in
SAS consisting of a single port is referred to as a PHY

[
1
]
.


Expander devices in a SAS domain facilitate communication between multiple SAS
devices.

Expanders have a typical
port

count of 12 to 36 ports while SAS controllers have
a
typical port count of 4
-
16 ports.

Ex
panders can be cascaded as well to increase
scalability.
One of the most significant SAS feature is the transition from 3.5” dr
ives to
2.5” drives. This helps

reduce floor space and
power consumption
[2
]
.

Another advanta
ge
of using SAS targets is that a SAS hard drive is dual
-
ported providing a redundant path to
each hard drive in case of an Initiator/Controller fail
-
over.

Also, unlike SCSI, SAS
employs a serial means of data transfer like fibre channel

[
4
]
.

Serial interf
aces are known
to reduce crosstalk
and related signal inte
grity issues.


Figure 1

shows an example of a typical SAS topology. SAS commands originate from the
HBA driver

(MPT

driver
)
and are eventually sent to the HBA.

The SAS controller
/HBA

sends commands
to the disk d
rives through the expander for expander attached
targets
/drives
. The target replies to the SAS command through the expander. The
expander simply acts like a switch and routes the commands to the appropriate target and
routes the responses from

a particular target to the Controller.



Master’s Pr
oject Proposal



Page
5

of
12


Figure 1



A typical SAS topology


The three protocols used in
SAS are Serial Management protocol, Serial SCSI Protocol
and SATA Tunnel Protocol.

Serial Management Protocol

(SMP)

[1
]
is used to discover
the SAS
topology and to perform system management.

The
Serial SCSI Protocol (SSP)

[1
]
is used to send SCSI commands and receive responses from SAS targets.

SATA
Tunnel Protocol (STP)

[1
]
is used to communicate with SATA targets in a SAS
topology.


4.

SAS
2
Expander
Zoning


As SAS is gaining popularity in
small
storage area networks (SAN)
, there i
s a need to
manage

device traffic through zoning

in a similar fashion
as
done in Fibre Channel
networks
.

Zoning

is an expander feature introduced in Generation 2 SAS expander
s

[2]
.

SAS2 zoning provides functionality to protect targets against unauthorized access, traffic
segregation

and resource sharing

[5
]
.

In simple words, zoning allows storage systems to
be divided and used by many hosts/controllers.




Master’s Pr
oject Proposal



Page
6

of
12



Figure
2



Zoning
in an expander

where HBA A only sees 2
HDDs and HBA B also only sees 2 HDDs


In

Figure 2
, the expander will setup zoning such that HBA A will discover 2 HDDs and
HBA B will also discover 2 HDDs.

With zoning disabled in the expander, each HBA will
di
scover 4 HDDs.



5.

Out of band communication

in storage
protocols


As expander devices are
gaining attention in small SANs, the need for expander
management over Ethernet has become important.


In SAS1,
expanders did not have support t
o receive
SAS
commands out
-
of
-
band.

SAS
1

controllers/HBAs

would need to
send
commands to an expander in
-
band

even for
expander diagnosis and management
.

SAS HBAs/controllers have a lot

more complex
functionally than expanders.


Diagnosing

issues by sending

commands in
-
band to
expanders made it harder and time
-
consuming to root cause where the problem is in the
SAS topology. Also, managing
expanders via

in
-
band lacked the advantage of remotely
managing expanders

via out
-
of
-
band over

Ethernet
.

With the gainin
g popularity of
zoning, expander vendors have implemented support for limited SMP zoning commands
out
-
of
-
band via Ethernet in SAS2

[1
]
.

A client management application is used to send a
limited set of SMP commands out
-
of
-
band to the expander. The expander
processes the
Master’s Pr
oject Proposal



Page
7

of
12

commands and sends the SMP responses out
-
of
-
band to the management application.

Figure 3

shows the communication between the client management application and the
expander.




Figure 3



SMP Request & Response Sequence Diagram.

This figure
doesn’t show the SAS primitives exchanged on the SAS wire



A few usage examples
of the client management application
include:

a.

Resetting a bad drive using the out
-
of
-
band mechanism


Instead of resetting
the drive in
-
band by sending a SMP Phy Control [Seri
al Attached SCSI (SAS
2) Specification] from the HBA, the out
-
of
-
band mechanism can be used to
send a SMP Phy Control to reset a misbehaving drive.

b.

Inducing test cases more conveniently while running IO


Instead of
physically removing and inserting drive
s, a drive removal and insertion can be
simulated by sending zoning commands out
-
of
-
band to the expander.

c.

Isolating issues more easily


On failing commands, the root cause of the
failure can be more easily determined as the complexity of the HBA and SAS
d
river is eliminated.



iSCSI and Fibre Channel already provide the

capability of communicating over

TCP/IP

[4][7][8]
. iSCSI protocol describes a means of

transporting of the SCSI packets over
TCP/IP

where the protocol encapsulates a SCSI subsystem within a TCP/IP connection

[
6
]
. Therefore, iSCSI takes advantage of existing internet infrastructure, internet
management facilities as well as addresses distance limitations

[4] [
7
]
.


Communication b
etween the initiator and target in iSCSI protocol
occurs over one or
more TCP connections.

However, SAS devices can’t be
directly
used in SCSI topologies.
Figure 4

describes a typical iSCSI frame embedded in an Ethernet frame.


Master’s Pr
oject Proposal



Page
8

of
12



Figure 4



iSCSI header a
nd data
embedded in an Ethernet

frame



Fibre Channel as well has support to send fibre channel commands, data and responses
over TCP/IP

[4]
.
The fibre channel proto
col is a lot more complex than the

simple point
-
to
-
point protocol such as SAS.
U
sing fibre channel as a fabric proves to be a very
expensive
solution [4]
[9]
.
Therefore, sendi
ng SAS over fibre is not a good

solution to
over
-
come the distance limitations of SAS.


6.

G
oal, Motivation and Challenges
of the
Project


The goal

of this project i
s to investigate sending
a set of
SAS commands, data and
responses over TCP/IP.
Since Ethernet contains its own physical layer, this eliminates the
need for the SAS physical layer overcoming the distance limitations of SAS.
Using eSAS
also overcomes the
6G and
12G limitations of
SAS2 and
SAS3

respectively
.

As
mentioned earlier in this paper, the main challenge of developing an eSAS client/server
application is that there is no standard specification for eSAS.


The Figure belo
w shows a typical usage of eSA
S

to

expand scalability, speed and
distance of
legacy
SAS.

Master’s Pr
oject Proposal



Page
9

of
12




Figure 5



eSAS Topology


The

back
-
end of eSAS is an eSAS
HBA

tha
t can receive eSAS commands,
strip off the
TCP header and pass on the SAS command to the expander and drives.

The back
-
end of
eS
AS will

talk in
-
band

to the SAS expanders and d
rives
.

The remote eSAS
Initiator
communicates with the eSAS target by sending eSAS commands.
Figure 6 shows the
typical SSP request/response
read data
flow.
The
eSAS
SSP Request is initially
sent by
the eSAS
Initiator to the eSAS Target over TCP
.
The
eSAS Target strips off
the TCP
header and sends the SSP

request using the SAS Initiator block on the eSAS Target to the
SAS
expander.

The SAS expander sends the data frames and the SSP Response to the
eSAS Target.

Finally, the eSAS Target embeds the SSP data frames and response frame
over TCP and sends the frames to the eSAS Initiator.




Master’s Pr
oject Proposal



Page
10

of
12

=


Figure 6



eSAS SSP

Request & Response

Sequence Diagram
.

This figure doesn’t show the
SAS
primitives exchanged on the SAS
wire



The
eSAS
client
and server
prototype
applic
at
ion will be written in C/C++. The

client
a
pplication will send certain eSAS

commands to the server application which will
process and send responses to the client.
Benchmarking of eSAS vs SAS will be
inve
stigated.

A SAS protocol analyzer
and other IO tools
can be used to calculate the
time to send
and receive IOs on the SAS wire.



7.

Projec
t Timeline


Researching
SAS

protocol


(SMP, SSP
, STP
)




1.5


months


Completed



Understanding iSCSI

at a high
-
level




15

days


Completed


Investigating how to embed




SAS

traffic over TCP/IP



7

days



Completed


Research how to time IOs on

the SAS wire




7 days



November 22
nd


Proposal

Completion



7

days



November 18
th



Master’s Pr
oject Proposal



Page
11

of
12

Phase I
Software Development

Software development
o
f
the

cl
ient

+


server prototype using C/C++


1.
5 months


January 10
th
, 2011


Fin
al Project Report

+
Code deliverable



1
5

days


January 28
th
, 2011



Final Presentation




5 days



January 31
st
, 2011


8.


Project Deliver
ables



Research
report on th
e design and implementation
of
the client and server
application



Research report on t
he b
enchmarking results of eSAS prot
otype
app
vs
legacy
SAS.



The
software client
simulator

developed for the project.



The software server simulator developed t
o test the client application



9.

References



[1] T10/1760
-
D Information Technology


Serial Attached SCSI


2 (SAS
-
2),


T10, 18 April 2009,


Available from
http://www.t10.org/drafts.htm
#SCSI3_SAS


[2
] Harry Mason,


Serial attached SCSI Establishes its Position in the Enterprise,


LSI Corporation,


available from
http://www.scsita.org/aboutscsi/sas/6GbpsSAS.pdf


[3] J Satran, K Meth, C. Sapuntzakis, M. Chadalapka, E. Zeidner,


RFC 3720
Internet Small Computer Systems Interface (iSCSI) Specification


IETF, April 2004,


available from
http://www.ietf.or
g/rfc/rfc3720.txt


[
4
] Cai, Y.; Fang, L.; Ratemo, R.; Liu, J.; Gross, K.;
Kozma, M.;


A test case for 3Gbps serial attached SCSI (SAS)



Test Conference, 2005. Proceedings. ITC 2005. IEEE

International, February 2006,


available from
http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=1584027



[
5
] Heng Liao, Tim Symons, Rachelle Tant,


Managing Access Control Through Zoning
,

Master’s Pr
oject Proposal



Page
12

of
12


PMC
-
Sierra Inc., Spetember

2005,


available from
http://www.scsita.org/aboutscsi/sas/WP_PMC_Sierra_SAS_0905.pdf


[
6
] Kalmath Meth, Julian Satran,

Design of the iSCSI Protocol
,

Mass Storage Systems and Technologies,

2003. (MSST 2003). Proceedings.
20th
IEEE/11th NASA Goddard Conference on

Mass Storage Systems
and Technologies,


April 2003,


ava
ilable from
http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=1194848&tag=1




[
7
] Stephen Aiken, Dirk Grunwald, Andrew R. Pleszkun, Jesse Willeke,


A Performance Ana
lysis of the iSCSI Protocol


20
th

IEEE/11
th

NASA Goddard Conference on Mass Storage Systems and


Technologies, 2003,


available from


http://www.st
orageconference.org/2003/papers/20
-
Aikens
-
Performance.pdf



[8] M. Rajagopal, E. Rodriguez, R. Weber
,


RFC 3821 Fibre Channel over TCP/IP (FCIP) standard,


IETF,
July 2004,


available from
http
://tools.ietf.org/html/rfc3821


[9] BLi Bigang, Shu Jiwu, Zheng Weimin,


SCSI Target Simulator Based on FC and IP Protocols in TH
-
MSNS
*


Department of Computer Science and Technology, Tsinghua University, Beijing



China, 2005