Hosted by
IP Storage: Best Practices
Marc Staimer, President & CDS
Dragon Slayer Consulting
marcstaimer@earthlink.net
503
-
579
-
3763
Hosted by
Agenda
IP Storage Level Setting
File vs. Block Storage
FCIP, iFCP & iSCSI
Fiction & Facts about iSCSI
Storage Replication over WANs
Considerations for Designing IP Storage Networks
Questions
Hosted by
IP Storage Level Setting
There are 3 types:
•
Block Storage
iSCSI
•
File Storage
NAS
•
Storage over WAN for business continuity = Block
iSCSI
FCIP
iFCP
Hosted by
IP Storage Level Setting
iSCSI Storage
•
Block
-
based external storage on Ethernet
Vs. SCSI, USB, 1394, or FC
NAS or Network Attached Storage
•
File
-
based storage = NFS & CIFS
•
No different than any other file server
•
Requires block storage behind it
Hosted by
Hosted by
By show of hands, what is
acceptable packet loss for IP
Block Storage on a LAN/WAN?
1)
1%
2)
5%
3)
10%
4)
0%
Hosted by
Acceptable Packet Loss for
IP Block Storage on a LAN/WAN
The Answer is:
Hosted by
Block vs. File Storage
IP block data is unlike any other IP data
•
Overwhelms most current LAN/WAN environments
•
Incredible amounts of traffic
•
Tolerates
“ZERO”
p慣k整 loss
•
Very low latency
File storage = specialized file server
•
NFS & CIFS
•
Higher prioritization is required depending on app
•
Volume of data may overwhelm untuned LAN/WAN
Hosted by
FCIP, iFCP, iSCSI
FCIP
•
Fibre Channel tunneled in TCP/IP
•
IP transport between FC switches
iFCP
•
IP header put on Fibre Channel frames for routing
•
IP connection services for FC devices
iSCSI
•
SCSI
-
3 mapped to TCP
Hosted by
FCIP
Pt
-
to
-
Pt: Becomes “One” FC SAN
•
Disruptions pass SAN
-
to
-
SAN
•
Large FSPF database
•
PSS between SAN sites
•
Gateways between fabrics (blades or boxes)
Ethernet LAN/WAN Switches
Cisco MDS 9216
CNT UESR 3000
Brocade 3xxx
SAN 1
SAN 2
SAN 3
SAN 4
TCP/IP
WAN
Hosted by
iFCP
Pt
-
to
-
Multi
-
Pt
•
Device specific passing only the data that is required
•
Devices can appear in multiple individual SANs
The SANs themselves remain independent
Ethernet LAN/WAN Switches
SAN 1
SAN 2
SAN 3
SAN 4
TCP/IP
WAN
McDATA/Nishan
3300
Hosted by
Parable
Hosted by
Hosted by
iSCSI: Ethernet
-
Based SAN
The Hype
•
Block storage on Ethernet
•
Leverage current infrastructure investment &
knowledge base
•
Lowers cost
•
Eliminates headaches
•
Ubiquitous
•
Makes FC another Ethernet Road Kill
Hosted by
iSCSI Defined
IETF standard protocol
•
Establishes & manages
connections
•
Carries storage (SCSI) blocks
From initiators to storage
targets
Encapsulates SCSI blocks in TCP/IP
•
Then tunneled in Ethernet
•
iSCSI is to Ethernet as FCP is to
Fibre Channel
Network application
•
One infrastructure for LANs,
NAS, & SANs
Ethernet
Frame
TCP/IP
Packet
SCSI
-
3
TCP/IP
iSCSI
NAS: File
Storage
GbE Switch
Mission Critical
IA App Servers
Hosted by
iSCSI Applications
NAS/SAN combined storage units
Entry level SANs
Limited budget SANs
Hosted by
SAN Benefit Assertions of iSCSI
Reduced costs
•
Professional services, implementation, management
& IT Staff time
Reduced complexity
Reduced Management
Increased Interoperability
Elimination of Multiple Networks
Unlimited SAN Distance
Equal or Better Performance
Hosted by
Fiction & Facts about iSCSI
Fiction
•
Cost is lots < Fibre Channel
(FC)
•
Complexity is much < than
FC
•
Uses current infrastructure
•
Requires no storage
knowledge
•
Is as fast as FC
•
Will replace FC in Enterprise
•
Easier to manage than FC
•
Eliminates SAN distance
limits
Latency (delay) is not an
issue
Facts
•
Known technology
•
Costs are relatively < FC
Cycles or hardware
•
Doesn’t require special HW
But benefits from it
•
Latency (Delay) matters
Can’t be > 1 millisecond
•
Deterministic routing
•
Doesn’t require any
-
to
-
any
Hosted by
Rating the iSCSI Value Props
Reduces or Eliminates SAN Professional Services
Lowers SAN Hardware Costs
Simplifies SAN Management
Eliminates Interoperability Issues
Converges SAN/LAN/MAN/WAN Fabric Infrastructure
Extends SANs over unlimited distances
Equal or better performance than FC SANs
Hosted by
iSCSI Reality Check
There are some real cost benefits for:
•
NAS/SAN on the same fabric infrastructure
•
SANs that don’t need the performance of FC
•
Entry SANs that may not even need GigE and TOEs
Hype overshadows reality:
•
GigE NICs with iSCSI and TOEs cost ~ same as FC HBAs
•
FC ports & GigE ports on server motherboards
Makes port cost differences higher for GigE w/TOEs
•
Very low cost simplified FC switches
Have erased much of the infrastructure HW differences
Hosted by
Sample iSCSI Vendors
Switches
•
Cisco
•
Extreme
•
Foundry
•
Enterasys
•
Nortel
•
Lucent
•
3Com
Gateways
•
Cisco
•
McDATA
Silicon
•
Adaptec
•
Alacritech
•
Intel
•
Siliquent
•
QLogic
Storage
NICs
•
QLogic
•
Intel
•
Alacrite
ch
•
Adaptec
•
Emulex
Hosted by
Hosted by
By show of hands, is latency
(delay) important to iSCSI block
storage?
Hosted by
Is Latency Important to IP Block
Storage?
The Answer is:
•
Yes, for the most part
•
It also depends on application
Hosted by
Hosted by
By show of hands, who
believes that TOEs & iSOEs
are an iSCSI block storage
requirement?
Hosted by
Are TOEs & iSOEs an iSCSI block
storage requirement?
The Answer is:
Not necessarily
Hosted by
Spectrum of iSCSI Adapter Solutions
Host
Adapter
Adapter Driver
TCP/IP
iSCSI
SCSI Port to OS
Software iSCSI
“NIC + Driver”
Media Interface
Ethernet
Media Interface
Ethernet
Fast Path TCP/IP
Software iSCSI
With Partial
TCP Off
-
load
TCP/IP
iSCSI
SCSI Port to OS
Media Interface
Ethernet
TCP/IP
iSCSI
Firmware TCP
and iSCSI Off
-
load
SCSI Port to OS
= SW or FW
= Hardware
Media Interface
Ethernet
TCP/IP
iSCSI
Hardware TCP
and Firmware iSCSI
Off
-
load
SCSI Port to OS
Hosted by
iSCSI: No TOE
Definition
•
Std Ethernet NIC
TCP/IP & iSCSI
Host
-
based in drivers
Who
•
Microsoft & Cisco
Advantages
•
Lowest cost
•
NICs available today
•
Easy integration with OS
Disadvantages
•
Lowest performance
•
High CPU load
•
High interrupts
Once/packet
Many/ TCP segment
Adapter Driver
TCP/IP
iSCSI
SCSI Port to OS
Software iSCSI
“NIC + Driver”
Media Interface
Ethernet
= SW or FW
= Hardware
Host
Adapter
Hosted by
iSCSI: Little TOE
Definition
•
NIC w/limited TOE
Packets in order & no frags
Out of order etc. go to OS
Who
•
Alacritech
Advantages
•
Relatively low cost
•
Small layout (low profile card)
•
Good throughput w/pristine
Ethernet
Disadvantages
•
Out
-
of
-
order & frags <
performance
•
Interrupts Once/TCP segment
Many/IO
•
OS interface challenges
Media Interface
Ethernet
Fast Path TCP/IP
Software iSCSI
W/Partial TCP Off
-
load
TCP/IP
iSCSI
SCSI Port to OS
Host
Adapter
= SW or FW
= Hardware
Hosted by
iSCSI: Firmware TOE + iSOE
Definition
•
TCP/IP & iSCSI firmware
On
-
board processors
Who
•
Adaptec, Intel, Emulex,
QLogic
Advantages
•
Flexibility to change code
•
Low CPU load
•
Low interrupt load: < 1/IO
Disadvantages
•
No 10Gb scaling
•
Performance
•
Power, size
Media Interface
Ethernet
TCP/IP
iSCSI
Firmware TCP
and iSCSI Off
-
load
SCSI Port to OS
Host
Adapter
= SW or FW
= Hardware
Hosted by
iSCSI: Hardware TOE + iSOE
Definition
•
Hardware ASIC
TCP/IP bulk data path
iSCSI digest (CRC)
•
iSCSI in processors
Who
•
QLogic, iReady
Advantages
•
Flexibility to change iSCSI code
•
Low CPU load
•
Low interrupt load < 1/IO
•
Performance, scaling to 10G
Disadvantages
•
Complex chip
•
Lack flexibility to change TCP
code
Media Interface
Ethernet
TCP/IP
iSCSI
Hardware TCP & Firmware
iSCSI Off
-
load
SCSI Port to OS
= SW or FW
= Hardware
Host
Adapter
Hosted by
A TOEs Impact on iSCSI
Hosted by
Hosted by
By show of hands, who believes
that iSCSI allows block storage
to go unlimited distance?
Hosted by
Will iSCSI allow block storage to
go unlimited distance?
The Answer is:
Yes & No
•
Latency is the limiting factor
•
Application dependent
Transactions cannot exceed 1ms one way (100
miles)
Asynch replication is not distance dependent
“The speed of light, is not
just a limit, it’s a law.”
Hosted by
Key Block IP Storage Issues
Distance
•
Latency
•
WAN bandwidth utilization of IP
Security
•
Encryption
•
Access
Performance
•
Must be = to, or > than current expectations
Hosted by
Storage Replication over WANs
Issues
•
Good Citizen on Shared TCP/IP WANs
•
Filling the pipe > 50%
•
End
-
to
-
end throughput
Compression
TCP latency
Hosted by
Changing Paradigm for Asynch
Storage Replication
Native Storage GigE interfaces emerging
•
EMC Symm5 and DMX are available today
•
EMC CLARiiON in development
•
Hitachi developing GigE for Lightning and Thunder
Software Replication Apps over native IP
•
Leverages IP WAN already in place
•
Eliminates SAN gateway requirement (FCIP or iFCP)
Significant < cost Mirror/Replication apps
Hosted by
High Speed TCP/IP
Data Transport Challenges
Optimized for
•
Small payloads & relatively short distances
Employs inefficient
•
Error recovery & session management techniques
Delivers poor bandwidth utilization
•
For most high performance applications
•
Usually < 30% efficiency at extended distances
•
Even less as distance and bit errors increase
Hosted by
Cost of Inefficiency
Higher Bandwidth Cost
•
Despite < costs, high speed (DS3, OC3, etc) circuits =
expensive
DSC survey of 200 end
-
users
•
BW = 50
-
70 % of storage replication costs
Operational Inefficiencies
•
Can’t complete within time window delaying production
ops
•
Explosion in data exacerbates the problem
•
Current = specialized equipment & separate networks
Can’t fully leveraging IP infrastructure = > costs
Hosted by
Native GigE Replication: SRDF
Adaptive Copy & SNAP/Asynch
•
Performance degrades starting at ~ 300 miles
At 500 miles performance degradation is noticeable
& significant
Ethernet LAN/WAN Switches
TCP/IP
WAN
EMC DMX
SAN
Hosted by
NetEx HyperIP
Native GigE Replication: SRDF +
RFC 3135
Adaptive Copy & SNAP/Asynch
•
RFC 3135 = TCP/IP Performance Enhancing Proxy
•
Up to satelite distances (46K miles roundtrip)
•
90% + bandwidth utilization (T1/E1, DS3, OC3, OC12)
•
Plus 2 to 4 to 1 compression
Who
•
NetEx (HyperIP), Expand, NetCera, Digital Fountain
Ethernet LAN/WAN Switches
TCP/IP
WAN
EMC DMX
SAN
Hosted by
EMC SRDF Replication over WANs
Replication Methodologies Illustrated
Ethernet LAN/WAN Switches
Cisco MDS 9216
CNT UESR 3000
Brocade 3xxx
TCP/IP
WAN
McDATA/Nishan 3300
EMC DMX
NetEx HyperIP
= GigE
= FC
Hosted by
Considerations for Designing IP
Storage Networks
Separate LAN fabric
•
Minimally, separate VLAN
Layer 2 switching
•
Best latency for Ethernet switching
•
Nothing less than GigE
Understand LUNs
•
Mapping and Masking
Hosted by
Questions?
Enter the password to open this PDF file:
File name:
-
File size:
-
Title:
-
Author:
-
Subject:
-
Keywords:
-
Creation Date:
-
Modification Date:
-
Creator:
-
PDF Producer:
-
PDF Version:
-
Page Count:
-
Preparing document for printing…
0%
Comments 0
Log in to post a comment