Highly Scalable Packetised correlators

Highly Scalable

Packetised

correlators

Jason Manley

CASPER workshop 2009

Contents

•
Where we’ve come from


•
CASPER


•
Correlator design

•
Dataflow

•
Engine operations

•
Scalability

•
Present and future deployments

In the beginning…

Computing
backend

Analogue
processor

Control &
Monitoring

Big
dish

Interferometry
…

ATA

Green Bank 300ft

…more recently…

EVLA
baseline
board

(Carlson, 2006)

CASPER

Center for Astronomy Signal Processing and Electronics Research

•
Open
-
source, collaborative environment

•
Reusable, platform
-
independent
gateware

•
Modular, upgradeable hardware

•
Low instrument NRE, shared by the community

•
Enables rapid development (8 instruments / 2 years)

•
Industry standard communication protocols

•
Low Cost


CASPER DBE concept

CASPER DBE concept

Clocking

•
Clocks:

•
Locally synchronous, globally asynchronous.

•
Sampling clock synchronized at F engines.

•
X engines each run off independent local clock.

•
Synchronized using global 1pps signal at ADCs

•
Propagated to X engines using out
-
of
-
band signaling on XAUI links.

•
Headers labeling and
timestamping

10GbE Ethernet packet data.

Correlator

Overview

How does it scale?

1
10
100
1000
10000
100000
1000000
F Engines
X Engines
43

100

200

330

760

0
100
200
300
400
500
600
700
800
1998
2000
2002
2004
2006
2008
2010
Logic Cells (thousands)

Xilinx
Virtex

Family

•
Processing power doubling every two years

•
V4 = ½ power requirements of V2Pro*

* Manufacturers claim
-

Xilinx Inc.

Existing systems:

PAPER
-

Green Bank

•
8 dual
-
pol

antennas, 150MHz.

•
1
st

Generation: one BEE2 & four IBOBs.

•
First deployment of packetized system.

•
Continuous upgrades.

Existing systems:

Northern Cross,
Medicina
, Italy

•
Designed and deployed in under 2 weeks.

•
32 single
-
pol

antennas, 32
MHz.

•
1
st

Generation:


•
one BEE2

•
eight IBOBs.

New deployments: KAT
-
7

•
7 dual
-
pol

antennas, 400MHz.

•
Pure ROACH.

•
New control and monitoring interface (KATCP),
coarse delay, high speed output (less than 100ms).

•
Expected early 2010.

New deployments: GMRT

•
32 dual
-
pol

inputs, 400MHz bandwidth.

•
Collaboration
–

shares design with KAT
-
7.


New deployments: PSA
-
32/64/128/256

•
PAPER in South Africa, staged deployment

•
Up to 256 antennas,

100MHz

•
Generation 1.5 (IBOB+ROACH) initially,

•
Generation 2.5 (ROACH + ROACH
-
II) eventually.

•
First field deployment of ROACH system.

•
Correlator deployment starting in January!

Jason Manley

jason_manley@hotmail.com