CHPC-to-SKA-HMS.PPTX - Karoo Array Telescope

disturbedtonganeseBiotechnology

Oct 2, 2013 (4 years and 12 days ago)

93 views

The CHPC is an initiative funded by the Department of Science and Technology of South Africa

managed by the Council on Scientific and Industrial Research



The Contribution of High Performance
Computing to the Success of SKA.


University Researchers


Petrochemical Industries
(SASOL,
PetroSA
)


Nuclear Industry (NECSA,
NNR, ESKOM)


Mining industry (gold,
platinum, diamond)


Auto manufacturing
(Optimal Energy)


Animation industry


Science Councils


Government departments


Financial sector


Sports


Large Scale Science Projects
(CERN, SKA, Climate
Change)

Drivers of HPC demand in S.A.


Computation

(Analysis)



Computation

(Simulation)



Theory



Experiment


3
rd

and 4
th

Modes of Research





© CSIR 2010

Slide
4


Definition of
Cyberinfrastructure





© CSIR 2010

Slide
5


Cyberinfrastructure

Components

Cyberinfrastructure

= Hardware + Software + Bandwidth + People





High speed
networks and
Software



Data, Data
Analysis and
Visualisation



Distributed
Communities and
Virtual
Organisations



Human Capital
Development



High Performance
Computing





© CSIR 2010

Slide
6


The National Backbone Network

Introducing the CHPC

Centre for High Performance Computing


The first node is situated at the Rosebank, Cape Town.


The centre aims to enhance the competitiveness of South Africa through


-

High performance Computing


-

High
-
end data curatorship


It is funded from cyberinfrastructure project, a line
-
item of the Department of Treasury,
through the Department of Science and Technology. (
i.e

the centre is of national interest)



Structure


Meraka

Centre Director



Meraka

Management
and Support Teams




Competency

Area





Competency

Area





Competency

Area





Competency

Area






Cyberinfrastructure



CHPC/VLDB


and


SANReN




HPC Users in South Africa

0
50
100
150
200
250
300
350
2007
2008
2009
2010
Over
500
active users

2000
:


Started as a
collaborative
HPC initiative
among computational scientists.



2001
:


The
DST

recognised

the potential
and importance
of this initiative and
invited
CSIR
as a partner.


2004
:


DST

appointed
scientific advisory committee
and develop business plan.


2005
:


The CHPC incorporated into CSIR's
Meraka

Institute
.


2006
:


Refurbishment of
Rosebank

Campus
.


Time
-
line of Progression

2007
:


Official
launch of the 1
st

Phase of computational resource
and
Flagship Projects



2008
:


Introduced
Human Capital Development
schemes and invite general users.
Implemented Innovative
Flagship Projects
.


Established a partnership with
IBM

and host a
Blue Gene/p
computer to promote
the development of HPC in Africa.


Introduced its own research
programme



Advanced Computer Engineering
Laboratory



Timeline of Progression



2009
:


Implemented the Phase 2 of computational resource. The
SUN hybrid system
was
ranked on the TOP500 list and Largest in Africa.



2010
:


Introduced new computer fabrics


a small cluster consists of the
General Purpose
Graphical Processing Units

(A test
-
bed made of 20 GPGPUs)


Increasing new users, implement new Flagship Project and introduce
commercial
partners

(such as SASOL, a computational engineering consultancy firm and two
animation companies)



2011
….


Foster more collaboration…….

Timeline of Progression

Each node is equipped with two dual
-
core AMD
Opteron

2.6GHz Rev. F processors (640
CPUs in total at approximately 2.5 Teraflops/s peak performance) and 16GB of DDR2
667MHz random access memory.


The nodes are interconnected with the shared
-
file system of the SAN, accessed over
the
Infiniband

4X SDR 10 GB cluster via HTX from Voltaire and
PathScale
.


Eight of the cluster nodes are equipped with
ClearSpeed

accelerator cards

In addition to local hard disks, all nodes have access to a shared storage system with a
capacity of 94TB via a General Parallel File System (GPFS)

Phase
1
Computational Resource

As part of IBM's Global Innovation
Outlook and the IBM
-
CHPC partnership,
the CHPC hosts a rack of Blue Gene
(BG4A) donated by IBM. The Blue
Gene®/P system is capable of 14 trillion
individual calculations per second, and
is five times more powerful than the
second
-
fastest research computer on
the African continent (in Egypt). The
Blue Gene®/P provides 1024 compute
nodes, each with four fully cache
-
coherent cores and 2GB RAM. The cores
run at 850MHz.


Blue Gene for Africa

Phase 2 Computational Resource

System Name


Sun Hapertown

Sun Nehalem

SMP (M9000
-
64)

CPU

Intel
Xeon

Intel
Nehalem

Sparc

CPU Clock

3.0 GHz

2.93 GHz

1.9 GHz

CPU Cores

384

2304

256 (512 Threads)

RAM

768 GB

3456 GB

2048 GB

Peak Performance

3 TFlops

24 TFlops

2 TFlops

Interconnect

Jupiter Connector

Infiniband Network

Infiniband

Network

Sun Shared Storage

480 TB (Multi
-
cluster)

Introducing new technologies

The CHPC constant watches the trend of development within the
areas of HPC and data management. The centre aims to introduce
cutting
-
edge technology and work along side with the communities
to facilitate the deployment of these technologies.


For example:

General Purpose Graphical Processing Units


Promoting Research and HCD





© CSIR
2010

Slide
19


Stakeholder Engagement

Research and HCD

Technical Advisory
Committee

HPC and Technical
Support

CHPC Management

Operations and
Corporate Affairs

Steering Committee

Scientific Advisory
Committee


Advanced Computer

Engineering



Astronomy/Cosmology



Bioinformatics



Epidemiology



Chemistry



Computer Science/GRID



Computational Finance



Earth Sciences



Physics



Materials Science



Visualisation




High Education Institutions



Government Departments



Science Councils



Research Agencies



Industry



International Collaborators



Policy Makers


SIGs

CHPC

Partners

Training Programs (Workshops, Short
-
Courses )


> 50 Course to date

Annual HPC Conference


4 successful

meetings

International visitors

Integration

with Main Stream Conferences


SANReN

Partners


CHPC


Partners

Partners

Portals /
middleware

Strong Emphasis on Creating Partnership

-

Infrastructure and services

-

Enhance collaboration (sharing common knowledge/expertise)

-

Training initiatives

-

Foster Innovation (own research lab and flagship projects)

-

Promote commercial usage



Partners


Engagement Model

Research Facilitation

Flagship Projects relevant to
MeerKAT
/SKA


-
Prof
.
Marius
Potgieter

(North
-
West University)


-
Prof.
David Davidson
(Stellenbosch University)


-
Dr
.
Kavilan

Moodley
, (UKZN
), Prof.
Catherine Cress
(UWC) and
Prof.
Bruce
Bassett

(UCT
)


-
26 students and postdoctoral fellows + 19 peer reviewed publications




(8 other Flagship Projects


approx. 50 students and postdoctoral fellows)








© CSIR
2010

Slide
23


Other Flagship Projects

Climate Change

Energy Security

Mineral Beneficiation

Drug discovery

Fluid Dynamics

Astronomy and Space
Physics


HIV Mutation

Brain Imaging

Cardio
-
Vascular

3D Animation

Prof. M.S.
Potgieter

(North
-
West University)

A
series of computational investigation on the cosmic Ray
Propagation in the galaxy, cosmic rays in the local interstellar
medium,
heliospheric

properties, space Climate, Space Weather
to
geospace

and neural networks

Professor David Davidson (Stellenbosch University)

This
project addresses electromagnetic computer simulation
of key elements of the proposed Square Kilometre
Array.

Dr.
Kavilan

Moodley
, (UKZN), Prof, Catherine Cress (UWC) and
Prof Bruce
Bassetts

(UCT)

The
project is complemented by
four focus areas which
collectively aims to gain insight on the evolution
of cold gas in
galaxies and radio sources in galaxy
clusters and estimate key
cosmological parameters estimation simulations.
Simulations to
analyse microwave background data from the Atacama Cosmology
Telescope and optimize
future supernovae surveys for dark energy
constraints
were also conducted.




Short Description of these Flagship Projects

Research Collaboration


Apart from facilitating a number of general users to make use of the
CHPC infrastructure, the CHPC aims to collaborate with universities
and noteworthy organisations such as
MeerKAT
/SKA, SAAO, SALT,
SAC, HMO, Space Agency, and the likes.



Agreements with other
Centers

around the world


MSU

RCC, NCSA, NCHC, EPCC,
HPCx
, Cambridge, Stanford, ESC
etc..





The CHPC is developing several
own research programmes
. For the
ACE Lab
Programme
, past
and current relevant
projects include:

Hardware
design


Mike
Aitken
: HTX co
-
processor card


Jason
Salkander
: Low cost PCI
-
express data acquisition board
-

Astrogig



Software acceleration and design

Andrew Woods: FPGA accelerated Software Radio Astronomy
Correlator

Arjun

Radhakrishnan
: GPU accelerated Pulsar
D
-
dispersion


A number students are currently engage in technologies of interest to the
astronomy communities but applying on other research areas such as
bioinformatics, finance and remote sensing.



Own Research (ACE Lab)

Test
-
Bed Team

OEM, Vendors

Technology development

Local Industry

-
Integration


Academia


Research in Novel architecture

Test Bed Teams


HCD

Production System

-
System Administration and User
Support


Provision of HEC (Tier1/0 systems)


USERS



Academia


Industry


Science councils

Tier
2
Facilities


Knowledge Transfer

The CHPC hosts a number of joint workshops with our flagship
project groups, South African research chairs and leaders within
different communities.


For example in the following financial years:


2008


2009 = 20 workshops



2009


2010 = 22 workshops



2010


2011 (to date) = 16 workshops




Annual HPC School

Each year, the CHPC hosts a HPC school (typically participated by
about 50


60 students and young researchers) aiming to


-
introduce researchers and students to the core skills and
advanced topics relevant to HPC.


-
enhance the skill transfer between international experts and
local communities.


-
promote the development of students, particularly those
originated from the previously disadvantaged background.


-
provide full sponsorship for South African students and partial
sponsorship towards the African students.

Towards an integrated Cyberinfrastructure
Training School

Apart from the HPC school, the CHPC aims to roll out the following
large training schools which cover:


-
research domain specific topics


-
data curatorship


-
Cyberinfrastructure IT courses (HPC, network, data
management)

Proposed collaboration

Joint studentship and postdoctoral fellowship on hardware and
software development.



Joint training initiatives on HPC and data
curation

technologies.