IASA IT division

excitingwonderlakeInternet and Web Development

Dec 13, 2013 (3 years and 10 months ago)

77 views

IASA IT division

Marios Chatziangelou

Technical Manager

Division key areas

2

Division expertise: focused on three main areas:



(Inter)National IT services (EU/National projects)



Institutional
-
level IT services



Software Development


Activities

3

An overview of the IT team/division activities:

IT division
Inter
-
National IT
Services
(
EU
/
National
projects
)
Institutional IT
Services
Software
development
Grid
Computing
Clustering
HPC
Cloud
Computing
Digital
Repositories
Databases
Network
Operations
Center
Collaboration
Services
Helpdesk
(
1
st
-
2
nd
line support
)
Multithreaded
applications
Web
applications
/
services
Embedded
/
RT
SW
Databases
Drivers
Client
/
Server
applications
Parallel
programming
(Inter)National IT services

4

Introduction


EU
-
funded projects



i.e. GRIDCC, EGEE I
-
II
-
III, EGI, PRACE etc.


Nationally
-
level projects



i.e. HellasGrid I
-
II, HellasHPC etc.


Current responsibilities/active roles (Grid
-
Computing)



EGI task leader for User Community Technical Services



Regional Coordinator for the operations in the South Eastern Europe



Country representative/coordinator for HellasGrid


Experienced WP leader in numerous EU projects


Preparation of EU proposals (related to Grid and/or Infrastructures)

5

Grid Computing (1)

Joint Research Unit on the EGI
-
InSpire project
-

the biggest, 4
-
year, pan
-
European project on Grid Computing (total project budget aprox. 100M

)


Operation of two production
-
level Grid Clusters


since 2005
(HG
-
02
-
IASA and GR
-
06
-
IASA)


Major stakeholder in HellasGrid




(Greek National Grid Initiative)


Since 2005 our clusters handled:



> 2.6M CPU hours; >1M concrete jobs


Extensive knowledge of gLite, UNICORE, ARC


middleware


Testing and certification of grid/distributed


services and middleware


Site & Region monitoring

(stand
-
alone/hierarchical mode mainly using

open
-
source frameworks i.e. Nagios, Ganglia, Cacti etc.)


Experienced in distributed systems security

Infrastructure:

Computing/storage


>300 cores


>20TB reliable storage

Networking


1Gigabit connectivity


CISCO, eXtreme & 3COM stack switches

General


>150kBTU
-

A/C


>100kVA


UPS


>160kVA


outdoor power
-
generator


permanent connection to the



UoA power
-
safe network

6

Grid Computing (2)

7

Grid Computing (3)

Software
:


Operating Systems


(Scientific Linux/Redhat, Debian, Fedora …)


Grid Middleware


(gLite 3.2)


Batch systems/Schedulers


(OpenPBS/PBS, LSF, Condor, MAUI




experimenting on SGE on Grid)


Information System


(OpenLDAP based architecture)


Mass deployment/configuration


(Quattor, OSCAR, Rocks)


Grid/Cluster Monitoring


(Nagios, Ganglia)


Central logging


(syslog
-
ng)


Updates management


(Pakiti)


Network monitoring

(SNMP
-
MRTG)


8

Digital Repositories (1)

Development & hosting of the EGI
-
InSpire International Software Repository
[http://repository.egi.eu/]


(joint effort with GRNET, AUTH, ICCS)


Release management of *
all*
Grid Middleware Distros


(
gLite, ARC, UNICORE,


UMD

[Unified Middleware Distribution])


Full support of the EGI release management process


Quality criteria verification process


Stage
-
Rollout process


Production level software provision


9

Digital Repositories (2)

Other software handled/managed by the Repo:


Globus


Grid Operations Tools



NAGIOS Grid monitoring suite



European Certificate Authorities packages


Jpackage (mirroring)


g
-
Eclipse


Ganga


Migrating Desktop


and many other ……

10

Digital Repositories (3)

Three
-
tier architecture:


Data Tier


ESciDoc (fedora
-
commons distributed repos arch)


Web services based API (SOAP & REST)

Business Tier


TAVERNA workflow management system


Java & perl combined implementation


Web services based API (SOAP & REST)


Request Tracker (RT) for SW Release workflow

(deposit & verification processes)

Presentation Layer


For end
-
users




EPrints & DuraSpace implementation


For automated
mass package installation



Multiple (hundreds) YUM/APT repositories provision








11

International databases (1)

IASA is responsible for the Development/Hosting/Maintenance of the
International Grid Applications Database [http://appdb.egi.eu]

The EGI Applications Database (AppDB) is a full
-
edged, community driven web
portal, meant to act as



a point
-
of
-
reference for scientic applications and tools available within the
EGI infrastructure (
over 260 registered Applications

and Tools
)



registry of persons involved in the development of said software (
over 500
registered scientists/developers
)


Main aims are to:



promote the significance of EGI/Grid in the e
-
Science



facilitate the reuse of applications and expertise that is available in NGIs



help create a vibrant user community, with information owing inwards
directly from involved individuals
-

developers and end
-
users alike


Descendant of the EGEE Applications Registry portal, developed and provided by
INFN and IASA.
Since the EGI era, the AppDB came under sole IASA responsibility

12

International databases (2)

[http://appdb.egi.eu]

13

International databases (3)

Technically speaking ….


Zend Framework



PHP basis for the Model
-
View
-
Controller (MVC) architectural pattern



Supports Table Data Gateway design patterns



Integrates with the dojo toolkit and jQuery

dojo toolkit



High
-
end UI experience through advanced widgets

jQuery


Application" look
-
and
-
feel through full use of AJAX


Use of jSON for data interchange

MySQL data back
-
end



InnoDB storage engine (triggers, referential integrity)



Extended use of optimized Views (Table/Row Data Gateway DP)


SOAP/REST web
-
services

for integration with external sources





14

HPC & Cloud computing

High Performance Computing:



Participation to the international project PRACE
-
1P



Major stakeholder of the HellasHPC consortium (IASA, ICCS, AUTH,
UPatras, led by GRNET)


Mainly focused on:



Benchmark suite for Tier
-
0 procurement



Integration of codes into the applications benchmark suite



Synthetic benchmark suite



Performance analysis tools



Cloud Computing (experimental phase)


Mainly focused on offering Grid services/resources over Cloud.

Institutional
-
level IT services


Local Network Operations Center


Provision of many Collaboration services


Operation of a Helpdesk


Provide clustering services




Local Network Operations Center


Our Local NOC offers the following services:


eMailing services
: MTA (Postfix), (s)IMAP (courier), Web mail (squirrel
& horde), central AV (ClamAV), centrall anti
-
spamming control
(spamassasign), mailing lists (mailman)


Ldap catalog

for local users auth/authz


Virtual Private Network

(CA based auth/authz)


DNS services

(4 domains, 2 private


2 public)


Web services

(two Web servers on a Round
-
Robin mechanism)


A local Certification Authority

(IASA CA)


Central firewall system


Gigabit network connectivity



All of our systems are centrally
monitored
:


Nagios, Gaglia, Cacti



Central logging facility

it is also available


Syslog
-
ng, with advanced filters enabled for immediate alerting

Collaboration Services & Helpdesk

Collaboration services


Wiki services (MediaWiki, Twiki)


TRAC/SVN for covering the SW development life cycle.


Open
-
xChange facility (an open
-
source MS exchange replacement)


Internal Document Management System


2 TB of reliable Samba store (internal) with “on access” antivirus scanning
(ClamAV)


eGroupware collaboration suite


And many other….

Helpdesk


Main aim, is to support and keep track of, our local users request.


The Helpdesk workflow, It is fully covered by


the Request Tracker (RT) ticketing system.


More than 65.000 tickets since 2005, originated from:


Local users (requests)


Automated alerting mechanisms


Clustering


Main aim: Provide, secure and reliable computational and storage
resources to our local (institutional) scientific communities
.


In terms of IT resources, the division offers:


One cluster with:
Computing >120 cores


Storage >10TB


Obligations:


Software installation & parameterization


Operating system (installation, updates, fine
-
tunning)


Libraries, Applications, Tools etc. (i.e. lapack, blas, fft, octave,
ROOT, NS2, FLUKA, MEEP, …. )


Application Porting & Support



Coding assistance


debugging


benchmarking


MPICH/OpenMPI, OpenMP support for parallel applications


System/Cluster parameterization


User management


Adjust the reservation policy based on local users needs


Cluster Monitoring


disk usage, cpu and memory utilization, networking, security ….


And many other …


Software Development


Indicative projects/developments


Driver Application for DAQ (Visual C/C++, multithreaded, )


Applications Database (php/Zend,Javascript/dojo,MySQL)


Digital Repository (perl, MySQL, php)


National Grid Hardware Registry (php, Javascript/AJAX, perl, MySQL)


Software Discovery Engine (php, perl, MySQL)


Institutional ERP system (Java, MySQL)


Driver developments


PCI, RS232, TCP/IP, USB PCI
-
X ….


Numerous, production
-
level applications for hardware/software
communication, integration and data acquisition (C/C++, Qt, posix
threads)


and many other ….


Skills/Expertise (1)


Deep knowledge on every phase through the software development life
-
cycle process, including:


Software requirements gathering


Preliminary design


Detailed design


Coding


Unit testing


Integration testing


Qualification testing


Deep knowledge in a software quality assurance techniques, based on both
ISO and CMM (Capability Maturity Model) standards.

Skills/Expertise (2)


Advanced coding skills and experience. Some, indicative, coding
languages/frameworks we are focused in, are:









Parallel/multithreaded applications development


Web application/service development


Databases: MySQL, PostgreSQL, Oracle (development and/or
administration skills)

C/C++

Python

Javascript


Fortran

Php/JSP

HTML


Shell scripting (bash, tch etc)

Ada

XML (XSD/DTD)


Perl

JAVA

Many other …

Affiliations (1)


The IASA IT team, keep strong relation with many Greek, as well as
International, academic institutes


National:


National and Kapodistrian University of Athens (NKUA)


National Technical University (NTUA)


Institute of Communications and Computer Systems (ICCS)


Greek Research NETwork (GRNET)


Foundation for Research & Technology (FORTH)


University of Ioannina (UoI)


Research Academic Computer Technology Institute (RACTI)


Aristotle University of Thessaloniki (AUTH)



Affiliations (2)

International

:


CyI
-

Cyprus Institute


CERN
-

European Organization for Nuclear Research


INFN
-

Istituto Nazionale di Fisica Nucleare


LIP
-

Laboratório de Instrumentação e Física Experimental de Partículas


UPV
-

Universidad Politécnica de Valencia



24

Questions & answers

Thank you!! ….