The GridUP Portal

diligencebusinessInternet and Web Development

Dec 7, 2013 (3 years and 8 months ago)

157 views

Proceedings of the Joint GISELA-CHAIN Conference
R. Barbera et al. (Eds.)
COMETA 2012
© 2012 The authors. All rights reserved
The GridUP Portal
João Rodrigues
1
, Inês Dutra
1,2
, Sérgio Afonso
3
, Rui Ramos
3
, Lígia M. Ribeiro
3
1
Department of Computer Science, University of Porto
joaorodrigues1989@gmail.com
2
CRACS / INESC TEC and Faculty of Sciences University of Porto
ines@dcc.fc.up.pt
3
University of Porto
{safonso,rramos,
lmr}@reit.up.pt
Abstract
As many other institutions involved in grid infrastructures, the University of Porto (UP)

has federated resources that are part of the European Grid Infrastructure. Many researchers

at the UP already successfully use those resources, through various virtual organisations

(VO), supported by our internal grid infrastructure. In particular, we support a local VO

(vo.up.pt) that aggregates the academic UP community. In this VO, researchers have access

to around 100 cores.
Despite the success of utilisation of this infrastructure by some researchers, there is still

scope for improvements. First, there are still several internal communities that could benefit

from using those resources, but do not know how. Second, various reasons prevent more

people from using grid infrastructures: (1) the bureaucracy involved in using grids (for

example, obtaining certificates or the need to have an account on a user interface), (2) the

overheads of grid services due to the various layers of software and centralised servers, (3)

the high rate of job failures, and (4) the change in methodology and environment to execute

applications.
In order to leverage these infrastructures and take advantage of their full potential, many

solutions have been implemented and adopted. One of them is to offer interfaces in the form

of portals or gateways, which reduces the user's learning time, at least minimising problems

(1) and (4) aforementioned. Following the same track of other institutions (e.g., University of

California at Los Angeles Grid Portal
1
or the GISELA eScience Gateway
2
and its previous

version Genius), we designed and developed a user portal, the GridUP eScience Portal, to

offer to users a friendly interface to interact with the grid infrastructure we have. This portal

1

http://grid.ucla.edu
2

https://gisela-gw.ct.infn.it/science-gateways
builds upon previous experiences with new grid users and on the development and support of

grid applications (e.g., the EELA Grid User Guide
3
, the UbiDis library
[15]
and the

execution of parallel inductive logic programming
[16]
and other applications
4
). The main

feature of this portal is the easy way of authorisation and authentication as well as single

sign-on based on eScience federated authentication. This reduces the bureaucracy of

requesting a certificate, since researchers can prove their identities by simply having an

institutional certification, without having to go personally to a Registration Authority or

Certification Authority. This method of federated authentication follows the guidelines

adopted by Terena (the Trans-European Research and Education Networking Association).

Moreover, using federated authentication, we can foresee the use of a smart card to access

grid infrastructures. Currently, the portal instructs the users on how to access the

infrastructure and how to prepare jobs to be submitted with the gLite middleware. Only basic

jobs are supported at the moment. The portal was built on top of LifeRay, a free and open

source enterprise portal written in Java and distributed under the GNU Lesser General

Public License and proprietary licenses, and uses the jLite library as an interface to call the

gLite backend commands. An account is created for each user in the portal server, and the

certificate is stored there to be used by the voms-proxy-init command.
1. Introduction
A web portal is a software platform for building websites and web applications. It brings

together information from different sources in a unified way [5]. Typically, the user can show

this information in a dedicated area of the page (portlets) and treatment of different sources of

information may be made independently of each other [1].
The portals usually offer a set of services that differ from traditional web sites. These services

may be [1]:

a Search engine

to Send email to the administrator/community

News

Information

Multiplatform and multilanguage integration, such as smart phones, tablets

Management of user groups/permissions
As a rule, the portals provide developers a look and feel and very persistent access control.

These features allow the developer to focus more on the content of the applications.

Examples of portals that are used very often by many different kinds of people are

government, cultural portals, and search engines such as iGoogle, MSN, Yahoo and many

others [7].
In this work, we present the architecture and main requirements of a grid portal to be make

available to the academic community of University of Porto. We first present its architecture,

then we explain the technologies used to develop the portal as well as some justification for

the choices made. We conclude with a summary of this work and perspectives of future

developments of the GridUP.
2. The GridUP Portal Architecture
3
http://www.eu-eela.eu/index.php?option=com_content&task=view&id=133&Itemid=209
4
Available from the EELA2 website: http://www.eu-eela.eu
The GridUP portal allows a user to submit common tasks that require large computational

power in a friendly and fast way. As stated earlier the main function of the gridUP portal is to

facilitate the use of the grid and make the existing computational resources easily available to

all users of the University of Porto.
The GridUP portal can be divided into three large groups of different components, these

being personal certificate management, submission of tasks and task queue inspection.
In the personal certificates management, the user has the opportunity to upload his/her

certificate to the portal. A secure account for each user is supported by the server.
The portal

supports the extension originally exported by the browser (e.g. Mozilla Firefox, Internet

Explorer), and this is ".p12".
When submitting a file of this type the portal automatically

separates the private key of the certificate. To be able to separate the private key and the

certificate, the user is notified to enter the password that was entered when the certificate was

downloaded in the browser (this password is needed to protect the user’s private key). The

user has also the possibility to directly submit the files "userkey.pem" and "usercert.pem" if

so desired, thus avoiding the application of passwords by the portal.
Figure
shows the

interface for that. The user has at its disposal information (help) of (1) how to request a

certificate, (2) how to export it in different versions of browsers and (3) linux instructions to

separate the private key and the certificate from

a file with extension ".p12".
Figure . Upload Certificate Page
The submission of tasks is the component

where users create their "jdl" files with the help

the portal provides to the user. Users have at

their disposal a simple form to fill out and an

area to upload files needed to run the task, as

shown in
Figure
. Currently the portal only supports the submission of simple tasks, but it is

prepared to include other types of tasks easily. In the simplest case, the user only needs to fill

up two fields and upload the executable/program to submit a task. The fields in the set are

"Executable" and "Arguments", where the user specifies the executable/program and its

arguments, respectively. For more complex jobs, the user needs to specify what is needed

through other fields. All the files necessary for proper execution of the program must be

submitted through the portal or must be represented by an URI, in case the user needs files

that are in the storage elements. For every attribute on the form, there is an image

representing a help contents that is activated by just passing the mouse through the image.

The portal opens a "tooltip" that explains the syntax of the given attribute along with a brief

description about it and also some examples of its use. If the attributes defined by default on

the portal are insufficient to describe the task, the user can add others by bringing a list of the

available gLite attributes choosing the ones that can meet the tasks’ needs. These other

attributes appear in a new window within a “ComboBox”. If the attribute does not appear in

the list, the user can create it. When the user selects an attribute, just below the attribute

name, a text box/table appears in order that the user can type in its value. The user can

remove and add the attributes as needed.
This portal has the ability to upload a "jdl" file, previously created. This is essential for users

to be able to quickly submit tasks that are quite similar, for example, when there are only

changes on the executable name and its arguments. When uploading a "jdl" file, the user can

create a name that represents that, as shown in
Figure
.
For each field entered by the user, the portal can check its syntax and consistency. For

example, it can verify that the files specified in the "InputSandBox" correspond to the files

that were actually uploaded by the user. It can also check for the types of arguments

(integers, characters etc). Currently, this verification is only performed for some fields, but

the aim is to extend to all fields.
The portal creates a specific environment for each task, where the “jdl” file, the input files,

executable, and outputs will be stored. This environment consists of a folder with the name of

the task under the user’s home directory. All this information is also stored in two mysql

tables that keep track of all tasks submitted by users, thus keeping the state of execution for

all tasks.
Finally, the task queue component is

responsible for all the tasks submitted by a

user. This information is essential for the user

to have real-time feedback of the status of their

tasks. The interface for that is shown in
Figure
.

This component is responsible for all the

interactions between the portal and the grid

infrastructure. It shows all tasks submitted so far, and it can clear the task history record upon

the user’s request, eliminating any entry to that task in the database and the server disk.

Through this interface, the user can submit tasks, check their status and get the output

resulting from their execution. Whenever a task is completed, be it with errors or not, the user

can re-submit it without having to upload the “jdl” file again. It suffices to click the "Submit"

button again (this is possible due to the environment saved in the portal).
Finally, the portal keeps the state of the proxy lifetime so that the user does not need to

recreate it when entering or exiting the portal. The certificate for each user is stored on the

server where the portal is hosted. Whenever the user wishes to perform some action, the

system loads the certificate already previously created, checks and validates it. Only in the

case of the certificate being invalid, for example by expiration of time or other reasons, is that

the website asks again the password of private key to create a new proxy certificate.
3. The Jlite library
Jlite is a Java library providing a simple

application programming interface (API) for

accessing a gLite based grid. It runs in any

Linux or Windows operating system as long as

you have the Java Development Kit (JDK)

installed [11]. One of its advantages is to be able to submit tasks without the need to have a

user interface installed. Another advantage is the fact that it is free. In addition to access the

infrastructure through the Java programming language, it also has a command-line interface

that can be used to interact with the infrastructure without great effort and outside the

administrative domain of the machine that uses Jlite. This means that users can access grid

resources anywhere without having to start a Secure Shell (SSH) session. In order to use this

library, a network port needs to be opened in the Workload Management System (WMS). All

commands are available on the folder "/jlite-0.2/cli" in the form "<path>.", in the case of sh

Figure . Job Submission Page
Figure . Job Queue Page
Linux, and in ".bat", in the case of Windows [12].
The current implementation supports the entire lifecycle management of tasks such as:

"VOMS proxy creation and delegation", submission of tasks, monitoring jobs (State), obtain

the output resulting from execution in the infrastructure, among others. Its configuration is

extremely simple and takes very little time to set up. An example of a configuration is given

below:
·
All configuration is done under the directory “<path>/jlite-0.2/etc”. This is

subdivided in three other directories: “certs”, “vomses” and “wms”.
·
All certificates of certificate authorities and voms configuration files are placed in

the “certs” directory.
For example, for the VO vo.up.pt:
o
Create a file “voms.up.pt.lsc” with the following contents:

/C=PT/O=LIPCA/O=UP/OU=IRICUP/CN=voms.up.pt
/C=PT/O=LIPCA/CN=LIP Certification Authority
·
The configuration files for the voms are placed in the directory. For example, for the

VO vo.up.pt
:
o
Create a file “vo.up.pt” with the contents:

"vo.up.pt" "voms.up.pt" "15000"
"/C=PT/O=LIPCA/O=UP/OU=IRICUP/CN=voms.up.pt" "vo.up.pt"
·
All configuration files for the WMS are placed in the directory “wms”, i.e., the URL

of the machine that runs the WMS. Example of configuration
:
o
Create a file “glite_wms.conf” with the contents:

[

WMProxyEndPoints = "https://wms01.up.pt:7443/glite_wms_wmproxy_server";
]
The Java code required to submit a task through our program is relatively simple and short,
just import some libraries that are responsible for simulating the entire configuration of a
gLte user interface.
4. Liferay
Liferay is a software that allows you to manage and create a whole suite of applications

such as portals. The Liferay Portal is a free and open-source portal, written in the Java

programming language and distributed under the GNU Library General Public License

(LGPL) [8]. This portal allows users to configure common resources of websites, that are

built through functional units called portlets, responsible for the structure of the website

[9]. The portlets are nothing more than web applications that run in a given portion of

space of a web page. The portlets are compatible with standard portlets JSR 168 (portlet

1.0) and JSR 286 (portlet 2.0) [9]. One of the great advantages of using portlets is that

these can be removed and added dynamically to the portal without any complication and

without having the need to program their integration. Each web page can contain various

types of portlets with, for example, news, time, resources, grid etc (Portlets). Each portlet

works independently in that removing or adding a new portlet does not affect the

execution of other portlets. The Liferay portal provides mechanisms for communication

between portlets in case the user requires it [9].
The major objective of the portlets is to allow an inexperienced user to create web pages

without great difficulties. Liferay provides a development environment that can be

integrated with Eclipse or NetBeans. In this environment, the programmer can create the

most varied projects such as portlets, themes, hooks (overriding Liferay core

functionality), among others [7].
The application server that is used for the gridUP portal is the apache tomcat 6.0, but

there are other types of servers such as Glassfish, Jetty, etc. This application server along

with the features of the Liferay portal enables the programmer to manage organizations,

communities, user groups, user permissions, user accounts, mail management, run the

garbage collector to clean up memory, among many other things. All these services can

be done graphically without introducing a single line of code. In terms of graphic aspects

of the page, the programmer has the possibility to put portlets in the zone that he/she

wants with a graphical interface. The user can also do configurations using the graphical

interface. For example, backgroud color, margins, text alignment etc can all be adjusted

thorugh the graphical interface. The user can also import new themes or create a new one.

The Liferay portal is distributed in two different editions [7].
·
Liferay Portal Community Edition:
version with the latest tools supported by a

community effort. This is the version used in the gridUP portal.
·
Liferay Portal Enterprise Edition
: commercial version that includes upgrades and

complete support, such as training, call help, installation and others.
To conclude this section, the Liferay platform is advantageous because it allows the

integration of small pages in just one (portlets), it can increase the productivity and allows for

the creation of quite robust web application. It also supports portlets written in other

programming languages such as PHP, Ruby, Groovy and Python (need to install their plugins

from the Liferay's public plugin repository on Github, https://github.com/liferay/liferay-
plugins) [10].
The main disadvantages of this platform is that the server needs to be quite robust and users

not experienced with Liferay can have a quite steep learning curve.
5. Vaadin
When we talk about web applications, we often think in languages such as HyperText

Markup Language (HTML), Cascading Style Sheets (CSS), Javascript, PHP, Perl etc. The

plugin Vaadin came to revolutionize this paradigm, in which the creation of web pages is

pure java. With Vaadin you don't have to know any additional language besides Java to

create web applications. Unlike Javascript, that runs in the client's browser, the Vaadin code

written by the programmer runs on the server side. To show the data, produced by the server

on the client side, Vaadin uses the Google Web Toolkit (GWT). The GWT is a tool that

allows you to optimize web applications without the programmer’s intervention, for example,

it can optimize Javascript, debugging, etc. For example, when writing a set of Java APIs,

Vaadin allows the user to write AJAX applications in Java and then compile to Javascript [6].
Building web pages on Vaadin is very similar to the construction of graphical user interfaces

in Java. You have at your disposal a set of components, such as Buttons, Tables, TextFields,

Labels etc, which can be used in the construction of its interface. As in graphical user

interfaces in Java, the programmer can use events, listeners, data binding to communicate

between the various components. Web applications programmer can catch mouse events

through Javascript shortcut keys. In Vaadin the programmer can put listeners in the defined

components and treat all actions accordingly [13].
U
sers can modify any graphical aspect using CSS. By default each component brings with it

some predefined classes so that they can be accessed through the CSS and change the whole

content. The classes here are called equivalent to classes that are defined in an html tag. The

programmer has the freedom to modify the name of the class. In addition to the CSS the

programmer also can define his/her own Javascript libraries using The Yahoo! User Interface

Library (YUI) or jQuery. These libraries are installed by default, but others can be installed

[13].
Finally the advantage of using this plugin is the ability to generate Javascript code (client-
side) efficiently and the rapid development of an application. Another advantage is the

modularisation of code, so the programmer is able to arrange all the code in a more readable

and clear way allowing a future understanding or reuse of the same code [6]. Vaadin is free

and is protected under the Apache License Version 2.0 license [3].
6. Conclusion and Future Work
We presented the architecture of the GridUP Portal, built on top of technologies such as

the Liferay Portal, the jLite library and Vaadin, a plugin that allows to integrate everything

using only the Java programming language. The GridUP Portal allows that users can submit

their jobs without having to understand very deeply a new distributed programming

environment. Users can submit jobs, check the status of their jobs, cancel and resubmit jobs

without the need to upload it again. Each user and each user’s job have a protected separated

area. In order to submit jobs the user still needs to obtain an X509 certificate. But the portal is

prepared to use a single sign-on based on eScience federated authentication. This reduces the

bureaucracy of requesting a certificate, since researchers can prove their identities by simply

having an institutional certification, without having to go personally to a Registration

Authority or Certification Authority. This method of federated authentication follows the

guidelines adopted by Terena (the Trans-European Research and Education Networking

Association). Using federated authentication, we can foresee the use of a smart card to access

grid infrastructures. The University of Porto is currently testing the federated authentication

according to the TERENA protocol.
Besides having federated authentication we intend to expand the functionalities of this

portal with
:
·
The integration of graphical tools (e.g., workflows);
·
A clever way of constructing the Requirements and Rank attributes
·
The integration with the Logical File Catalog (LFN), in order that the users can use

grid files in their applications;
·
The extension to other types of jobs (parametric, dag etc)
·
The possibility of selecting a virtual organization;
·
The production of videos and help that can instruct the user on how to use a grid

infrastructure;
·
Templates of “jdl” files that can be adapted to be reused by users.
References
[1]
J. Bourke, 10 July 2010. [Online]. Available:
http://www.cpa2biz.com/Content/media/PRODUCER_CONTENT/Newsletters/Articles_
2008/CPA/Apr/Portals.jsp. [Acedido em 22 June 2012].
[2]
Serving Swiss Universities (SWITCH), 2012. [Online]. Available:
http://www.switch.ch/grid/slcs/about/. [Accessed 21 June 2012].
[3]
Vaadin Ltd, 2011. [Online]. Available: https://vaadin.com/license. [Accessed 21 June
2012].
[4]
2012. [Online]. Available: http://www.liferay.com/community/wiki/-/wiki/Main/Portlets.

[Accessed 21 June 2012].
[5]
Liferay, 2012. [Online]. Available: http://www.liferay.com/products/what-is-a-
portal/web-platform. [Accessed 21 June 2012].
[6]
Google, 13 March 2012. [Online]. Available: https://developers.google.com/web-
toolkit/overview. [Accessed 21 June 2012].
[7]
Liferay, 2012. [Online]. Available: http://www.liferay.com/documentation/liferay-
portal/6.1/user-guide/-/ai/introduction-to-liferay. [Accessed 21 June 2012].
[8]
Liferay, 2012. [Online]. Available:
http://www.liferay.com/web/bryan.cheung/blog/-/blogs/liferay-adopting-the-lgpl-license.

[Accessed 21 June 2012].
[9]
Liferay, 2012. [Online]. Available: http://www.liferay.com/downloads/liferay-
portal/overview. [Accessed 21 June 2012].
[10]
Liferay, 2012. [Online]. Available:
http://www.liferay.com/community/wiki/-/wiki/Main/PHP+Portlets. [Accessed 21 June
2012].
[11]
O. Sukhoroslov, 20 May 2011. [Online]. Available: http://code.google.com/p/jlite/.
[Accessed 21 June 2012].
[12]
O. Sukhoroslov, "jlite-manual-0.2," 2010.
[13]
Vaadin Ltd, 2011. [Online]. Available: https://vaadin.com/features. [Accessed 21 June
2012].
[14]
Vaadin Ltd, 2011. [Online]. Available: https://vaadin.com/learn. [Accessed 21 June
2012].
[15]
N. A. Fonseca e I. Dutra, “Ubidis: a flexible and general top-level middleware to manage

applications in grids and clusters,” em Proceedings of the 3rd Iberian Grid Infrastructure
Conference, Valencia, Spain, 2009.
[16]
I. C. Dutra, “Solving ilp problems in the eela infrastructure,” em Proceedings of the 3rd
EELA Conference, Catania, Italy, 2007.