GitHub Migration Summary Report

idiotcanvasSecurity

Nov 17, 2013 (3 years and 4 months ago)

98 views

SAIC
-
Frederick

Version 0











GitHub
Migration Summary

Report




caLIMS


Duke University


January 8, 2013


April 12, 2013









Pankaj Agarwal

p.agarwal@duke.edu


April 12, 2013




2


1.

Personnel
Who
W
orked on

Migration

List all the team members who worked on the migration effort.


Name

Title/Role

Contact Information

Dr. Steve Patierno

PI

steven.patierno@dm.duke.edu

Pankaj Agarwal

Project Manager

p.agarwal@duke.edu

Kurt Morehouse

Technical Lead

k
urt
.m
orehouse
@dm.duke.edu

Insert additional rows as needed.



2.

Migration

Overview


Insert a brief summary of the migration of the product and current status.

Include a description of the
structure and contents of the repository or repositories created in GitHub. Report on the current status of
the build and
the performance status of
any automated build and test sequences.


caLIMS

v2 project has been migrated to github and the url for the new repository is located at:

https://github.com/NCIP/calims

The base code or the existing code that was used before the open source migration proces
s was
downloaded from the caLIMS v2 Wiki site:

https://wiki.nci.nih.gov/display/caLIMS2/caLIMS+v2+Software


Structure and Contents of the Repository
:

The directory structure of the

repository up to 3 levels is as follows:


|
--

build

| `
--

lib

|
--

calims2
-
api

| |
--

src

| | |
--

jar

| | `
--

java

| `
--

test

| `
--

unit

|
--

calims2
-
csm

| `
--

src

| `
--

java

|
--

calims2
-
model

| |
--

db

| | |
--

db
-
install

|

| `
--

db
-
upgrade

| `
--

src

| |
--

generator

| |
--

hibernateddl

| |
--

java

| |
--

models

| `
--

resources

|
--

calims2
-
taglib

| `
--

src

| |
--

java

| `
--

resources

|
--

calims2
-
uic

| |
--

src

| | `
--

java



3


|
`
--

test

| `
--

unit

|
--

calims2
-
util

| |
--

src

| | `
--

java

| `
--

test

| `
--

unit

|
--

calims2
-
webapp

| |
--

src

| | |
--

dojo

| | |
--

java

| | |
--

resources

| | `
--

web

| `
--

test

| `
--

unit

|
--

common

| |
--

bd
a
-
download

| | `
--

lib

| |
--

build
-
utils

| | `
--

resource

| |
--

checkstyle

| |
--

findbugs

| |
--

grand

| |
--

ivy

| |
--

javancss

| |
--

maven

| |
--

pmd

| |
--

resources

| | |
--

jboss
-
conf

| | `
--

security

| `
--

simian

|
--

deploy
-
config

| `
--

server

| `
--

default

|
--

docs

|
--

grid
-
sample

`
--

local
-
ivy
-
repo


`
--

ncicb


|
--

clm


`
--

csmapi



The project builds and deploys successfully in Windows environment. Build and deployment is currently
being tes
ted on Linux environment. The steps of cloning, building, and deploying the code are given in
the document “
caLIMSv2 Build and Deployment Guide.docx
”, which is included in the git project.



3.

Description of Work Completed

List all the activities that were

performed to complete the migration (preferably in the order they were
performed)
. Account for any automated tools that were created and/or used in support of this process.



ID

Activity

Description

Time Spent

1

Original source code migration

Migrated
the source code from caLIMS v2
Wiki to git repository for caLIMS

1 day



4






2

Research tools

Eclipse plugins, git, Ruby, svn2git on
Windows, GitGui on Windows

1 day

3

Understanding git

Research on how to use git from online
resources and forums

2 days

4

Removed NCI SVN
dependency from the build
process

Changed the existing build script to remove
any dependency on NCI SVN so that all
dependencies are fetched from publically
available resources

28 days

5

Editing and placing the
README, LICENSE and
README file on git

Edited the template file with the appropriate
information and additional editing based on
feedback.

1 day

6

Understanding the original NCI
BDA process

caLIMS V2 was created to be build and
dep
loyed using the NCI BDA (full, not the
lite version) process. In order to reverse
engineer to remove dependency on this
process, examined the existing build and
deploy artifacts

3 days

7

Reverse engineer the NCI BDA
process

Removed all dependencies on t
he NCI BDA
process, such as using the NCI version of
Jboss 5.

20 days

8

Installation of requisite software
on Linux and Windows

JDK 1.6, ANT 1.8, Jboss 5, Postgresql 8.4.4

1 day

9

Testing the build and
deployment

Tested on Linux and Windows, with several

iterations

5 dats

10

Documentation, reporting etc.

Scrum participation, required
documentation, Status Report etc

2 days

Insert additional rows as needed.



4.

Migration Artifacts

List all the artifacts of the product with appropriate locations.


ID

Artifact

Location
(s)

1

Source code

https://github.com/NCIP/calims

2

Documentation

(
including
repository
-
based, wiki,
other)

Installation document is in the git

project. All other project documentation is on
the NCI Wiki located at:

https://wiki.nci.nih.gov/display/caLIMS2/caLIMS+v2+Wiki+Home+Page
.
This will be migrated to Atlassi
an as part of community development.

3

Data

(test data
and seed data)

N/A

4

Issue Tracker

https://tracker.nci.nih.gov/browse/CALIMS2
. This will be migrated to
Atlassian as part of community deve
lopment.

5

Binaries and
installation
packages

(If
There is a Virtual Machine which is available on the NCI Wiki and which will be
migrated to Atlassian

along with the rest of the Wiki contents.



5


there are any
binaries and
installation
packages,
please include
these links in
README.txt)

6

Release notes

https://github.com/NCIP/calims

7

Production /
Development
/QA URLs if
deployed at NCI

N/A

8

Other useful
links, if any

N/A

Insert additional rows
as needed


5.

Issues

List all the issues that were encountered during the migration and the steps taken to resolve them.


ID

Issue

Status

Resolution/Proposed Resolution


There were some
issues with
cloning and
pushing code to
git form the linux

environment due
to security setup.

Resolved

This was resolved. GitGui on Windows was also used for
cloning and pushing code to Git successfully, which did
not have any security issues.


The original
caLIMS
Installation guide
described the use
of the Use
r
Provisioning Tool
(UPT), but this
was not migrated

Pending

The UPT was not part of the Virtual Machine, so it was
decided to exclude this from the migration. It has been
noted in the Lessons Learnt/Future Improvements section
as something which can be t
aken up by the open source
community.





Insert additional rows as needed.


6.

Success factors

List or describe what went right during the entire migration process.


ID

Success factor


Imported the source code in zip file from the caLIMS v2 NCI Wiki into the caLIMS git repository.



Removed the dependency on NCI resources for the build.


Git operations like cloning, committing and pushing.


Was able to engage several community members

for the validation process, although, as
mentioned in the Lessons Learned section, the actual validation was delayed

because of lack of
urgency from the community members, which was understandable given the voluntary nature of


6


this task
.


With the help

S
AIC
-
F, was able to resolve some of the issues with the build process.


Removed the dependency on the NCI version of Jboss 5 and migrated the
installation/deployment to the Open version of Jboss 5.



Insert additional rows as needed.


7
. Lessons learned
/ Future Improvements

List the lessons learned during the migration process including things that you would do differently if you
have to do migration of other tool and suggestions to other teams


ID

Lessons Learned/Enhancements/Improvements

1

Not having a deployed and working caLIMS instance was a big hinderance because there was
no baseline for a successful build and deploy process which could be used for incremental
changes to the build and deploy scripts. It would have been very useful to h
ave built and
deployed the existing source code so that all the pre
-
requisite software/applications, such as the
appropriate version of JDK, ANT, IVY, JBoss, and Postgresql would have been established as
working and appropriate. A lot of time was spent in

determining the appropriate version of
JBoss, because the current documentation mentioned version 4.x whereas the build and install
scripts mentioned different 4.x and 5.x versions.

2

Since the original code was not built and deployed at the beginning of the project, at least the
Virtual Machine with a deployed caLIMS v2 application, which is available on the Wiki site,
should have been brought up earlier. A lot of information could be

gathered about the
deployment

3

The biggest hurdle was in understanding the existing build and deployment process and
removing the dependencies on it. caLIMS v2 was originally built and deployed the full version of
NCI automated BDA (Build and Deployme
nt). The original BDA was very structured and was not
transparent, making it hard to understand the process. It was highly automated in the sense that
one simple build command did all the build and installation in a black box manner. This process
was la
ter revoked by the NCI and was replaced by a lighter version called BDALite, but caLIMS
v2 was never changed to use the lighter version. In order to take away the dependency on the
NCI BDA process a significant amount of time was invested in understanding

the process and
then reverse engineering the process so that all the dependencies on the NCI BDA could be
removed. The build is still automated now, required one ANT command, but the install process
is manual. As an extension to the project, that instal
l process could be automated. As it exists
now, only the database creation is automated, requiring one ANT command.

4

There were two locations of the original caLIMS v2 source code, one in the NCI SVN and the
other in the caLIMS v2 NCI Wiki site. Ther
e was some confusion about which was the most
current source code, but it was determined from the community that the one on the NCI Wiki was
the latest, although it was just a copy of the master branch on the NCI SVN. But some time was
spent in establishi
ng the location of the source code for migration.

5

There was a delay in getting community involvement for the validation of the
modified build and
deploy process. Since this is a voluntary activity on part of the community members, it was hard
to impose

upon them with a deadline. It might help to engage the community earlier in the
project and incentivize their participation by making them real stakeholders.

6

It was not possible to install the required version of Postgresql (8.4.4) on the version of L
inux
used (
RHEL 6
), which was the only version of Linux available to the development/migration
team. But there was no problem in installing this version on Windows 7 32
-
bit. The latest
version of Postgresql (9.x) was installation on the Linux server, but

there was
a
problem with the
deployment on this version of Postgresql. The migration process had to be
moved to Windows &


7


32.bit and was successful on it. As an extension to this project, it is suggested that issues with
Postgresql 8.4.4 installation on

Linux be resovled.

7

There are many files which whose name contain “bda” or “nci” or some other legacy names. It
would be ideal to rename these files so that there is no connotation of the legacy naming system,
which would make it more appealing to the
open source community. This task would take a
while and was out of scope for this project, but it is recommended that this be done after the
migration with the help of the open source community.

8

There are still some dependencies on the NCI IVY reposito
ry, but due to time constraint, these
were not removed. It is recommended that this be done after the migration with the help of the
open source community.

9

The User Provisioning Tool was not migrated and it should be taken up by the Open Source
communi
ty.

Insert additional rows as needed.



8
. Verificatio
n
(Please refer to
https://github.com/NCIP/ncip.github.com/wiki/Migration
-
Checklists#wiki
-
quality
-
assurance

for more details)


ID

Verification Check

Result

1

Check if a
1
-
2 line
description is added to the
repository

Pass

2

Check that the repo clones correctly and leaves
"master" checked out.

$ git clone
https
://github.com/NCIP/calims
.git

$ cd calims

$ git status



Pass
.


3

Check that the

LICENSE
,

NOTICE
,
and

README

files appear in the

master

branch


Pass

4

Check that the

LICENSE.txt

file also appears
in the

gh
-
pages

branch


Pass

5

Check that the

LICENSE
,

NOTICE
,
and

README

template placeholders have been
replaced


Pass

6

Check that all links in the

README

file work


Pass

7

Check that the

text in
LICENSE
,

NOTICE
,
and

README

files
matches the text provided in
the appropriate templates at
https://github.com/N
CIP/ncip.github.com/wiki/
Te
Pass



8


mplates

8

Check files of several source code types to
ensure the appropriate copyright header is
placed and template placeholders have been
replaced

Pass

9

Check that the URL to the

LICENSE.txt

file
works

Pass

10

Check that the URL to the

LICENSE.txt

file is
of the form:

http://ncip.github.com/calims/LICENSE.t
xt


Pass

11

Do a web search for the tool and check that the
appropriate links are found for



User Documentation



Reference Web Sites



Wiki Documentation



GitHub repository


Knowledge Center

Pass

12

Check that Subversion revision numbers are
mentioned in the

commit messages for
future reference. Run:

$ git log

and look for commit message lines
containing the corresponding Subversion
revision numbers. For example:

SVN
-
Revision: 1234

N/A

The code was migrated from the zip file on the
Wiki and not from the SVN.

13

Check that a
README

file appears in the
corresponding docs and data directories.

Pass

Note: There is no “data” directory, only a
“docs” directory

14

Check that the format of
README

file in the
corresponding
docs repository

match the
template at

https://github.com/NCIP/ncip.github.com/wiki
/Templates#wiki
-
readme
-
for
-
documents

and
the template tokens are replaced.


N/A

15

Check that the format of
README

file in the
corresponding data repository match the
template at

https://github.com/NCIP/ncip.github.com/wiki
N
/
A



9


/Templates#wiki
-
readme
-
for
-
data

and the
tokens are replaced.


16

Inform the PO to create a helpdesk ticket to
remove write access to all the user of the
repository.

?

Insert additional rows as needed.