Enterprise Drupal @ SF State - Coder in the Rye

watermelonroachdaleInternet και Εφαρμογές Web

30 Ιουλ 2012 (πριν από 5 χρόνια και 2 μήνες)

328 εμφανίσεις

Enterprise Drupal @ SF State


Campus: San Francisco State
University

1600 Holloway Ave.

San Francisco, CA 94132

http://www.sfsu.edu

Presented By: Kevin O'Brien w/ Supakit Kiatrungrit

Division of Information Technology

San Francisco State University

nowarninglabel on drupal.org

http://www.coderintherye.com

Greetings Drupal Ninjas

Choosing Drupal

Began with search for WCM for DoIT

Tested many (Plone, OpenCMS, Joomla, etc.)

Settled on Drupal for functionality and a11y

Started w/ 1 virtual server, no real environment

Started with 5.x and moved to 6.x

Campus already had some Drupal exposure

Ancient History

Portals required on CSU campuses


Project Approach (IBM
Websphere)


Collaboration between SFSU Team


and IBM Team


Approach according to the

established IBM Method


An emerging consensus regarding

portal development includes the

following major best practices and
considerations:


There should be one AND ONLY
ONE horizontal portal on campus;


Portals should be developed
iteratively;


The portal should support “single
sign
-
on”; that is, with a single user id
and password each user can access
all the applications and data that she
or he is allowed to use;


CSU campuses should consider
integration with both legacy systems
and CMS (PeopleSoft);


Courseware management tools
should be integrated with the portal;


From: "
http://www.calstate.edu/tis/iam/documents/presentations/2005
-
simi/SIMI_2005_SF_PresentationFINAL.ppt
" and
http://its.calstate.edu/systemwide_it_advisory/ITAC_keydocuments/PortalP
aper_02012001.doc

What happened previously


IBM Portal took years to implement base
features. Required constant maintenance
and time
-
consuming PMRs.


Other groups/depts. Didn't fully buy
-
in,
saw IBM WebSphere as a “black box”


Project Manager/Maintainer left


Said hey “How about going to Drupal,
they have it running successfully for
WCM, would maybe take a couple months
to switch”

IBM Lotus WCM vs. Drupal


Java


JDK/JavaEE/JSR
168/WSRP


Portlet Factory


Monolithic


Broke weekly


Few expertise


No community


Only support is
PMRs


PHP


drupal_hook API


Services module


Any IDE/Editor


Core is relatively
small


Huge community


Support via
forums, IRC,
Acquia

The A Team


Me


Architect /
Developer / Themer


Supakit “superkid”
Kiatrungrit


Developer


Bora Kou


Systems
Admin

(Part time on this
project)


Jason D'Silva


Student Asst.
-

Developer (10
hours/week)


Management position
was actually vacant
during our 2.5 month
sprint. So had a part
-
time interim project
manager.


Phase 1 time: 3
months


Now on Phase 3

Scrum style agile development


What we liked:


Quick
turnaround


Weekly
feedback


Clear deadlines


Daily checkins


Learning a lot


What we didn't
like:


Scope creep


Loss of
resources


Cramming at
end


Not enough
time for
testing/QA


Working a lot

Demo


[demo] of SF State Drupal Portal
http://inside.sfsu.edu/portal


http://drupal.sfsu.edu/drupal

Architecture


2 Dedicated RHEL 5 boxes run Apache (Xeon 2 ghz + 4 GB RAM)


2 Dedicated RHEL 5 boxes run MySQL (Xeon 2 ghz + 4 GB RAM)


1 Dedicated box runs MySQL cluster mgmt node (+ MMM
replication)


Load balancer in front + 2 mysql db connection scripts

Big IP Load Balancer

MySQL Cluster 7.x

MySQL Cluster 7.x

MySQL

Mgmt Node

MySQL DB

connection test

script

MySQL DB

connection test

script

Apache+PHP 5.2

Apache+PHP 5.2

Drupal & MySQL Cluster

:(

Drupal will not run out of the box on ndbcluster tables

Problem: menu_router table and locale_sources tables

A bug was filed with MySQL
, rejected because ndbcluster
doesn’t support row lengths of that size

Issues on d.o. filed to address this in core
http://drupal.org/node/391130

and
http://drupal.org/node/703916

Split the menu_router table into smaller, more manageable tables

Remove the key reference on locale_sources or change the key

data type to be something other than text/blob

Workaround at http://www.coderintherye.com/

SF State Custom Modules

User Interface Modules

Address change

Benefits pps

Distribution Groups management

Emergency contacts pps

Ethnicity resurvey

Financial Aid Offer Letter

Financialaidofferletter admin

Military status

Name Change Self Service

Nextmuni

Official Transcript Request

SF State Calendar viewer

SFState Library help

SF State News

Student schedule viewer




Utility Modules

connectioncheck

deletecookies

encryption

langhide

Scheduled message

Show more filter

Shibboleth data mapper

sfsulogout

shibbolethdata

validator

xmlservice



Getting the Campus Community
to Develop


Offered two trainings


1) How to install Drupal on your localhost


2) How to create Drupal modules


3) (Future) How to create web services


Need hand
-
holding


Monthly meetings

and code workshops

How to get a Module (portlet) into Portal

First content should be discussed with the oversight group
before beginning development to ensure Portal is the
correct place for it

Some departments or groups may have a page in Portal
that they are responsible for and control the content for

All changes must go through the development cycle
including unit and load testing

Security testing of code (Plug for “Cracking Drupal”)

Clean up and review of code

Development cycle


Develop code


Run unit tests


Check into SVN


Additional development


Interface testing


Tag in SVN


Verify automated tests


User acceptance testing


Content is added directly here


Management approval


Automated tests run


If any tests fail, there is no push to
production


Automated tests


If any tests fail, system is rolled back to
previous build

Prepare

Define requirements including functionality and UI

Remember you are working within the Portal template so many UI elements are
predefined

Define interfaces to backend systems

The most common interfaces are



REST/ WebServices that should follow w3 standards


LDAP (V3)

Determine where portlet will be located and who should have access


Local development

Build portlet

Run unit tests

Unit tests should include all UI components and all backend interfaces

Check into svn

Request dev build


Backend

Work with the data and application owners of the source systems to
build the required backend interfaces

Make sure you follow the defined security model if there is one, or
work with us to define one if the system you are connecting to has no
model defined


Set up web services


Development (DEV)

Run unit tests

Run integration tests, finalize development

Peer review all code

Tag SVN code for test

Request Test build

Code is pulled from SVN into Dev



Test/ Quality Assurance (TEST)

Define test plan and execute all functional tests

Tests must include all functions, UI components, and backend interfaces

Perform user acceptance testing

Perform load/performance testing

Submit automated test suite for Staging and Prod

This test suite must be comprehensive

Make any content only changes directly in Test

A manager must sign off indicating the following:

He/she is aware of all changes going into production and approves them for the
specified date

The automated tests sufficiently cover all functionality

All code has been peer reviewed

The load/performance tests prove that the application meets the minimum requirements

Tag code in SVN for roll out

Code is automatically pulled from TEST at 1am and deployed into staging


Staging

Automated test suite is run

If any tests fail, production roll out is not performed

If all tests pass, promotion to Prod occurs automatically at 6am


Production (PROD)

Automated tests are run

If any tests fail, production is rolled back to previous release



Accessibility


508 compliant


All pages were run through manual checks


Theme is color contrast compliant


Testing out using the accessible_content
module developed at CSUMB

Security



IP based Access for back
-
end services


Everything is behind a firewall


Everything is behind Shibboleth auth


Core PHP Module is removed


Role based access to pages, panels


Daily log checks (in future, archive logs in
central change mgmt repository)


Grendel
-
Scan


OWASP standards

Scalability


We are not very big

Handled ~500 concurrent users without
problem


Peak ~5,600 visitors/day


357,000 visits and 1.7 million page views
from March
-
May

Shibboleth single
-
sign on & Google
Analytics


Servers are way underutilized, use about
15% cpu on norm


Load balance / Auto failover (actually
tested)


No outages yet over 6 mos. (Though
some performance issues that required
caching content)

Performance


Initial Build took 7.5+ sec. Load times


Used Xdebug to find drupal_http_request
was problem. Used cron to cache


Down to 3.5 sec load times.


Installed APC via PECL (quite easy)


(Make sure to give it enough memory)


Down to ~1.5
-
2 sec load times.


Ensure Oracle db connects quit if can't
reach (need to upgrade to 11g)

Where we are Now vs. Where we
want to be


Now campus websites
are fragmented,
outdated


Not all sites use Univ.
Template, and some
don't use it correctly or
update it timely


Depts want to move to
central hosting


We have a monthly
meeting Drupal User
Group


Depts. want to put
sites in centrally
managed highly
available Drupal install
for easily keeping sites
up to date


1 Drupal theme means
a consistently updated
look


Need installation
profile for point & click

Tools to consider


Aegir (still maturing when we looked at)


Open Atrium


Pressflow


Memcached/Boost


Cloud hosting via Acquia/AWS or
Rackspace as well as consultants

Drupal is not the solution for
everything (though I love it)


Drupal is not


our document management system
(though it could be someday) We use
Sharepoint


our project management system (again
Sharepoint)


our help desk ticketing sys (remedy, gia)


It can be made to do all these things, but
other products are out there

The Importance of Community



Drupal.org


Bay Area / Berkeley / South Bay DUGs


IRC: #drupal
-
support or #drupal


On campus:
http://drupal.sfsu.edu/drupal


Monthly Drupal meetups (now workshops)


California Higher Education Group:
http://groups.drupal.org/california
-
highered

California Higher Ed Drupal Sites


California Institute of the Arts


CSU Monterey Bay portal (login
required)


CSU Monterey Bay Catalog


Moss Landing Marine Laboratories


San Francisco International Airport
(SFO)


SF State University Portal (login
required)


SF State Drupal Sites (login
required)


Stanford University Sites using
Drupal


UC Berkeley
Drupal Sites


UCLA Graduate
Students
Association


UCLA Semel
Institute


UCLA Family
Commons


UCLA School of
Public Affairs


UC Merced:
Although not our
main site yet. We
have 60+ sites
live and 40+ sites
in development.


UC Irvine Henry
Samuel School of
Engineering


UC Irvine School
of Social Ecology


UC Irvine School
of Social Sciences


UC Irvine Claire
Trevor School of
the Arts

Questions?


Contact:
kobrien@sfsu.edu


@nowarninglabel on twitter or on d.o.


http://www.coderintherye.com