Breakout: XNAT System Administration - XNAT Documentation Wiki

hipshorseheadsServers

Nov 17, 2013 (3 years and 10 months ago)

483 views

XNAT System
Administration

Chip
Schweiss

chip.schweiss@wustl.edu


June 26, 2012

XNAT System Administration

1.
System Requirements

2.
Java, Tomcat &
PostgreSQL

Essentials

3.
Scaling
Vertically & Horizontally

4.
An Automation Demonstration with VMware
and Puppet

5.
ZFS file
system

6.
Network Monitoring


XNAT System Requirements


Highly dependent on size of data set and
volume of users


Minimum for a working build:


2 GB free RAM


Tomcat 6 on Java 6


PostgreSQL

9.0 (9.1 not yet supported)

XNAT System Requirements


Two Examples:

1.
CNDA


A large instance with many
users

2.
XNAT Central


Small instance with light
usage


CNDA


System Specifications


Hosted on our VMware
vSphere

5 cluster


3.46 GHz Core i7 hex core CPUs


10
GbE


BlueArc 15k SAS pool NFS backing


CNDA Virtual Machine


8
vCPU


20 GB ram fully reserved


20 GB
vmdk


Separate VM for
PostgreSQL

CNDA
-

System Specifications

NAS File Storage


34 TB BlueArc NL
-
SAS + 34 TB BlueArc on DR site


45 TB ZFS backup

Sun Grid Engine Processing Cluster


10
-

4 core i7 2.4 GHz systems w/ 8 GB ram


6


8
vCPU

virtual machines w/ 16 GB ram

Supporting VMs


Shadow for database related processing


Files system shadow for
cron

triggered reporting


Development VMs


XNAT Central


System Specifications

XNAT Central Virtual Machine


4
vCPU


4 GB ram

Separate
PostgreSQL

virtual machine

NAS File Storage


2

TB BlueArc + 2 TB BlueArc DR


3 TB ZFS Backup

Java, Tomcat &
PostgreSQL

Essentials

Java & Tomcat:


Memory settings


Xmx



Maximum memory allocation


Xmn



Minimum allocation


Set to 1/5
Xmx


Xms


Initial heap size


Set to 1/3
Xmx


XX:MaxPermSize


Permanent memory allocation for classes & libraries


Set to 256m for XNAT

Java & Tomcat

Additional configuration:


Debugging w/ Eclipse

-
Xrunjdwp:transport
=
dt_socket,server
=
y,suspend
=
n,address
=8000


Running on port 80/443 with
modjk

server.xml:

<Host name="
localhost
"
appBase
="/
var
/lib/tomcat6/
webapps
/empty"


unpackWARs
="true"
autoDeploy
="true"


xmlValidation
="false"
xmlNamespaceAware
="false
">


<Context path=""
docBase
="/
var
/lib/tomcat6/
webapps
/
xnatcentral
">


<
Resource name="
UserTransaction
"
auth
="
Container“



type
="
javax.transaction.UserTransaction
"


factory
="
org.objectweb.jotm.UserTransactionFactory




jotm.timeout
="60
"/>



<
Manager pathname=""/>


</
Context>


DICOM Port

Java on Linux does not support listening on
ports < 1024 unless running as root


IP Tables makes an easy work around:


i
ptables


t
nat


I PREROUTING

p
tcp

\



--
dport

104
\



-
j DNAT

-
to
-
destination
<host
ip
>
:8104


PostgreSQL

Tuning

Performance parameters:


max_connections

= 100


shared_buffers

=
xxxMB



¼ memory available


work_mem

=
xxxMB


Typically 50 to 80 MB


m
aintenance_work_mem

=
xxxMB


Typically 128 to 250 MB


e
ffective_cache_size

=
xxxMB


½ memory available

Scaling Vertically & Horizontally



Reasons to Scale Vertically


More users


Higher activity level


Real
-
time processing


Reasons to Scale Horizontally


Pipeline processing


Scheduled jobs



Automated Demonstration

Development, Test and Production Cycles


VMware
vSphere


Puppet


ZFS

ZFS File System


Developed by Sun Microsystems


Highly scalable file system


Scales in performance with SSD read and write caches


Scales to Exabyte file system sizes


Snapshots, compression,
deduplication


Writeable clones of snapshots


Extremely efficient backups with ZFS send/receive


Multiple OS support


OpenIndiana
: maintained fork of
OpenSolaris


FreeBSD


ZFS on Linux


Oracle Solaris


Network Health Monitoring

Preferred tools


vCenter


Nagios


Pingdom


Spiceworks

Useful links


Our ZFS
Backup Server

http://xnat.org/blog/category/xnat
-
hardware/zfs
-
storage
/


Build Java RPMs
for RHEL/Centos

http://
www.city
-
fan.org/tips/SunJava6OnFedora


Scale
PostgreSQL

horizontally

https
://
github.com/greg2ndQuadrant/repmgr


Puppet

http://www.puppetlabs.com


Apache Redirect HTTP to HTTPS
using
mod_rewrite

http://
www.sslshopper.com/apache
-
redirect
-
http
-
to
-
https.html


OpenIndiana

http://www.openindiana.org


ZFS on Linux

http://zfsonlinux.org


Spiceworks

http://www.spiceworks.com


VMware Perl Scripts

http://
www.virtuallyghetto.com