Control System Overview

shrubflattenUrban and Civil

Nov 25, 2013 (3 years and 8 months ago)

177 views

1

BROOKHAVEN SCIENCE ASSOCIATES

Control System Overview

L. R. Dalesio

EPICS Collaboration

March 11, 2008

2

BROOKHAVEN SCIENCE ASSOCIATES

Outline


Requirements


Standard Control System Components


Development Opportunities


Embedded Device Control


Client/Server Architecture for High Level Applications


Relational Database and tools for all configuration
parameters


Near Term Plans


Scope of Work and Budget


Concluding remarks

3

BROOKHAVEN SCIENCE ASSOCIATES

Control System Requirements


1 of 3


Bunch Length 1
-
40 psecs


2.6 usec ring revolution


Top off every 1 minute


Top off bunch train 140
-
300 nsec


Top off damping time 10
-
50 msecs (no extraction)



Synchronous reading/writing

(latency?jitter?future settings?rate?)



Manual control of orbit trims,


quadrupoles, sextupoles, and insertion devices are
asynchronous


~10 Hz write/read is suitable for

turning knobs


for a power supply


5 Hz updates to operators of up to 1000 chosen parameters


Archive up to 6000 parameters at a rate of 2 Hz continually


What is done elsewhere?



Must scale to support 150,000 physical I/O connections and 400,000 computed
variables


99.99% availability 24/7

4

BROOKHAVEN SCIENCE ASSOCIATES

Control System Requirements


2 of 3


Transient Recording


Take coherent turn by turn orbit data for up to 800

channels 1024 turns


Latch the last 10 seconds of data from all parameters in the storage ring


Others?


Beam line needs 1 msec archiving over 1 minute for temperatures and positions


Provide data for all control aspects


5 KHz RF Feedback on beam phase


10 kHz orbit feedback, (100 usec loop time)


300 BPMs (10 per cell)


2 * 120 Corrector PS in 90 I/O Controllers (IOC)


20 msec equipment protection mitigation
(too long?)


1 Hz model based control


10 kHz power supply read backs triggered from timing sys


10

s of Hz Data Collection for RF loop correction.


80 psecs pulse to pulse timing jitter.


During top off, some beamlines will need 1.1
-

1.8 psecs of timing jitter


Machine physics thinks this number should be < 2 psecs


Provide electron detector as event for beam line?

5

BROOKHAVEN SCIENCE ASSOCIATES

Control System Requirements


3 of 3


First need dates


2008/03


vibration tests on girders done using stand alone software.
Some vibration sensors will be used online.


2008/06


support libera BPM evaluation


Where do the rest of the diagnostics fit


Where does the conventional facility demonstrator fit


Where does the power supply test stand fit


2008/08


X
-
9 upgrade for NSLS beam line


2008/09


low level RF lab support


2009/06


support water systems


2009/09


support thermocouple measuring


2009/09


support vacuum test stands


2010/03


support high power RF


Control Room


Must be able to be relocated


Control room in the LOB for commissioning


Control room re
-
located with accelerator team for operation


6

BROOKHAVEN SCIENCE ASSOCIATES

Nomenclature Standard

Psy:PI
-
Ssy:SI
-
Tsy:TI<Dev:DI>Sg:SgI
-
SD

Control System Naming Examples

S:C30
-
VA:G1<SGV:A>Pos:1
-
Opn

S:C20
-
FE:B<FV>Pos:1
-
Cls

S:C20
-
VA:G1<IP:A>E:1
-
RB

S:C20
-
VA:G4<CHMA:B>

S:C20
-
VA:G4<BLWR>

S:C24
-
RF:G1<TMP:B>

S:C24
-
RF:G1<CCG:B>P

S:C24
-
VA:G1<CCG:D>P


System


Device Signal

Comment

cell 30, vacuum, girder 1, sector gate valve A (upstream), open

cell 20 BM front end fast valve

1st sputter ion pump at C20 girder #1

2nd aluminum chamber at C20 girder #4

rf shielded bellows at C20 girder #4

TMP station at 2nd SC cavity insulating vacuum

cold cathode gauge at 2nd SC cavity insulating vacuum

cold cathode gauge at 2nd SC cavity beam vacuum

7

BROOKHAVEN SCIENCE ASSOCIATES

Control System Software Standards


EPICS based


Embedded Real
-
Time Operating System choices: RTEMS which is
in use at LCLS, Spear, and CLS or VxWorks which is in use at APS,
SNS, Diamond, and SLS.


Linux Workstations


Use of standard EPICS engineering tools: Extensible Display
Manager, Channel Archiver, Striptool, Alarm Handler


Evaluate the possibility of participating in the CSS development


Visual Database Configuration Tool w/ Modifications for Table Entry


Use of physics applications: Matlab Middle Layer Toolkit (MMLT)
which is in use at LBL, Spear, Diamond, SLS, CLS, and ASP, and
eXtensible Accelerator Language (XAL) which is in use at SNS and
Elegant which is in use at APS.


Use of Beam Line Applications:
Matlab in place of SPEC
, BluIce,
IDL, SynApps

8

BROOKHAVEN SCIENCE ASSOCIATES

Control System Architecture

Shared Memory

Ethernet


EPICS Channel Access Protocol

Events/ / Timing Data

P
L
C

C
N
E
T


I
/
O

Field I/O

Field I/O

…..

Instrumentation Ethernet

A
L
S

N
E
T

Field I/O

Field I/O

…..

PLCS ,Slow I/O, High Reliability,
Low Accuracy, High Density
Vacuum, PPS, MPS, Non FOFB PS,
Cryo., Facility control

C
P
U

E
V
R

I
/
O

I
/
O

M
T
R
S

Read Remote Ethernet and serial
devices. High density IO, Motor
Control

Position Control, Image
Acquisition, Non FOFB
Diagnostics

Operator stations: Displays,
Archiving, Alarm Management,
Strip charts, Save/Restore Utility

C
P
U

E
V
G

C
EL
L

Timing Master



C
P
U

E
V
R

C
EL
L

BPM IOCs



C
P
U

E
V
R

C
EL
L

PS IOCs



BPM

BPM

BPM

BPM

BPM

BPM

PS

PS

PS

PS

Fast
Equipment
Protection
Signal

9

BROOKHAVEN SCIENCE ASSOCIATES

Control Hardware Standards for Test Stands


Dell Linux development workstations


PLC Solutions



Allen Bradley

Control Logix



Allen Bradley

FlexIO


Building Automation



ALS


Needs fast Ethernet based interface board


VME crates



Rittan 4 slot

VME64x
-
2U4S
-
PS300C
-
SM


4,395.00



Rittan 7 slot

VME64x
-
4U7S
-
PS900C
-
SM


5,510.00



Weiner 9 slot

VME195xPO



6,050.00



Weiner 21 slot

VME6023/611_JL



7,271.00


CPU Boards



Motorola


MVME 5500



3,000.00





PP410




3,500.00


Motor Controllers



Hytec


8601 with motor driver

32 axis

~1,000/axis



Newport XPS

Intelligent motion controllers

10

BROOKHAVEN SCIENCE ASSOCIATES

Areas For Technical Development


Embedded Controllers need an open standard for high
speed, deterministic functions. Work with other labs and
board manufacturers to develop one.


High Level Applications currently tie together functions
through data or file structures. To make the components of
High Level Applications modular and distributed, a
client/server architecture is needed.


Relational Databases support data management through
the life of a project. Development of adequate tools to
enter and report this data is required early.

11

BROOKHAVEN SCIENCE ASSOCIATES

Control Hardware For Beam Synchronous Applications

2 GB Shared Memory


Communicates Cell Data @ 20 usecs

Ethernet


EPICS Channel Access Protocol

Events/ / Timing Data


Beam Trip Event, Synch Data Event

C
P
U

E
V
G

C
EL
L

Timing Master



Physics Applications: Correlation
plots, orbit calculations, beam
dump data

C
P
U

E
V
R

C
EL
L

BPM IOCs



C
P
U

E
V
R

C
EL
L

PS IOCs



BPM

BPM

BPM

BPM

BPM

BPM

PS

PS

PS

PS

Fast
Equipment
Protection
Signal

Compute Fast
Orbit Feedback

12

BROOKHAVEN SCIENCE ASSOCIATES

Embedded Controllers
-

Per Cell

BPM

Controller

RF Clock Distribution

45 MHz

Fiducial Distribution

1 Hz

BPM

Controller

BPM

Controller

BPM

Controller

BPM

Controller

BPM

Controller

100 MB ENET

PS

Controller

PS

Controller

PS

Controller

PS

Controller

VME I/OC EPICS








Core

Controller

VME Interface

VME I/OC EPICS








Next Cell

Core

Controller

VME Interface

VME I/OC EPICS








Prev Cell

Core

Controller

VME Interface

2 GBit ENET

+T0

+
-

Settling time on BPMs

+0.0 usecs
-

BPM to Compute Controller


384 bits = 64 bits * 6 BPMs


4.9 usecs = 384 bits over 100 MBit enet

+4.9 usecs
-

Compute Controllers to each other


10,290 bits = 30 nodes * 384 bits


10.5 usecs = 10,290 bits over 2 GB enet

+15.4 usecs


Compute local matrix


0 usecs

+ 15.4 usecs
-

Communicate t Power Supply Controllers


24 = 4 PS * 4 bytes each


3.5 usecs

+18.9 usecs


loop complete


settling time for magnets


communicate diagnostic waveforms etc…

+200 usecs


start again

13

BROOKHAVEN SCIENCE ASSOCIATES

Embedded Controllers
-

Approach


Develop a prototype cell controller:


Redundant 2 Gbit communication paths for peer to peer communication


RF timing signals


Verify communication and timing jitter meet requirements


Develop the interface from the cell controller to a processor for
integration into EPICS


In parallel:


Develop the inexpensive device controller with redundant 100 MBit controllers


Develop the 100 Mbit receive and transmit circuits for the Cell Controllers


Develop the EPICS interface to the Cell Controller


14

BROOKHAVEN SCIENCE ASSOCIATES

Embedded Controllers
-

Status


FY 08


Purchase order in place with LBL (Alex Ratti and Larry Doolittle)


Develop a prototype cell controller:


Redundant 2 Gbit communication paths for peer to peer communication


RF timing signals


Verify communication and timing jitter meet requirements


Develop the interface from the cell controller to a processor for integration into
EPICS


FY 09


Develop the inexpensive device controller with redundant 100 MBit controllers


Develop the 100 Mbit receive and transmit circuits for the Cell Controllers


Begin integration of these device controllers


Libera


Power supply control


LLRF control

15

BROOKHAVEN SCIENCE ASSOCIATES

L
attice

O
ptics

Beam Resp.

Matrix (
S
)

Model Data

Measured

Orbit

Orbit

Differences

State Data

Application / Family


Control System / Channels

I/O Controllers

Beam Resp.

Matrix Diff’s

Optics

Deviations

EPICS Client/Server

Gradient Errs

& Corrections

Name

Mapping

Mid Level Model Client/Server

Configuration

Parameters

Physics

Applications

Conversions

Need to develop

Existing

Need to port

Computed Data

High Level Applications


Client/Sever

16

BROOKHAVEN SCIENCE ASSOCIATES

High Level Applications
-

Approach


Install an operational NSLS II simulation using Matlab Middle Layer
Toolkit


Define an interface library for physics applications to communicate to the
configuration parameters, calibrations, model data, and measured data.


Port some subset of the available packages (Matlab Middle Layer
Toolkit, Elegant, XAL, Dimad, and SAD) onto the interface library one
application at a time.


Develop a client/server implementation of the interface library.


Use an RDB to store configuration parameters and calibrations.

17

BROOKHAVEN SCIENCE ASSOCIATES

Relational Database


Support all configuration data


Provide tools as they are needed throughout the
project.


Make the RDB the source of all configuration data and
derive it from there for all uses.

18

BROOKHAVEN SCIENCE ASSOCIATES

Relational Database
-

Approach


Use the IRMIS database as the basis for our component and wiring
database which is currently in use at APS, SNS, CLS, Diamond, and
SLS.


Extend the database to support Optics early through a contract with
Tech
-
X in conjunction with our physicists and control group.


This contract includes a name mapper and lattice input / report tool


Develop tools through contracts, collaboration, and the hire of a second
RDB expert.


We are preparing SOWs for Naming Tool and Wiring Input


Schedule the time and put in the project rigor to use the RDB and tools
as new portions of the project are needed.


19

BROOKHAVEN SCIENCE ASSOCIATES

WBS Dictionary:
Hardware and software for the global controls of the linac, booster ring,
transport lines, storage ring, network, relational database, data archiving, timing, fast
feedback, and physics application support.

WBS 1.03.05 Global Controls System

~ 10% of Accelerator

WBS Description



Direct $

FTEs/yr

Total FTE Costs

Total Material

1.03.05/01


1.03.05.09

Apps: Diagnostics, RF, Power Supply, Vacuum
Subsystem

Integration of: Insertion Devices, Beam lines,
Safety Systems, Facility

7

5.3 M

1.9 M

1.03.05.12 Timing

1.5

1.2 M

0.9 M

1.03.05.11 & 1.03.05.13

High Level Applications

6

4.8 M

0

1.03.05.10 Accelerator Control Room

75K

0.5 M

1.03.05.11 Network

.5

0.3 M

1.3 M

20

BROOKHAVEN SCIENCE ASSOCIATES

Near Term Plan


Start R&D


Contact Nearly in place for development of open
-
source embedded device controllers with
LBL team (Larry Doolittle and Alex Ratti)


RDB server is purchased and we are ready to Install IRMIS


Tech
-
X contract ready to go out the door to extend IRMIS to support Optics


Position open for someone to Develop RDB tools for entry and reports


Installed NSLS II Simulation through IOCs


Installed Matlab Middle Layer Toolkit for Commissioning


Two positions in place to Start Development of API for High Level Applications (one starts in
May, the other position should be filled by March)


Make Preliminary Hires


RDB job ad is placed. A candidate has been indentified and could start by March 15
th
..


High Level Applications Engineers


Guobao Shen starts May 10, 2
nd

position starts by
March 15
th
..


2 positions are open for project engineers


1 position is open for an embedded systems expert


Completed preliminary design


Completed extensions to Visual DCT for Spreadsheet based input

21

BROOKHAVEN SCIENCE ASSOCIATES

Concluding Remarks


Global requirements are complete


Equipment control S/W and H/W infra
-
structure is selected.


A collaboration of LBL and SLS EEs along with NSLS II team are
prepared to begin the open
-
source, embedded device controller
development


RDB tables and tools have begun building on the success of IRMIS.


High Level Application environment is selected. The Diamond simulation
running under EPICS is installed along with the Matlab Middle Layer
Toolkit as a starting point at KEK by Guabao Shen.


The API for High Level Applications is being developed in conjunction
with MMLT, SDDS, and XAL authors.


Staff Acquisition is a critical activity and we have made good
progress.