Convey System Administration Guide

boundlessbazaarΔιακομιστές

9 Δεκ 2013 (πριν από 3 χρόνια και 6 μήνες)

109 εμφανίσεις





i
















Convey System Administration Guide




November

2010

Version
1.
6

901
-
000002
-
000




















Convey Computer
TM

Corporation 200
9
-
2010
.

All Rights Reserved.

1302 East Collins

Richardson, TX 75081


Convey System Administration Guide v1.6


ii

The Information in this document is pro
vided for use with Convey Computer Corporation (“Convey”)
products. No license, express or implied, to any intellectual property associated with this document or
such products is granted by this document.


All products described in this document whose nam
e is prefaced
by “Convey” or “Convey enhanced

(“Convey products”) are owned by Convey Computer Corporation (or those companies that have licensed
technology to Convey) and are protected by patents, trade secrets, copyrights or other industrial property
ri
ghts.


The Convey products described in this document may still be in development. The final form of each
product and release date thereof is at the sole and absolute discretion of Convey. Your purchase,
license and/or use of Convey products shall be su
bject to Convey‟s then current sales terms and
conditions.



Trademarks

The following are trademarks of Convey Computer Corporation

in the United States and other countries
:

Convey

Computer

The Convey Logo

Convey HC
-
1


Trademarks of other companies

Red Ha
t and Red Hat Enterprise Linux are registered trademarks of Red Hat, Inc.

Fedora and RPM are trademarks of Red Hat, Inc.

SUSE and openSUSE are registered trademarks of Novell, Inc.

Linux is a registered trademark of Linus Torvald

Python is a registered tra
demark of the Python Software Foundation

GNU is a trademark of The Free Software Foundation

PERL is a trademark of the Yet Another Society DBA The Perl Foundation

Ubuntu and Canonical are registered trademarks of Canonical Ltd.

Debian is a registered trade
mark of Software in the Public Interest, Inc
.




Convey System Administration Guide v1.6


iii

Revisions

Version

Description

1.0

May 2008.
Original printing
.

1.1

August

2008.

New
system setup procedures.

1.2

Ma
y

200
9
.
Reworked for Beta Release of HW, OS, and SW
.

1.3

June

2009. Revised reboot ins
tructions

when coprocessor is hung
.

New scripts.

1.4

September 2009. Recovery DVD usage instructions. New SW repo
conventions.

1.5

April 2010. New licensing mechanism. New DVDs for updating
systems.

New cross
-
development system install instructions.

1.6

November 2010. DoWebUpdate command for updating Convey
servers. Web interface for adding/changing web access
users/passwords.




Convey System Administration Guide v1.6


iv

Table of Contents

1

Overview

................................
................................
................................
.....................

6

1.1

Intended Audience

................................
................................
................................
..

6

1.2

Topics Covered

................................
................................
................................
.......

6

1.3

Other Suggested Documents

................................
................................
.................

6

2

Installing Convey Software

................................
................................
.........................

7

2.1

Internet connection

................................
................................
................................
.

8

2.2

Convey Software Repositories for Cross
-
Dev
elopment Systems

.........................

8

2.2.1

Where to install the Convey .repo/.list files

................................
.....................

8

2.2.2

Downloading and installing the Convey repository descri
ption files

...............

9

2.3

Convey Server Software

................................
................................
.........................

9

2.3.1

Convey O.S. Installation

................................
................................
................

10

2.3.2

Installing Convey O.S. from a DVD

................................
...............................

10

2.3.3

Convey Firmware Installation

................................
................................
........

10

2.3.4

Convey Coprocessor Recipe Packages
................................
........................

10

2.3.5

Updating Convey OS, firmware, and Recipe Packages

...............................

10

2.3.6

Installing other Convey Software products on a Convey Server

..................

11

2.4

Installing Convey Software on Cross
-
Development Systems
..............................

11

2.4.1

Compatible Linux® Distributions

................................
................................
...

11

2.4.2

SELinux Considerations

................................
................................
................

12

2.4.3

Available Convey Software Development Packages

................................
....

12

2.4.4

Identifying the App
ropriate Installation Tool

................................
..................

13

2.4.5

Installing Convey Software Packages

................................
...........................

14

3

Updating Convey Software Packages

................................
................................
.....

15

3.1

Updating Convey Software Packages using a GUI

................................
.............

16

3.2

Installing older versions of Convey Software Packages

................................
......

16

3.3

Version Numbers and Package Filenames

................................
..........................

16

3.4

Manual Installation

................................
................................
................................

16

3.5

Additional Installation Notes for Ubu
ntu/Debian and openSUSE

........................

18

3.5.1

How to downgrade gcc and g++
................................
................................
....

18

3.6

Installing Convey Software without an Internet Connection

................................

18

4

Booting and System Management

................................
................................
...........

20

4.1

Overview

................................
................................
................................
...............

20

4.2

Powering Up the Convey

Server

................................
................................
..........

20

4.3

First Power
-
Up and Boot

................................
................................
......................

20

4.3.1

Intel Remote Management Module

................................
...............................

20

4.3.2

First Boot

................................
................................
................................
........

20

4.3.3

Manual Configuration

................................
................................
....................

21

4.4

Boot Options

................................
................................
................................
.........

21

4.4.1

enable/disable

................................
................................
................................

22

4.4.2

caemode

................................
................................
................................
........

22

4.4.3

memory

................................
................................
................................
..........

22

4.4.4

wind
ow

................................
................................
................................
...........

22

4.4.5

pagesize

................................
................................
................................
.........

22

4.4.6

interleave

................................
................................
................................
.......

23

4.5

Coprocessor Modules

................................
................................
...........................

23

4.5.1

CNYMP Module

................................
................................
.............................

23



Convey System Administration Guide v1.6


v

4.5.2

CNYSYS Module

................................
................................
...........................

23

4.6

Kernel Configuration

................................
................................
.............................

24

4.7

Convey System Services

................................
................................
......................

24

4.7.1

Convey Configuration Files

................................
................................
...........

25

4.8

Copro
cessor Sanity Test Program

................................
................................
.......

25

4.9

When to Shutdown the Convey Server

................................
................................

26

4.10

Activating a Coprocessor Recipe

................................
................................
.........

26

5

Convey System Memory Management

................................
................................
....

27

5.1

Overview

................................
................................
................................
...............

27

5.1.1

Intended Audience

................................
................................
.........................

27

5.1.2

Terminology

................................
................................
................................
...

27

5.2

NUMA Memory Configuration

................................
................................
...............

27

5.2.1

Physical Memory Configuratio
n

................................
................................
.....

28

5.3

Address Extension Feature

................................
................................
..................

28

5.3.1

Fully
-
mapped mode

................................
................................
.......................

28

5.3.2

Dy
namic
-
mapped mode

................................
................................
................

29

5.4

Memory Interleaving

................................
................................
.............................

29

5.4.1

Binary Interleave

................................
................................
............................

30

5.4.2

31
-
31 Interleave

................................
................................
.............................

30

5.4.3

31
-
31 Interleave Page Allocation

................................
................................
..

30

5.4.4

31
-
31 Interleave Faults

................................
................................
..................

31

5.5

Variable Page Size Support

................................
................................
.................

31

5.5.1

Maximum Coprocessor Page Size

................................
................................

32

5.5.2

Linux HugeTLB Feature

................................
................................
................

3
2

5.6

Memory Migration

................................
................................
................................
.

32

6

Personality Management

................................
................................
..........................

34

6.1

Personality Signa
tures

................................
................................
..........................

34

6.2

Default Base Signature

................................
................................
.........................

34

6.3

System Default Signatures

................................
................................
...................

35

6
.4

Convey Personality Management Script

................................
..............................

35

7

Recovery DVDs

................................
................................
................................
........

37

7.1

How to use the Convey System Recovery DVD

................................
..................

37

7.1.1

Preparation

................................
................................
................................
....

37

7.1.2

Boot from DVD

................................
................................
...............................

37

7.1.3

Configure the Installation and Installing the
OS

................................
............

38

7.1.4

System Configuration

................................
................................
...........................

38

7.2

Fixing the Convey SW Repository Files

................................
...............................

39

7.3

Adding Convey Provided License Keys

................................
...............................

40

7.4

Installing the Remaining Convey Software Products

................................
...........

40

7.5

Checking the Managemen
t Processor

................................
................................
.

40

7.6

Updating the Server to the latest SW Versions

................................
....................

41

7.7

Restoring a Convey Server without a DVD drive

................................
.................

41

8

Customer Support Procedures

................................
................................
.................

42

9

Licensing

................................
................................
................................
...................

43

9.1

Convey Licensed Software

................................
................................
...................

43

9.2

Old Licensing Scheme

................................
................................
..........................

43

Appendix A
-

Installing YUM on older SUSE™ Distributions

................................
...........

44

Index .

................................
................................
................................
...................

45



Convey System Administration Guide v1.6


6

1

Overview

This guide describes how to perform various tasks that are typically performed by a
system

administrator
. These tasks require system administrator privileges.

Please note that thi
s document contains clickable hyperlinks. If you are browsing this document
on Convey‟s web site, or downloaded
all

of Convey‟s documents to the same directory, those
clickable hyperlinks
should

work. If you only download a single document, clicking on t
he
hyperlinks will fail.

1.1

Intended Audience



Administrators of a Convey hybrid
-
core server



U
sers of desktop Linux
®

systems (who have root access) used for cross
-
development of
applications for a Convey
hybrid
-
core
server



A
dministrators of customer provided L
inux
®

systems used for cross
-
development of
a
pplications for a Convey
hybrid
-
core
server

1.2

Topics Covered

How to update Convey server specific software, including:



Convey OS and patches



Convey server firmware

How to install and update

other Convey software

o
n a cross
-
development system or a Convey
server
, including the



Convey
c
ompilers,
a
ssembler,
l
inker, and system libraries



Convey provided coprocessor personalities



Convey documentation

How to reboot a Convey server

How

to check for updates
for
Convey
‟s

soft
ware packages

Personality Management

Recovering a Convey Server using the recovery DVDs

Customer Support Procedures

Licensing

1.3

Other Suggested Documents



Convey HC1 Service Guide



Describes how to install, upgrade, and troubleshoot the
HC1 hardware



Convey System Administration Guide v1.6


7

2

Convey’s Support Website

Convey provides a technical support website for customers:
http://
www.conveysupport.com
.

This website contains a variety of useful information for developers, system administrators,
installers, etc.

A Convey provided web access user name and password are required to access the site,
althoug
h anyone with such a username/password can create new web access usernames, by
visiting
http://www.conveysupport.com/protected/webaccess.html
.

The most recent versions of all Convey docume
ntation is provided, as well as older versions of
most documents.

Support bulletins, FAQs,
a bug

reporting/tracking system,
software repositories, and other
resources are available.

All users of Convey servers or development software tools should have acce
ss to the Convey
support website.






Convey System Administration Guide v1.6


8

3

Installing Convey Software

In general, Convey recommends using the Convey provided software repositories for
most

Convey
software
installation
s
.

These repositories are accessed using
the standard Linux
software installation tools (i.e.
yum
, zypper, apt
-
get)
, and require the
system

to be connected to
the internet.

Users with a large number of Convey servers or cross
-
development systems may
wish to create a local copy of the Convey soft
ware repositories, or download the individual
packages and install them with the low level package installer provided by their Linux distribution.
See
Installing Convey Software without an Internet Connection

f
or more details.

Some Convey software products require a license to be installed in order to
use

that product. As
of the April 2010 release, all Convey software products may be installed on a compatible system
before the license is installed. Convey serv
ers are delivered with appropriate licenses already
installed, based on the sales order for the server. Licenses for cross
-
development systems must
be installed by a system administrator. Typically, Convey will email the contact
person
identified
on a sa
les order for a cross
-
development system with instructions for
obtaining
the appropriate
license(s).

3.1

Internet connection

Most of the

directions below assume

that
a working internet connection
exists
. If a proxy server is
required, the
appropriate

proxy con
figuration
should

be updated

for the software installation tool
of choice
1
.
In addition, the
wget

command requires that the system‟s network settings correctly
specify any required proxy for http access to conveysupport.com.

If the system does not have a
working internet connection, or you wish to install the Convey
software from a Convey Software DVD, s
ee the section titled

Installing Convey Software
without an Internet Connection
.

3.2

Convey Software Repositorie
s

for Cross
-
Development Systems

On user provided cross development systems,
the
Convey s
oftware repository

description files

must be
installed

before other Convey software ca
n be installed from those repositories
.

On
Convey servers, the software repos
itory description files are already
installed.

3.2.1

Where to install the Convey .repo
/.list

files

On rpm based Linux distributions

(Redhat, SuSE, Fedora
, CentOS, Convey‟s OS
)
,



Reposit
ory description filenames have a
.repo

suffix



A
ll
repository description files are located in either

/etc/yum.repos.d
/

(
for Redhat,
CentOS, Convey‟s OS,
Fedora, and older Suse/OpenSUSE
systems),

or

/etc/zypp/repos.d
/

(for newer OpenSUSE systems)
.



T
he Conve
y
.repo

files should be installed in whichever directory exists and
already
contains other
.repo

files
.

On Debian based Linux distributions

(including
Debian and
Ubuntu)
,



1

For Convey Servers, see
/etc/yum.conf



Convey System Administration Guide v1.6


9



Repository description filenames have a
.
list

suffix



T
he repository description file
s are located in
/etc/apt/sources.list.d/
, and
the Convey
.list

files should be installed there. This directory may be empty before installing the
Convey
repository description files.

3.2.2

Downloading

and installing
the Convey
repository description
files

Once

you know
which direc
t
ory

to install Convey‟s
repository description
files

in
, you may
download them from the Convey Support website by:



Log
g
in
g in
to the system you want to install Convey‟s software products on,

as the
root

user



cd …

(
where


is
the direct
ory that contains all of your system

s
repository description
files, as described above)



wget http://conveysupport.com/repofiles/hc1
-
1/filelist



wget
-
i filelist



On an RPM based system, delete the unneeded
.list

files just downloaded, and on a
Debian based

system, delete the unneeded
.repo

files just downloaded



See the
wget

man page for options required when an http proxy server must be used



If the
wget

utility is not
already
installed on your system, most Linux distributions provide it

in the vendor‟s soft
ware repository.

Alternatively, you can click on each
repository description
file in your browser, and copy/paste the
contents of the file to the same name file on your Linux system.

After installing the Convey repository description files, each of those f
iles must be edited, and the
strings “
username

and


password
” must be replaced with the username and password provided for
your use when any Convey software product was purchased. E
-
mail
support@conveycomput
er.com

for the name of the contact at your site that can provide the
appropriate username and password.


If your site is using a local copy of the Convey repositories, it may not be password protected.

In
this case, you should delete the string “
userna
me:password@
” from each repository description file.

3.3

Convey Server Software

If you are installing Convey‟s cross
-
development tools on a non
-
Convey system, please
skip
over

this
Convey Server Software
information
.

Con
vey servers come with the following items pre
-
installed

(
package names
are
in this font
)
:



Many
CentOS

packages



Convey OS package
s

(Convey enhancements to CentOS)

convey
-
kernel

(or

convey
-
kernel_beta
)
,
convey
-
platform_utils
,
convey
-
platform_base



Convey Firm
ware package

convey
-
mp_fw



Conve
y Coprocessor recipe package(s)

convey
-
reci
p
e
-
release



Convey coprocessor base image package

convey
-
aebase
-
image



Convey System Administration Guide v1.6


10



Convey
P
ersonality packages(s)
with an Application Engine FPGA bit image

(these are optional

licensed

products)

c
onvey
-
aesp

(where

sp


can be any Convey provided personality name)

as well as many of the Convey Software packages described
later
, that can be installed on both
Convey servers and non
-
Convey systems running a
compatible Linux distribution.

A
dditional licensed packages
may be installed with the standard
yum

package manager, from the
repositories on Convey‟s support website, or from a Convey Software DVD.

3.3.1

Convey O.S. Installation

A Convey server must have the Con
vey
provided

Linux
®

distribution installed. No other Linux
®

distributions may be installed on a Convey server.

The Convey provided Linux distribution is pre
-
installed on all Convey servers.
This

distribution

(and various updates)

is also available on
Co
nvey‟s support website,
http://conveysupport.com
.

Convey does not recommend installing Convey‟s Linux distribution on non
-
Convey systems.
Since Convey‟s Linux distribution is based on CentOS 5.
3
, CentOS

may

be instal
led on
cross
-
development
systems if you want the most similar Linux distribution on
those
system
s
.

3.3.2

Installing Convey O.S. from a DVD

If the Convey server includes the optional DVD drive, Convey‟s O.S. may be installed using a
Convey supplied
recovery
DVD.

You should never install Convey‟s O.S. from a DVD unless
instructed to do so by the Convey Support group (typically required only when replacing the main
disk in a Convey server). Please
contact Convey

if you

believe you need to reinstall the O.S. on
a Convey server.

3.3.3

Convey Firmware Installation

A Convey server, in addition to the Convey
provided

Linux
®

distribution, requires
the

Convey
firmware package

(
convey
-
mp_fw
)

to be installed.
Th
is package is

pre
-
inst
alled on all Convey
servers.

The Convey firmware
package

should not be installed on non
-
Convey systems.

3.3.4

Conv
ey Coprocessor Recipe Packages

Recipe packages contain

(or indirectly reference)

“bit image files” that are loaded into some of the
FPGAs that are
part of the Convey coprocessor. A recipe is a Convey tested set of these images
that are known to work well together. Typically, only the “released” recipe should be activated on
a Convey server unless a Convey support representative recommends installin
g another recipe.



The “released” recipe is
always in a package named
convey
-
recipe
-
release
.



Installing or updating

a receipe does not activate that recipe. Any installed recipe may be
activated with the
pickRecipe

or
activaterecipe

commands in
/opt/conv
ey/sbin
.

Recipe packages should not be installed on non
-
Convey systems.

Note that when a recipe package is installed, it copies the relevant files to
/opt/convey/fpgaimages, and then removes itself. The relevant files will still be present on the
Convey s
erver. This automatic removal allows older recipes to be installed after a newer recipe
has been installed.

3.3.5

Updating Convey OS,
firmware
, and Recipe Packages

Updates to all Convey server software packages may be performed with the
yum

tool, if
either
inte
rnet access

or a locally mounted Convey DVD repository is available
.

IMPORTANT NOTE:

Af
ter updating or installing any Convey OS,
firmware
, or recipe

packages
,
the
host
system need
s

to be
shutdown

for
at least two

minutes, and then powered up again
.

Any
package that requires a
shutdown

will indicate so during installation.




Convey System Administration Guide v1.6


11

For example, to update every Convey package installed on a system, logon the system as
root
,
and enter the command:


yum
update

convey
-
\
*

and
answer “
yes
” when prompted.

Occasiona
lly the
yum

database becomes out
-
of
-
date or corrupted, and
yum

won‟t be able to find or
update packages. If you see unexpected error messages while using
yum
, you can purge the
yum

database with the command:


yum
clean

all

New packages, such as new r
ecipe packages,
or newly purchased personality packages,
may be
installed with the command:


yum install convey
-
xxx

wh
ere
convey
-
xxx

is the package name to be installed.

3.3.6

Installing other

Convey Software products on a Convey Server

See the very next se
ction,
Installing
Convey Software on Cross
-
Development Systems.

3.4

Installing
Convey Software on Cross
-
Development Systems

Other Convey Software products, including

Convey
‟s

software development products
,

may be
installed on a Convey server or on a
ny

compati
ble
user provided Linux
®

system. When the
C
onvey compilers, assembler, linker,
personality tools,
and
,

optionally,
the
personality
development kit

are installed on a non
-
Convey system, they provide a cross
-
development and
debug environment that permits de
veloping and debugging code for the Convey server and
coprocessor, using a software simulator. See the
Convey Programmers Guide

for more
information.

Any other Convey
products/
packages that were purchased, such as the
Convey Math Libraries,
the
Personality Development Kit
,

or
a
Convey
Application Engine fpga image (for a particular
personality)
, c
an be individually installed.

If you are installing

or updating
Convey software products on a Convey server, you may skip the
section titled
Compatible Linux
®

Distributions

below
.

The directions below assume

that an appropriate Linux
®

distribution is alread
y installed on an
x86
-
64 system
.

If a proxy server is required, the
appropriate

proxy configuration
should

be
updated

for the software installation tool of choice
2
, or you may use the manual installation
instructions below
. See the section titled

Installing Convey Software without an Internet
Connection

for systems that do not have a working internet connection.

3.4.1

Compatible Linux® Distributions

The Convey software development products were developed and have
bee
n extensively

test
ed

on the
5.1,
5.2 and 5.3 releases of the CentOS and Red Hat
64
-
bit
distributi
ons. If you have no
preference

for any particular Linux distribution for your cross
-
development systems, Convey
recommends you install either the CentOS or Re
d Hat distribution.

The
Convey
PDK and other Convey software development products have been installed and
verified to work on an x86
-
64 system running the following 64
-
bit Linux
®

distributions
:

Red H
at®
SUSE™
Fedora™ Core
OpenSUSE™


2

For Convey Servers, see
/etc/yum.conf



Convey System Administration Guide v1.6


12

Desktop 5.1

Desktop 10

8

and 9

10.3

and 11.0

CentOS 5
.2/5.3

Ubuntu® 8.0.4

Red Hat
Enterprise

Server 5.1
-
5.3

Debian




Different

minor releases of these distributions are
also
likely to work
. Newer major releases of
these distributions

are likely to work if no substantial incompatibilities with respect to
libc
,
python
®
,
and
perl


are introduced

by the new release
.

In addition to the base distribution, the “Software Development”
super
-
package

was typically
installed (
x86
-
64 versions of
g
cc

and
g++

are required
)
.
Most
l
ive

distributions (Linux distributions
that boot off of a DVD or CD but are not installed on a hard disk) are likely to require additional
packages that
most DVD based distributions
include automatically.

Other packages tha
t are required, but typically included by default, are
python
®
, and
perl

.
yum
,
zypper,

or

apt
-
get

is
required

if you
want

to install and update software
using the Convey provided
repositories.

Other 64 bit Linux
®

distributions that have
gcc 3.2

(or newer) and a corresponding
g++

compiler,
GNU


libc

2.5,


a recent version o
f

zypper,
rp
m
or

apt
-
get
, and
python

2.4

installed are likely to
work well, but have not been tested by Convey Computers. Most up
-
to
-
date Linux
®

distributions
with a 2.6.xx kernel will meet the above criteria, after any necessary optional packages are
ins
talled.

Convey
s
oftware

development products are available

over the internet, as RPM


and Debian
®

packages.
These packages
may be downloaded individually, or installed using the provided
yum

and
Debian

repositories.

These packages are also available on a CD by request (
email your
request to
support@conveycomputer.com
).

3.4.2

SELinux Considerations

If SELinux is enabled on a cross
-
development system, please see the SELinux FAQ at
www.conveysupport.com/protected/faqs

for more information abo
ut configuring SELinux
(required in order for the Convey compilers to operate correctly).


3.4.3

Available Convey Software
Development
Packages

Users of Debian based distributions should note that any underscores in the package names
below will be replaced with
dashes in the Debian package name (i.e. convey
-
sp_per
-
> convey
-
sp
-
per).

Note that the full filename for a given package includes version and architecture information. The
rpm

packages have an architecture specification of x86_64, while the
d
ebian

package
s have an
architecture specification of amd64. These two architecture specifications are equivalent.
Convey software packages should only be installed on systems with x86
-
64 compatible
processors.



T
he
convey
-
1st

package
provides a few administrative sc
ripts

and
adds
/opt/convey/bin

to
everyone‟
s PATH



The
convey
-
per
-
tools

package
provides the Convey personality database and some
personality management tools



Packages containing
Convey compilers,
support libraries
, debugger, …

(install all of these
package
s together)



Convey System Administration Guide v1.6


13

o

convey
-
open64

(also installs the following)

o

convey
-
utils64

o

convey
-
utils64
-
opensource

o

convey
-
platform_libs



Convey personality packages for software development

o

convey
-
sp_per

o

convey
-
dp_per

o

convey
-
fap_per

o

convey
-
none_per

o

convey
-
pdk_per


(only
for use with the

convey
-
pdk

package)



Convey Personality Development Kit

for developing a custom
personality

o

convey
-
pdk




Convey math libraries


o

convey
-
cml





OpenMPI

o

co
nvey
-
openmpi



InfiniBand Drivers

o

convey
-
ofed
-
basic, convey
-
ofed
-
hpc, convey
-
ofed
-
all

(only install one of these)



Documentation not included in another package

o

convey
-
manuals

3.4.4

Identifying the Appropriate Installation Tool

One of the following installation tools

(or a compatible equivalent)

is required to utilize the Convey
repositories.

If
none of these tools are installed, either install the appropriate tool or skip to the
section titled
Manual Installation
.



yum



zypper



apt
-
get

On all Redhat, Fedora, SUSE, and o
lder openSUSE distributions,
yum

is the tool of choice. It
may be necessary to install
yum

on SUSE and openSUSE distributions (available on the
openSUSE installation media and from the openSUSE repositorty).

On openSUSE 10.3 and 11.x distributions,
zypp
er

is the tool of choice

and should be installed

by
default
.

On Debian distributions (including Ubuntu),
apt
-
get

is the tool of choice.

On all other distributions, check to see if one of these tools is available, or can be installed from
the repositories p
rovided by that distribution. If none of these tools can be used, skip to the
section titled
Manual Installation
.

Once you know which installation tool is available, follow the repository setup directions for that
to
ol below.



Convey System Administration Guide v1.6


14

3.4.5

Installing Convey Software Packages



Log

in as
root



The example commands below, for installing a package, use

the

yum

command
.

Substitute
zypper

or
apt
-
get

if

appropriate for your Linux distribution. Remember to
change “
_
”s to “
-
“s in package na
mes for Debian based distributions.



Install the
convey
-
1
st

package
first



yum install convey
-
1
st



Install any Convey provided licenses as instructed in the accompanying email
. Email
support@conveycomputer.
com

for your site‟s contact person who can provide
the
appropriate licenses

and instructions
.



Install the
C
onvey

personality tools package


yum install convey
-
per
-
tools



Install the C
onvey

compilers
, debugger, and other software development tools

(inst
all all 4
packages together)


yum

install


convey
-
open64





Install any of the personality packages you
n
eed


yum install convey
-
sp_per


yum install convey
-
none_per






Install any

licensed packages you have a valid license for
, such as


y
um inst
all convey
-
cml


yum install convey
-
pdk


yum install convey
-
sp
_ae



If you would prefer
/opt/convey/bin

to appear before
/usr/bin

in
$PATH

(and
$path
)

on

cross
-
development systems
,
execute (as root) the script
/opt/convey/sbin/preferoptconveybin
, log
o
ff your terminal, ssh, or telnet session and start a new
terminal
session. This script will
automatically be executed when certain Convey server specific software packages are
installed on a Convey server. The effect of this script can be undone by editi
ng the files
/etc/profiles.d/convey.sh

and
/etc/profiles.d/convey.csh
, and moving
/opt/convey/bin

to
the end of the path.



Convey System Administration Guide v1.6


15

4

Updating Convey Software Packages

4.1

Updating Convey Sofware on a Convey Server

On a Convey ser
ver, Convey provides a script to update most Convey SW packages (except
those that are customer specific/proprietary), and ensure that compatible versions of FPGA
recipes, AE personality firmware, management processor firmware, and other Convey software
ar
e installed, and the proper recipe is activated. This script may be used by:



Log in as
root



yum update convey
-
support
-
tools


if this command complains that there is "No Match for argument", install the package with:

yum install convey
-
support
-
tools




/opt/
convey/support/DoWebUpdate

This script updates all installed Convey provided software on your Convey Server,
including various firmware packages, the firmware recipe, coprocessor personalities,
development tools, the Convey OS and associated CentOS package
s. This script ensures
that the various firmware, recipe, operating system, and other packages are compatibile
and up
-
to
-
date.


4.2

Updating Convey Software on non
-
Convey systems

A
ll
previously installed Convey
software development packages and Convey server s
pecific
packages

may be upgraded to the
newest

release
d version

with

the usual Linux update
mechanism for your distribution. For Convey OS, this is:


yum update convey
-
\
*

Note that
yum‟s
metadata
cached

on
your

system
is typically reused for at least

an hour, and that
metadata may need to be reloaded from the Convey repositories if you are attempting to use a
repository that has just been updated. For Convey servers, the command



yum clean metadata


will flush the existing metadata, and the nex
t
yum

command will reload metadata from all the
repositories.

Also note that the database of installed software on most Linux distributions does not care where
a package has been installed. Therefore, if you install a newer version of a package in a locat
ion
other than the default, you
may need to

remove that version of the package before you can install
it in the default location.

On Debian based distributions, the
apt
-
get

command is used to update Convey packages.


apt
-
get


update




(flush
s

old me
tadata)


apt
-
get
upgrade

convey
-
\
*


On
systems where
zypper

is available, packages may be updated with:


zypper update
--
type package convey
-
\
*

Zypper may report multiple available versions for some packages. Please choose the newest
(bigger

version number) version of each package.



Convey System Administration Guide v1.6


16

4.3

Updating

Convey Software Packages using a GUI

After the Convey repositor
ies

ha
ve

been setup, the standard installation tools provided with a
Linux distribution, including graphical user interfaces to those tools (i
.e. YaST, aptitude
, Synaptics
Package Manager
), may be used to install, upgrade, and remove Convey supplied packages.
A
lmost a
ll Convey supplied package
s have names that start with


convey
-
„, and are provided in
both
.rpm

and
.deb

versions
in

correspondin
g
yum
and
d
ebian

repositories.
Customer proprietary
packages developed by Convey typically do not have “
convey
-
“ as part of the package name.

Note that the
d
ebian

package names have dashes („
-

) in their names wherever the
rpm

package
names have

underscor
es. This difference in package names is an artifact of the tool used to
automatically

convert

rpm

packages to
d
ebian

packages.


4.4

Install
ing

older versions of Convey Software Packages

Some of the

native Linux installation tools support installing older ve
rsions of a package.

First, remove any installed versions of the package

(except for
kernel

and AE personality image
packages)
. Then install the package
specifying the desired version
.

Installation
Tool

Sample Syntax

zypper

zypper install convey
-
1st=
0
.
9.0

yum

yum install convey
-
1st
-
0.9.0
-
0

apt
-
get

apt
-
get install convey
-
1st=
0
.
9.0

Note that the
convey
-
kernel

package and the personality packages with application engine images
(i.e.
convey
-
sp_ae
) may (and should) have multiple versions installed concu
rrently. Do not remove
any older (or newer) versions of these packages when installing a different version.

Note that
some

personality package
s

(
convey
-

_per
)
generally include

all of the old
er

versions of
that particular personality.


4.5

Version Numbers

an
d Package Filenames

Throughout the discussion of package installation, package version numbers

are depicted as
w.
x.y
-
z
, always as part of a package filename, i.e.
convey
-
1st
-
w.
x.y
-
z
.x86_64.rpm
.

The actual
packages on the we
bsite and installed on a system will
typically

have numeric version numbers,
such as
1.
1.1
-
1 or 1.
2
.1
-
2
.
w

is the major version number,

x

is
the minor version number
, and
y

is the
very minor
version number
.
z

is
the so
-
called release number, and in Conve
y packages is
typically an internal version number
, and may not be strictly numeric
.

T
he major version number

(
w
)

is changed whenever a major new release of a product occurs.
The minor version

number (
x
)

is changed whenever minor functionality changes or
some
significant bug fixes are being released. The 3
rd

version number (
y
) is changed when minor
changes are being released.

4.6

Manual Installation

These instructions assume either the
rpm

or
dpkg

command is available. On some distributions,
another instal
lation program may be appropriate.

You will need the Convey assigned username and password for your site in order to access the
packages on the
Convey support website. No password is needed to explore the Convey
Software DVD.

In the URLs below, “release”

may be replaced with one of the other release


Convey System Administration Guide v1.6


17

stream
names
currently on the Convey support website, if you have an appropriate password for
that release stream (i.e. beta).

All Convey created packages have a filename that starts with “
convey
-
“.

Review the

list of available packages in the following directories:


http://conveysupport.com/protected/install/hc1
-
1/convey_sdk/x86_64/RPMS/


http://

conveysupport
.com/protected/install/

hc1
-
1
/convey_sdk/updates/x86_64/RPMS/


htt
p://

conveysupport
.com/protected/install/

hc1
-
1
/convey_platform_sdk/x86_64/RPMS/


http://

conveysupport
.com/protected/install/

hc1
-
1
/convey_platfo
rm_sdk/updates/x86_64/RPMS/

and for Convey servers only, the following directories also:


http://

conveysupport
.com/protected/install/

hc1
-
1
/convey_platform/x
86_64/RPMS/


http://

conveysupport
.com/protected/install/

hc1
-
1
/convey_platform/updates/x86_64/RPMS/


http://

conveysupport
.com/protected/install/

hc1
-
1
/convey_centos/...


(in the convey_centos directory, please select the CentOS release # that matches your
system)

Note that there may be multiple versions of the e
ach package. Typically, you want to
download/install the newest version of each package (the one with the largest version number).

The Convey Software DVD provides a similar structure.


To

download and

install
one or more packages

from Convey‟s website
:

Logi
n
as root
(typically with the
su

command)



cd di
r

where

dir

is the directory you want to
store

the packages



Check which Convey packages are already installed, and remove any you want to
upgrade or replace

(except for
kern
el

and application engine fpga image packages)
:

o

On rpm based systems, use

rpm



qa


|


grep convey

rpm


-
erase


package
-
name
(s)

o

On Debian based systems, use

a
ptitude

search

convey
-
\
*

apt
-
get remove

package
-
name
(s)

If
aptitude

is not available, use the

graphical interface to the software installer
provided by your distribution to seach for packages with „convey‟ in their name



retrieve

the packages you want to install

o

wget


http://username:password@conveysupport.com/protected/...

replacing

„username‟ and „password‟ with the Convey provided username and
password for web site access
, or

o

copy the packages from the Convey Software DVD



Install
each

package

o

rpm

ihv convey
-

-
w.
x.y
-
z
.x86_64.rpm
, or


o

dpkg

i convey
-

-
w.
x.y
-
z_amd64.deb
(Debian
ba
sed distributions)

The above steps can be repeated at any time to update the installed Convey software with a
newer version of any package
.



Convey System Administration Guide v1.6


18

4.7

Additional
Installation Notes for Ubuntu
/Debian

and openSUSE

Some distributions, including Ubuntu 8.0.4, openSUSE 10
.3, and openSUSE 11.0, have a
problem linking 32 bit C/C++ applications with the newer version of
gcc

and
g
++. In particular,
when compiling the examples in the pdkSample directory that is part of the PDK personality, the
linker will complain that it can‟
t find the library
stdc++
.

If you receive this error message, you
need to downgrade to an older version of
gcc

and
g
++.

4.7.1

How to downgrade gcc and g++

First, see what versions of gcc and g++ are already installed:


ls /usr/bin/gcc* /usr/bin/g++*


If
version

3.4 of
gcc

and
g++

is not listed, please install the 3.4 version of those compilers from
your Linux distributions CD or vendor repository.

After

the 3.4 versions of
gcc

and
g++

are available, execute the following commands logged in as
root:


rm gcc g++


ln
-
s gcc
-
3.4 gcc


ln
-
s g++
-
3.4 g++


Any scripts or Makefiles that use „
gcc
‟ or „
g++
‟, without a version number, will now use the older
3.4 version of th
os
e products. The specific versions of
gcc

and
g++


(i.e.
gcc
-
4.1
) can still be used directly
, and the system can be restored to its original state by
linking
gcc

and
g++

to the newest install version of those products.

4.8

Installing Convey Software without an Internet Connection

If a
reliable internet connection is not available on your system, you may install the Convey
Software Products
several different ways:



Install Convey Software products using a Convey

provided

Update DVD

o

Email or call Convey support, and request a Convey
Update

DVD
,
and follow the
directions that come with the update DVD, or



Make your own DVD or USB installation media
:

o

On another computer that has internet access, browse to
http://conveysupport.com/protected/install/hc1
-
1
, a
nd
for each directory of interest
therein:

o

Descend into directorties of interest, until you find the
.rpm

or
.deb

packages,

o


right
-
click on each
package

whose name ends with
„.rpm‟

for Red Hat /
SUSE based distributions, or
„.deb‟

for Debian/Ubuntu base
d distributions,
and pick Save Target As (in Internet Explorer) or Save Link As (in
Firefox/Netscape) to save each file to a local disk.

o

You may also want to save the repository database files (
Packages.gz

for the
debian repository, and the
repodata

dire
ctory for the
yum

repository)

(you
must preserve the entire existing directory structure to create a usable
repository)
.

o

You may also use the
wget

command

(with the

r

option)

to download an
entire repository

directory structure
, and t
hen copy

it

to a USB
memory stick,
a writable CD/DVD, or some other portable media.



Convey System Administration Guide v1.6


19

o

It may be easier to use a copy of the repositories if you create a separate CD
or USB memory stick for each repository of interest, where the repodata or
Packages.gz file is in the root direct
ory of the CD or USB memory stick.


o

Do not copy the convey
-
platform repository for use on a non
-
Convey system.
The packages in this repository should only be installed on a Convey Hybrid
-
Core server.

o

If the CD or USB memory stick include
s

the repository d
atabase files
(
Packages.gz

and/or the
repodata

directory) then the software installation
tools provided with your Linux distribution can probably treat the CD or USB
memory stick as a repository

(possibly
after enabling the CD/USB media as a
repository
),

a
nd you may install the Convey packages like any other
package
, using
yum
,
zypper
, or
apt
-
get
.

You might need to create the
repository description files by hand, to point to the correct subdirectory on the
CD or USB memory stick.

o

On Ubuntu, a USB memory st
ick can be enabled as a Software Source by
adding a line such as


deb file:/media/UsbName

/

to /etc/apt/sources.list, where „UsbName‟ is the name the USB memory stick
was mounted as in /media.

The
Software Sources

tools in Ubuntu distributions can als
o add a CD as a
Software source.

o

If you cannot use the CD or USB memory stick as a Linux repository:

o

Copy the
packages (
.repo

or
.deb

files) from the
Convey Software CD
/DVD

(or other portable media c
ontaining the Convey packages)
onto your Linux
®

system (
/
tmp/rpms/
in the
example

below
).

Delete any Convey server
specific packages if you are installing on a non
-
Convey server, especially
any Linux kernel packages.

o

Login as root
(typically with the
su

command), and enter the following
commands

for Red Hat or
SUSE based distributions:


cd /tmp/rpms


rpm

ihv convey*


or these commands for Debian based distributions, such as Ubuntu:


cd /tmp/rpms


dkpg
--
install *.deb

o

The above steps should be repeated whenever a new release of any Convey
Software Pr
oduct is required, except the command should be


rpm

uhv package
-
file
-
name


or


dpkg

list | grep convey


# list
s

installed convey packages

by
name


dpkg
--
remove package
-
name



# not
the

package
-
file
-
name


dkpg
--
install package
-
file
-
name

o

E
ach
package
-
file
-
name

to be installed
must

be spelled out completely
,
including the
.rpm

or
.deb

suffix
, but a
package
-
name

does not include the
version number, the architecture, or the
.rpm/.deb

suffix



Convey System Administration Guide v1.6


20

5

Booting

and System Management

5.1

Overv
iew

This chapter describes the process of booting the system, initialization of the Convey
coprocessor, and management of the system.

5.2

Powering Up the Convey Server


When the server is first plugged in, the blue Chassis

Identity light will begin blinking. It will blink
for about one minute. Once it has stopped blinking, you can push the front panel button to power
up the Intel server. Do not power up the Intel server until the blue Chassis Identity light has
stopped blin
king.

5.3

First Power
-
Up and Boot

The Convey System comes pre
-
installed with Convey Linux, Convey software products, and
basic system configuration.

5.3.1

Intel Remote Management Module

If your system is configured with the
Intel Remote Management Module (RMM
), then this is the
recommended interface (console, keyboard, and mouse) for initial system power
-
on and
configuration. The MAC address for the RMM will be on a label on the outside of the server.

The Inte
l website, intel.com, provides manuals and information about the RMM and configuration
utilities. The psetup(1) utility that can be downloaded from the Intel website is useful in attaching
to the RMM and assisting in the initial configuration. The psetup
(1) utility identifies all RMMs on
the local subnet and presents them based on their MAC address.

A link to the psetup utility is available on the Convey Support website, conveysupport.com. Click
on the “Convey Hardware Help” tab on the top menu bar.

The
RMM is initially configured with DHCP and the default user/password is „admin/password‟.

5.3.2

First Boot

The system is configured such that on the first boot at your site, the system enters a
graphical
configuration menu. This allows initi
al configuration of several aspects of the system. Some of
the configuration tasks available are:



Change the password for the root user.



Add an initial user to the system (not root) and/or setup NIS.



Setup firewall rules.



Configure the network interface.



Configure date/time, time zone, and Network Time Protocol.



Install additional software packages via CDs.

If at some point later you would like to invoke this graphical configuration again, login as root and
execute the following commands:


chkconfig firstb
oot on


touch /etc/reconfigSys



Convey System Administration Guide v1.6


21


reboot

5.3.3

Manual Configuration

The following configuration updates are required at a minimum to place the system in general
operation and connectivity to your network if you are not using the
FirstBoot

reconfiguration
interface

just described in the previous section. Login as root and perform the following actions.



Select a password for super
-
user and update using the
passwd(1)

utility.



Select a host
-
name for the machine and update the
HOSTNAME

variable in the file
/etc/sysconf
ig/network
.



Select an IP
-
ADDRESS for the system and update the
IPADDR

variable in the file
/etc/sysconfig/network
-
scripts/ifcfg
-
eth0
.



Add a host
-
name and IP address entry to the file
/etc/hosts
.



If you are enabling a serial connection to the machine, add t
he following boot options to
the kernel command line in the file
/boot/grub/grub.conf
.

o

earlyprintk=serial,ttyS1,115200 console=ttyS1,115200



Reboot the system.

5.4

Boot Options

The Linux operating system is rich with boot options

to control

many aspects of system
configuration. The goal for the Convey system is to provide as seamless as possible system
initialization. Toward that end, a properly installed Convey system is designed with a set of
defaults that allow the system to be fully co
nfigured and functional without any additional boot
options. The system defaults are as follows:



The Convey coprocessor is fully enabled.



To the extent possible based on the installed physical memory on the Host server, all
memory on the coprocessor is co
nfigured and accessible.



The memory interleave for the coprocessor defaults to 31
-
31 interleaving.



The page size for the coprocessor is set to the maximum available based on board
configuration.



A 2 node NUMA configuration is established for memory domains

by the kernel. Node 0
is the Host server and Node 1 is the Convey coprocessor.



Address extension mode is configured as fully
-
mapped.

The Convey boot options follow similar syntax to most standard Linux options. All Convey boot
options are preceded by th
e keyword
convey

followed by a comma separated list (no spaces) of
keyword
-
value pairs. Note, keywords do not have to have an associated value. All keywords and
values are lowercase. For example:


convey=keyword1=value1,keyword2,keyword3=value3

Multiple


convey‟

boot option tags are also permitted:


convey=keyword1=value1 convey=keyword2 convey=keyword3=value3

The following Convey boot options are provided to allow system administrators detailed control
over the coprocessor configuration.



Convey System Administration Guide v1.6


22

5.4.1

enable/disabl
e

The
enable/disable

keywords control the discovery and initialization of the Convey
coprocessor. By default, the coprocessor is enabled. The
disable

option results in the
coprocessor not being initialized or available by the system. The memory on the c
oprocessor
board is not configured. All other Convey boot options are ignored.

5.4.2

caemode

The
caemode

keyword controls the behavior of the Convey Address Extension mode.
There are
currently 2 options available: fully
-
mapped (default) and dynamic
-
mapped. Th
e caeinfo(1)
command provides detailed information on the address extension configuration. Examples are:


convey=caemode=fully
-
mapped


convey=caemode=dynamic
-
mapped

5.4.3

memory

The
memory

keyword allows the administrator to restrict the amount of memory

config
ured

on the
coprocessor. By def
ault, the system will configure

all coprocessor memory up to limits that may
be imposed by the physical memory configuration of the Host server. The option takes a single
value that is the amount of memory to configure for
the coprocessor. This can be expressed as a
decimal value, hex value, or shorthand memory
-
suffix notation (k (kilobytes) = 1024; m
(megabytes) = 1048576; g (gigabytes) = 1073741824). Examples are:


convey=memory=0x200000000


convey=memory=2g


convey=memo
ry=1024m


convey=memory=0

The last example is a valid technique to configure the coprocessor with no local memory.

5.4.4

window

The
window

keyword allows the administrator to set the amount of Host memory allocated to the
memory window. The window size is const
rained to a minimum of 4G and a maximum equal to
the size of coprocessor memory. The window size must allow a minimum of 4G of free memory
to be available
on the Host when the configuration is complete. Currently, the window size must
be a multiple of 1G

bytes.
The window size can be expressed as a decimal value, hex value, or
memory
-
suffix notation. Examples are:


convey=window=0x200000000


convey=window=8g


convey=window=8192m

5.4.5

pagesize

The
pagesize

option allows the administrator to override the archi
tectural maximum page size of
the coprocessor. This value will be ignored if it exceeds the architectural maximum page size for
the installed board. Once the system is booted, the configured page size can be queried using
the
cnyinfo(1)

command. The pag
e size value can be expressed as a decimal value, hex
value, or memory
-
suffix notation. Examples are:


convey=pagesize=256k


convey=pagesize=1m


convey=pagesize=4096



Convey System Administration Guide v1.6


23

5.4.6

interleave

The
interleave

option allows the administrator to configure specific interleav
e schemes for the
coprocessor. The coprocessor interleave can only be set at boot time and is configured during
discovery and initialization. There are currently 2 types of interleave supported:
binary

and
3131
. Examples are:


convey=interleave=binary


convey=interleave=3131

The coprocessor interleave can be queried using
cnyinfo(1).

5.5

Coprocessor Modules

The initialization and configuration of the Convey coprocessor as a seamless part of the system is
accomplished by two loadable Linux system modules. Th
e
CNYSYS

module and the
CNYMP

module
are dynamically loaded at boot time and perform all discovery and initialization functions. The
command
lsmod(1)

displays all currently loaded modules in the system including the Convey
modules. The following sections

describe in detail the function of these modules.

5.5.1

CNYMP Module

The first Convey module to load during boot is
CNYMP
. This module provides out
-
of
-
band
communication with the management processor (MP) on the Convey coprocessor board. This
communication is

provided over a dedicated PCIE link with the MP. If the
CNYMP

module fails to
load and initialize, the system will boot but the coprocessor will be inaccessible. The
CNYMP

module provides the following general features:



Discovery and initialization of t
he MP PCIE link



General I/O communication path with the MP



Supports remote login for field personnel



Transport path for dynamic personality loads and coprocessor firmware



Error handling and logging between the Host server and the MP

5.5.2

CNYSYS Module

The
CNYSY
S

module loads immediately following the load of the
CNYMP

module. The
CNYSYS

module depends on the
CNYMP

module for communication with the coprocessor management
processor (MP). If the
CNYMP

module fails to initialize then the
CNYSYS

module will also fa
il. Failure
of the
CNYSYS

module to initialize correctly results in the system booting without coprocessor
access. The main functions of this module are:



Discovery and initialization of the Convey coprocessor



Setting the coprocessor memory interleave



Con
figuration of coprocessor memory into the system



Configuration of the coprocessor interrupt path



Management of the personality cache on the MP



Handling coprocessor faults and traps



Coprocessor allocation policies (attach/detach)



Debug control (GDB save/res
tore context)



Convey System Administration Guide v1.6


24

As just mentioned, one of the key functions of
CNYSYS

is configuration of coprocessor memory into
system. Coprocessor memory is configured into Linux and is accessible by all the standard
memory management policies. For example, x
-
86 applic
ations requiring large amounts of
memory will spill over into coprocessor memory based on the NUMA system policies configured
and not anything unique to the Convey system.
The memory on the coprocessor board will be
part of the memory statistics in
/proc/
meminfo
.

From an application perspective, the
CNYSYS

module provides the allocation polices for acquiring
access to the coprocessor. The
Convey Programmer’s Guide

provides details of these policies
and programming models. But it is important to note tha
t the Convey coprocessor is modeled as
an extension to the x
-
86 instruction set and not a separate stand
-
alone processor. For that
reason, the coprocessor will not show up in the file
/proc/cpuinfo

as an independent processor.

5.6

Kernel Configuration

This
section describes the basic kernel configuration

for the Convey Linux operating system. The
Convey Linux kernel was d
eveloped based on the Centos 5.4 kernel source base. Centos5.4

is a
2.6.18 released Linux kernel with extens
ions and fixes.

The overriding goal for the Convey Linux development was maintaining complete binary
compatibility. This extends to middle
-
ware, system call interface, tools, drivers, and loadable
modules. There are no changes to any system libraries or
header files.

The kernel configuration has been tailored for the Intel Shoffner platform and its devices. There
are several configuration selections made specifically for the Convey platform and its extensions.
Some of the key kernel configuration option
s are:



Kernel crashdump support



64
-
bit Intel Processor (32
-
bit kernel is not supported)



No forced preemption (server platforms)



NUMA (multi
-
node Non
-
Uniform
-
Memory
-
Access; 8 node maximum)



Sparse Memory design



Maximum CPUs (8)



Timer Frequency (1000
)



Kernel
Profiling

The Convey Linux kernel has been fully qualified with the configuration described. Changes to
this configuration, addition of drivers and/or modules, or specification of complex boot options
may result in conflicts within the system and untested

code paths.

5.7

Convey System Services

A service, as defined under Linux, is a shell script in the
/etc/init.d

directory. The Convey
platform supports a new service:
convey
. The
convey

service supports the full set of service
commands (start, stop, status, restart, reload). In general, the
convey

service is responsible for
the following events:



Setting of Convey specific
sysctl(1)

variables



Setting of the
/proc/sys/kernel/printk

level



Load and initialization of the
CNYMP

modul
e



Load and initialization of the
CNYSYS

module



Convey System Administration Guide v1.6


25



Configuration of the management processor

The utility,
chkconfig(1),

provides an easy to use interface to query the status of a service and
the run levels it is enabled for. For example, the following command
,


chkconfig



list convey

will display the following line indicating the run levels that will execute this service.


convey

0:off 1:off 2:on 3:on 4:on 5:on 6:off

This indicates that the
convey

service is enabled for run levels 2, 3, 4, a
nd 5. See the man page
for
chkconfig(1)

for a description of managing these run levels.

The
service(8)

utility provides an interface to execute service commands (stop, start, etc.)
against a specific service. The most frequent use of this command is to q
uery the loaded state of
the Convey modules.


service convey status


5.7.1

Convey Configuration Files

This section describes the Convey specific configuration files associated with the services just
described. The Convey configuration files

are maintained in the directory
/etc/convey

on Convey
installed systems. Each configuration file has a Convey installed version and may have a local
site
-
specific version. The local site
-
specific configuration files can override or extend Conv
ey
configuration options.

The current configuration file set is:

modules.conf

Defines the device names, permissions, owner, and group for the CNYSYS
and CNYMP modules.

modules_local.conf

Local site definitions that override the installed modules.conf.

mp
.conf

Configuration parameters for the Convey management processor. This
includes hostname, Ethernet settings, console settings, and logging
configuration.

mp_local.conf

Local site definitions for management processor. This will typically include
the IP
/Mask for the management processor Ethernet.

sysctl.conf

The format of this file is identical to
/etc/sysctl.conf

and contains Convey
specific
sysctl

variable settings.

sysctl_local.conf

Local site
sysctl

variable settings.

yum.conf

This file is referen
ced from
/etc/yum.conf

and contains Convey specific
yum(8)

configurations. No local version of this file is defined.



5.8

Coprocessor Sanity Test Program

The program
/opt/convey/sbin/cnydiag.aebase

exercises basic functionality of the Convey
coprocessor, i
ncluding memory access, coprocessor scalar instructions, internal datapaths in the
coprocessor, and
an

application engine vector instruction. The coprocessor must be unattached,


Convey System Administration Guide v1.6


26

and booted with “binary”

memory interleave. If the server is booted with 31
-
31 memory interleave
enabled, the vector test will fail, but the earlier tests should all pass.

5.9

When to Shutdown the Convey Server

If the coprocessor is attached by a process but appears to be hung (the application won‟t
abort/exit), or w
hen the Convey co
processor is not attached (see the man page for
cnyps
)

and
you cannot run the sanity test program successfully, please follow these steps:



Log onto the Convey server as root, and run

/sbin/shutdown


r


1


rebooting to reset the coprocessor




After
the
ho
st finishes
re
booting, try running the sanity test program again. If it still fails,
please contact
Convey Customer Service
.

5.10

Activating a Coprocessor Recipe

Installing a recipe package does not cause the coproc
essor to use that recipe.

A Convey server can have many recipes installed, but only one is “active” at a time. To activate a
particular recipe, Convey provides two tools:



/opt/convey/sbin/pickRecipe

o

This script

lists the available (installed) recipes, and

lets the user select one to be
activated.



/opt/convey/sbin/activerecipe
recipe
-
filename

where
recipe
-
filename

is
one of the “
recipe...
” files in /opt/convey/fpgaimages.

Important Note:

After activating a recipe, you must shutdown the host system, wait
for for two or
more minutes, and then power up the host.




Convey System Administration Guide v1.6


27

6

Convey System Memory Management

6.1

O
verview

This chapter provides high level details about the Linux operating system physical and virtual
memory management for the Con
vey Coprocessor. The goal behind these changes is to provide
high memory bandwidth and performance for the coprocessor. There are four concepts that form
the basis of Convey memory system that will be discussed.



NUMA Memory organization



Memory Interleav
ing



Variable Page Size Support



Inter
-
Node Memory Migration

The
Convey Reference Manual

will be re
ferred to frequently for general description of the
system organization and the hardware architecture.

6.1.1

Intended Audience

This chapter is intended for system administrators and application designers. The goal is to
provide sufficient details about Convey m
emory management to enable development of high
performance applications. Administrators need to understand how the operating system
manages the Convey memory architecture and presents this to applications.

6.1.2

Terminology

The following terms are used to descr
ibe various components of the Convey system as it relates
to memory management.

Host memory

This refers to memory physically installed on the Intel x
-
86 Server.

Coprocessor
memory

This refers to memory physically installed on the Convey coprocessor.

Me
mory window

This refers to the physical memory on the Host server that is reserved/used
to enable coherent access to memory on the c
oprocessor.

6.2

NUMA Memory Configuration

Physical memory is populated on the commodity Intel

motherboard and on the Convey
Coprocessor board thus forming a classic Non
-
Uniform
-
Memory
-
Architecture (NUMA). The
degradation in memory latency and bandwidth when accessing remote memory (non
-
local to the
processor) is significant and must be considered

in any coprocessor application.

The Convey Linux kernel manages the physical memory of the system by using kernel nodes to
separately configure the physical memory associated with the Intel motherboard and the Convey
coprocessor. By convention, the cop
rocessor node will be the highest numbered node in the
system. For most system configurations, node 0 will be the Intel board and node 1 will be the
coprocessor board. This also implies the Linux kernel is configured with NUMA enabled and a
maximum node c
ount of 8.

All available Linux techniques and tools are supported for managing a multi
-
node NUMA
configuration. The
/sysfs

file system describes each node and its attributes. For example, the
file
/sys/devices/system/node/node1/meminfo

provide all the
memory statistics for the


Convey System Administration Guide v1.6


28

coprocessor node. The file
numastat

in the same directory provides statistics on page allocations
with respect to the NUMA configuration. The
numastat(1)

utility is available from the
numactl

package. This outputs a nicely format
ted display of the
numastat

information for all nodes in the
system. Finally, the file
/proc/meminfo

provides memory statistics for the entire system (both
nodes).

6.2.1

Physical Memory Configuration

The previous section discussed the memory configuration from
a logical Linux perspective. The
physical implementation is different due to the architecture of the Convey coprocessor. Physical
memory installed on the coprocessor board is actually implemented as a cache of memory on the
Intel server board. This desi
gn is an artifact of the Intel Front Side Bus (FSB) and providing a
single cache
-
coherent address space for the system. The implications of this design are as
follows:



Because coprocessor memory is a cache, a portion of host memory (less than or equal to
the size of memory on the coprocessor board) is configured as coprocessor memory.



The Linux kernel will automatically reserve and configure host memory equal to the size
of coprocessor memory if available

(default)
. No boot options are required.



The Linux

kernel requires a minimum of 4 gigabytes of physical memory for the host
server. The remainder of physical memory may be configured as coprocessor memory

(memory window)

based on coprocessor memory size.



If in
sufficient host memory is configured
,
the
mem
ory

boot option is used,

or the
window

boot option is specified,

the host memory configured for the coprocessor can be less than
the physical memory size installed on the coprocessor (even zero).

6.3

Address Extension Feature

The initial release of the Conve
y system only supported 16G of coprocessor memory and
required that 16G of Host memory be reserved for access to the coprocessor (one to one
mapping; i.e.
fully
-
mapped
). The Shoffner X
-
86 motherboard supports a maximum of 128G of
p
hysical memory (16 x 8G
DIMMS) that would have limited the maximum amount of memory
supported on the coprocessor given the previous architectural rules.

The Convey Address Extension feature was developed to enable support for maximum memory
configurations of the Host Intel server

and the Convey coprocessor (128G Host memory; 128G
coprocessor memory) without requiring the one to one mapping of Host memory to coprocessor
memory. A region of Host memory

(known as the memory window)

is still reserved to enable
accesses to
coprocessor

memory but it is not
required to be a one to one mapping. This new
configuration capability is known as
dynamic
-
mapped
.

The architecture now supports a
hardware
mapping table between the Host memory window and
coprocessor memory that enables mapping a re
gion of the memory window onto any region of
coprocessor memory. Memory window regions can be remapped dynamically as the X
-
86
references different regions of coprocessor memory.

The remapping process is managed by the
operating system in a manner simila
r to virtual memory page faults.
The
caeinfo(1)

utility
provides detailed information on the hardware and software management of the memory window.

The kernel boot option,
caemode
, allows the system administrator to select between the 2
modes:
fully
-
mappe
d
,
dynamic
-
mapped
. The following sections describe each in detail.

6.3.1

Fully
-
mapped mode

Fully
-
mapped mode is the system default and provides a full mapping of the Host memory
window onto coprocessor memory. In this mode, each byte of coprocessor memory must

be
mapped by memory on the Host. The hardware mapping table is initialized at boot time to a one


Convey System Administration Guide v1.6


29

to one mapping of the Host memory window onto coprocessor memory.

The size of the memory
window is configured by the operating system at boot time based on
the following constraints:



Boot option
convey=memory=###

sets an upper limit on the size of coprocessor
memory configured.



Boot option
convey=window=###

sets an upper limit on the size of the Host memory
window.



A minimum of 4G of Host memory is reserved f
or the system.



A successful configuration must result in 3
-
4G of Host memory being
free and allocatable
for the kernel and applications.

The following example demonstrates the configuration rules for fully
-
mapped mode. A system
with 64G of Host memory and

128G of coprocessor memory
without any boot options
will
configure a memory window of approximately 58G and coprocessor memory of 58G.
This results
in 70G of coprocessor memory not being configured.
T
he

6G of Host physical memory
remains
for
the operati
ng system, system data structures, and 3
-
4G of free memory

(this assumes 2
-
3G
consumed by the operating system)
.

6.3.2

Dynamic
-
mapped mode

Dynamic
-
mapped mode must be specified via the boot option
caemode
. The size of the memory
window is configured at boot tim
e based on the following constraints:



Boot option
convey=window=###

sets an upper limit on the size of the Host memory
window.



The Host memory window defaults to a size equal to 25% of the size of
configured
coprocessor memory.



A minimum of 4G of Host memo
ry is reserved for the system.



A successful configuration must result in 3
-
4G of Host memory being
free and allocatable
for the kernel and applications
.

Using the example from the prior section, a system with 64G of Host memory and 128G of
coprocessor memo
ry would configure a memory window of size 32G. The full 128G of
coprocessor memory would be configured into the system and available.

32G of Host memory
remain for the operating system, system data structures, and applications.

Determining the size to c
onfigure the Host memory window is completely application specific. If
the application has limited X
-
86 access of its data in coprocessor memory, then smaller windows
should provide good performance.
As the application increases its X
-
86 accesses or the
accesses are very random/distributed, memory window region swapping will occur. The size of
the memory window will affect the amount of swapping in this case.

6.4

Memory Interleaving

This section describes the supported memory int
erleaving schemes on the Convey coprocessor.
The coprocessor supports two types of interleaving; binary and 31
-
31 as described in XXX. The
coprocessor memory interleave is set at boot time and currently defaults to 31
-
31 interleave if no
boot option is sp
ecified. See the chapter
5.4.6

interleave

for a detailed description of interleave
boot options.

The interleave schemes are independent of the type of DIMMs installed in the coprocesso
r
(standard DIMMs or scatter
-
gather DIMMs). The interleave scheme affects which memory
controllers, DIMMs, and banks are accessed based on memory address. The performance
obtained by the interleave scheme is a function of the memory access pattern of the

application
and is not discussed in this section.



Convey System Administration Guide v1.6


30

The type of interleave configured is determined at boot time and cannot be reconfigured after the
coprocessor has been brought online. The kernel performs a coprocessor discovery and
configuration process

during boot. The coprocessor attributes are logged into the file
/var/log/messages

that includes the configured interleave type. The configured interleave can
also be determined using the
cnyinfo(1)

utility or displaying the file
/sys/devices/system/nod
e/node1/interleave/summary

(note, this assumes the coprocessor is
node 1).

6.4.1

Binary Interleave

The binary interleave scheme is designed to provide near peak performance for unity stride
applications. All available memory will be co
nfigured; there are no interleave holes in the physical
address space.


6.4.2

31
-
31 Interleave

The 31
-
31 interleave scheme was designed to meet the following system requirements:



Provide as high a bandwidth as possible for all memory acc
ess strides, with particular
focus on power of two strides.



Keep each memory line (64 bytes) on a single memory controller. This is required to
simplify the cache coherency protocol.



Maintain the interleave pattern across virtual memory page crossings. T
his helps large
strides where only a few accesses are to each page.



All virtual addresses must map to unique physical addresses.

The scheme uses a two level hierarchical interleave approach. The Convey memory system
using Scatter/Gather DIMMs has 1024 mem
ory banks. The 1024 banks are divided into 32
groups of 32 banks each. The first interleave level selects one of 31 groups of banks. The
second interleave level selects one of 31 banks within a group. Note that one of the groups of the
32 groups of ban
ks is not used. Similarly, one bank within each group of banks is not used. The
prime number (31) of banks and groups of banks is used to maximize the sustainable memory
bandwidth for as many different strides as possible. This skipping of banks results

in a 6% loss of
available memory and a decrease in peak memory bandwidth of 6%. A coprocessor with 16
Gbytes of physical memory and 31
-
31 interleave configured, will have an available physical
memory of 15.04 Gbytes (a loss of 960 Mbytes).

6.4.3

31
-
31 Interle
ave Page Allocation

The 31
-
31 scheme is a 2 level interleave. The first level has a required mapping function and the
second level is a best effort. If a physical page of the required mapping cannot be found on the
coprocessor for a specific virtual addr
ess, then one of several actions may take place:



If the memory policy for the virtual memory region allows, the memory allocator will
attempt to allocate a page out of host memory (x86 Intel server board).



If host memory is also not available or the memory

policy does not permit off node
allocation, then the process is pended and the paging daemon is woken to free additional
memory on the node.



Note, host memory does not have the mapping requirement for 31
-
31 interleave. The
mapping function only comes int
o play for allocation of coprocessor memory.

The
cny3131(1)

utility provides detailed information about allocation patterns, color utilization,
page order usage, etc. The
„p
-
miss‟

column when displaying primary color statistics indicates
that a page of th
e correct mapping was not available. This would trigger one of the above


Convey System Administration Guide v1.6


31

actions. Allocation of a host memory page or starting of the paging daemon will result in
performance degradation for the coprocessor application.

6.4.4

31
-
31 Interleave Faults

The Linux
system call interface and general programming model gives any application almost
complete control of its virtual address space. For example, the
mmap(2)

system call permits the
application to specify the exact virtual address at which to map a new object.

This feature is used
heavily by the dynamic library loader but rarely by other applications. Use of this capability can
result in invalid physical
-
to
-
virtual mappings for a coprocessor application with 31
-
31 configured.
For this reason, it is strongly
advised not to specify the mapping virtual address to
mmap(2)
.

The Linux kernel makes an extreme best effort to avoid and prevent invalid 31
-
31 mappings. But
given the virtual address space control an application has, there is not a 100% guarantee. For

this reason, the coprocessor has a detection algorithm in the TLB walker that will detect an invalid
page mapping and generate a
SIGBUS

to the application. Detailed information is dumped to the
system console and system logs. This information should be
captured and sent to Convey
Customer Support.

6.5

Variable Page Size

Support

This section discusses the Linux operating system support for Convey variable page size support.
The goals for providing variable page size support are:



Re
duce the number of TLB misses and page faults by the coprocessor



Increase the utilization of the coprocessor TLB cache



Allow the coprocessor TLB to cover all physical memory

The
Convey Reference Manual (Chapter 4)

describes the architectural details behind the
coprocessor variable page size support. The system suppo
rts page sizes 4K to 4M and every
power of 2 page size in between. The kernel aggressively allocates the largest page possible
during the following events:



Page faults (these can be copy
-
on
-
write)



File system reads



Reads from the swap area



Data migration

The allocation of contiguous memory (large pages) is applied only to processes that are currently
attached to the coprocessor. Host X
-
86 processes cannot benefit from the large page allocation
feature of the coprocessor.

The allocation of large pages is
applied only to Convey attached processes and is independent of
the node the memory is allocated from. Because the coprocessor can access Host and
Coprocessor memory, large pages in either memory domain improves TLB performance. Note,
that due to the mem
ory activity in the Host domain, the ability to allocate the maximum large page
is reduced.

The
/proc

pseudo
-
filesystem implements several files that provide detailed information about a
process. From a memory management point of view, the files
maps
,
num
a_maps
, and
smaps

contain
data on the virtual address space of the process. The data is organized around each memory
mapped region with the address space. There are man pages for
proc
(5)

and
numa_maps
(5)

that
cover the format of this information. Becaus
e of the importance of page placement and page size
on application performance, a new
/proc

file has been added for Convey systems. The file
pmaps

contains detailed data on physical memory placement and page size for the entire virtual address


Convey System Administration Guide v1.6


32

space. This

is extremely useful when tuning coprocessor applications. See the man page
pmaps(5)

for more detail.

6.5.1

Maximum Coprocessor Page Size

The current architecture of the coprocessor is designed to support a maximum page size of 4M.
The architecture data of the

coprocessor is acquired by the kernel at discovery time during boot.
The maximum page size can also be controlled at boot time (boot option) or dynamically at run
-
time. The boot option, pagesize, defines the maximum page size to be used for the coproces
sor
up to the architectural limit. Also, the maximum page size of the coprocessor can be queried and
set by the file
/sys/devices/system/node/node1/pagesize

(assuming node1 is the coprocessor).
The following command allows setting of the maximum page siz
e to 16k bytes.


echo 16k > /sys/devices/system/node/node1/pagesize

This change

takes effect immediately on the next memory allocation for an attached coprocessor
application. This can be useful when diagnosing system problems or performance anomalies.

6.5.2

Li
nux HugeTLB Feature

The HugeTLB feature in Linux enables use of X86 2Mbyte pages by applications and is fully
supported by the Convey Linux operating system. The coprocessor can access HugeTLB pages
mapped into an application. However, because this featu
re is more restrictive than the
coprocessor page size support and due to the complexity of maintaining 31
-
31 interleave
mappings, coprocessor memory cannot be reserved into the HugeTLB memory pool. In addition,
the HugeTLB feature does not currently have
an API to specify the node from which to reserve
memory for the pool. HugeTLB memory will always be allocated out of Host memory.

Applications being ported to use the Convey coprocessor that depend on the HugeTLB feature
will execute correctly. The issue

is one of performance. If the application port results in the
coprocessor heavily accessing virtual memory that maps HugeTLB pages, changes will need to
be made to allocate coprocessor memory or to be able to take advantage of page migration. This
is co
vered in detail in the
Convey Programmers Guide
.

6.6

Memory Migration

This
section describes memory migration within the Convey system architecture. The Convey
system is configured as two NUMA nodes typically numbered 0 and 1. Memory migration allows
the moving of the physical location of pages between nodes while the process i
s running. The
virtual addresses the process sees do not change but the physical location of pages mapped to
those virtual addresses do. The goal of memory migration is to reduce the latency of memory
access by moving pages near to the processor accessin
g the memory. This is critical toward
achieving maximum performance of the application.

It is important to note that page migration is a best effort by the kernel. All references to a page
must be removable at the time of migrate or the page is dropped f
rom the migration list.
Examples of this are pages that are locked for I/O, shared pages that still have references from
other processes, or dirty pages in the process of being cleaned. Other factors include lack of
physical memory on the target node of
the migration.

The Convey Programmers‟ Reference Guide describes the library API available for data
management. The routine
cny_migrate_data()

provides a user interface for migrating virtual
regions within a process to/from the coprocessor. This is bui
lt on top of standard Linux system
calls and is provided as an easy to use interface that is cognizant of the Convey coprocessor
system architecture. The routine
cny_memory_locale()

assists in identifying the location of a
specified virtual region within
the application. The information provided by
cny_memory_locale()

may not be completely accurate if the memory has not been referenced following a migration.

When the routine cny_migrate_data() is applied to a memory region that is unpopulated (empty),
the

pages are not faulted in by this call. Instead, the memory policy for the region is updated with


Convey System Administration Guide v1.6


33

the target node of the migration. Later, the pages will be faulted into memory of the target node
thus avoiding a memory to memory copy. However, the cny_m
emory_locale() will still return
empty until the pages are faulted into memory.



Convey System Administration Guide v1.6


34

7

Personality Management

7.1

Personality Signatures

The Convey Coprocessor supports multiple personalities
, and multiple versions of each
personality. Although

the user can exercise substantial control over which particular personality a
given program will use, several tools are available for a system/site manager to guide users
towards particular personalities and versions thereof. Note that there are multiple

personality

upgrading

steps, only two of which are described here. A full discussion of
personality upgrading

is available in the
Convey Programmers Guide
.


A complete signature

identifies a particular version of a coprocessor personality. Signatures are
composed of
five

numeric values, such as „2.1.
1.
1
.0
‟. The
five

fields (in order) are:



personality
number



major version
number



minor version
number



hardware model
number



canonical instruction set version number


The personality number indicates the type of instructions implemented in a particular pers
onality,
such as single precision vector, double precision vector, etc.

The major and minor version numbers indicate various updates to the personality, including bug
fixes, performance improvements, and new instructions.

The hardware model number indicate
s which version of the Convey coprocessor this signature
supports. Signatures that are identical except for the hardware model numbers are compatible
,
and an object file compiled for a particular signature can be executed using a compatible
signature
.


Th
e
fifth numeric value
,

the
canonical instruction set version number
, is

reserved f
or future use
.

If
specified as part of the signature, it must be zero.

7.2

Default Base Signature

After installing the
convey
-
per
-
tools

package for the first time (or anytime th
ereafter), the system
administrator may update the file


/opt/convey/personalities/DefaultBasePersonality


This file contains a single
default base signature

for the system. This
default base signature

is
used to fill in
some
missing parts of a user specified signature when compiling or assembling
source code. If the user specifies a zero (or does not specify a value) for the
personality number

or the
hardware model number,

the corresponding value from the
default base si
gnature

will be
used.

The major and minor version numbers in the default base signature are not used for
anything.


This „upgrading‟ of the user specified signature allows the system administrator to:



select which
personality
number

to use when the user d
oes not specify one, such as
„single_precision‟,



Convey System Administration Guide v1.6


35



select which
hardware model

number

to use when the user does not specify one,
typically the
architecture version
number

of the Convey coprocessors installed on the
Convey server this program will be run on,

and



If the
personality
number

in the
default
base
signature

is zero or an invalid personality
number, the user must specify a valid
personality
number

when compiling or assembling
source code.

Convey recommends that the default value for the
hardware mode
l

number

be set to the value for
the Convey coprocessor the program will be run on.
Although signatures with different hardware
model numbers are compatibile, the compilers use the hardware model number to optimize code
for better performance.

Users of
the Convey compilers and assembler will typically specify a
personality
number
, and
possibly a
major

and
minor

version
numbers
, but will not usually specify a
hardware model

number
.

At the time this users guide was written, the
default signature

installed
the first time the convey
-
1
st

package is installed was 2.
1
.
1.1.0
, which specifies the single precision personality number,
and a
hardware model

number of
0

(the first version of the Convey coprocessor released. Future
installations of Convey packages will

not overwrite this file, even if the
convey
-
per
-
tools

package
is uninstalled.

7.3

System Default Signatures

After installing the
convey
-
per
-
tools

package for the first time (or anytime thereafter), the system
administrator may update the file


/opt/convey/
personalities/SystemDefaults


This file contains zero or more signatures. The system administrator can modify this file to
establish
default signatures

for each personality. These personality specific
default signatures
are use
d to „upgrade‟ a user supplied partial signature. When the user specifies a partial
signature, that signature is upgraded (if possible) to match an actual installed personality.
Typically, the major and/or minor version numbers are upgraded. See the up
grading discussion
in the Convey Programming Guide for more information.


For example, if the SystemDefaults file contains
2.2.3.1
.0
,
and the user specified “sp” as a
partial signature when compiling a routine (“sp” maps to “2” in the Convey Personality Da
tabase),
then the partial signature will be upgraded to 2.2.3.x, where the value (x) for the
hardware model

number is obtained from the system‟s
default base signature

value (see above).

7.4

Convey Personality Management Script

After installing the convey
-
per
-
tools

package for the first time (or anytime thereafter),
the
/opt/convey/sbin/cpm

script may be used to manage personality signatures in the personality
database.

The
cpm

script allows the system administrator to disable and enable particular
personaliti
es. A disabled personality, although still installed on the system, cannot be used to
compile, assemble, or execute coprocessor code. The
cpm

script can also be used to list installed
signatures, and get additional info about a particular signature.







Convey System Administration Guide v1.6


36

Usage:


/opt/convey/sbin/cpm action [personality list]


a
ction

What it does

list

Lists the specified signatures, if installed.

$
cpm list 2.1.1.1.0

1.1.1.1.0

produces

Personality Compiler License Coproc Nickname(s)

Number

Support Status Support

---------------

-------

------------------

------

--------------------------

1.1.1.1.0 Yes Permanent No double dp double_precision


do
uble_vector dp_vector

2.1.1.1.0 Yes Permanent No single sp single_precision


single_vector sp_vector

If no signatures are specified, all signatures in the personality database
will be listed.

A “compiler support” value of “yes” means the compiler can compile code for that
signature. A “coproc support” value of “yes” means an executable requiring that
personality may be executed on this system. The “license status” field shows
i映瑨a琠
per獯na汩瑹tha猠s⁶ 汩d 汩捥n獥n⁴hi猠獹獴em.†A⁶ 汩d 汩捥nse⁦or a⁰er獯na汩瑹ti猠
required inrder⁴o⁥xeu瑥⁡⁰rogram⁵獩ng 瑨a琠per獯na汩瑹t

in景

jore⁶ rbo獥 瑨an

„list‟, the „readme‟ file for the specified signatures is also included.


able

qhe⁳ ecified⁳igna瑵re猠sre⁥nabled.

di獡b汥

qhe⁳ ecified⁳igna瑵re猠sre⁤i獡b汥l.

A⁤i獡ble⁳igna瑵re⁣ nno琠be⁵sed⁢礠yhe
捯mpiler⸠




Convey System Administration Guide v1.6


37

8

Recovery DVDs

Convey provides

a

recovery DVD

with each Convey Server shipped (1 set of

DVDs per cluster or
head node).

Additional DVDs can be ordered through Convey‟s Support group.

Th
is recovery DVD

allow
s

the system to be restored
after a catastrophic disk or system failure
,
with a
modest

amount of
user interaction.

WARNING: The
System

R
ecovery DVD

will completely erase the systems main hard
drive, and all data on that drive will be lost.

Note:
the recovery process for non
-
head nodes (Convey server nodes WITHOUT a DVD drive) is
described later, near the end of this chapter.

8.1

How to use
the
Convey System Recovery DVD

The Convey
System
Recovery DVD installs
CentOS plus Convey platform extensions and
enhancements.

In general
,

recovery / installation from the DVD involves the following steps:



System preparation
including:

o

Attaching a graphical
(PC compatible)
display to the VGA adapter
, and attaching
a
USB keyboard
/
mouse
, or

o

Logging into the previously configured Intel RMM console for the host server



Booting off of the DVD.



Reviewing and possibly modifying the
installation parameters.



Allowing the configured installation to install.



Removing the DVD and booting off of the installed disk.



Conf
i
guring the system via CentOS‟s post install configuration menus.



After the system is completely installed and running, th
e second DVD may be used to
install various additional Convey products, including compilers, personalities, …

8.1.1

Preparation

Convey recommends

that
Convey OS
installs be performed using the CentOS GUI installation
program. The GUI installation program offers
the full functionality. To use the graphical
installation program, either:



attach a
PC compatible
graphical display, USB keboard, and USB mouse to appropriate
ports located on the back panel of the Convey sever, or



Log into the previously configured Intel
RMM host server console from a web browser

8.1.2

Boot from DVD

Install the
Convey System Recovery DVD

into the servers DVD drive. If necessary, configure
the system BIOS to boot from the DVD and allow the system to boot. A successful DVD boot will
result in a C
entOS splash sceen being displayed on the console with the following text menu
options:


-

To install or upgrade in graphical mode, press the <ENTER>

key.



Convey System Administration Guide v1.6


38


-

To install or upgrade in text mode, type: linux text <ENTER>



-

Use the function keys listed bel
ow for more information.


[F1
-
Main] [F2
-
Options] [F3
-
General] [F4
-
Kernel] [F5
-
Rescue]



To abort the installation process at this time, reboot your machine and then eject the boot media
DVD.

To continue with the installation, press
enter

and allow the sy
stem installer to load.

8.1.3

Configure the Installation and Install
ing the OS

After loading the Anaconda CentOS system installer, the CentOS welcome menu will be
displayed; proceed through the following menus to configure the installation. Please note that
netw
ork, firewall, system name, and a few other items will be configured after the installation,
during the first hard disk boot:

1.

Click
Next

to enter the disk partitioning menu.

2.

Partition the disk as to your local requirements; a recommended partitioning is as

follows:

a.

2G /boot partition

b.

64G Swap partition

c.

The remainder of the disk allocated to the
“/


partition.

3.

Configure the time zone

4.

Create a
root

password

5.

Add additional packages; all recommended and required Convey base platform RPMs
are already selected

6.

Cl
ick
next

to arrive at the final installation menu; you may abort the installation at this
point by rebooting your machine and then ejecting the boot media.

7.

Click
next

and wait for the installation to complete.

When the installation is complete the DVD will

be ejected (remove the DVD from the drive) and
the system will reboot. Upon a successful disk reboot the system will enter the CentOS “first boot”
configuration GUI; proceed through these menus to configure the Convey Server according to
your local requir
ements.

8.1.4

System Configuration

Additional

system configuration

occur
s

during the first, post install, boot. A GUI steps through the
following menu options; complete details of the menu options can be found at
http:/
/www.centos.org/docs/5/
.

This is

a summary of the configuration menus:

1.

Cent OS welcome menu:

Click
Forward

2.

Keyboard configuration menu:

Select the appropriate keyboard type.

Click
Forward

when complete.

3.

Root Password menu:

Create a root password.

Click
Forward

when complete.



Convey System Administration Guide v1.6


39

4.

Network configuration menu:

The default configuration is DHCP. Click
Change Network Configuration

to
configure static IP addresses; a pop
-
up menu will be displayed that provides IP, DNS,
/etc/hosts, and other configuration panels. No
te if you click outside of the network
configuration popup then the window manage will push the network pop up to the
background; use the
Alt+Tab

keyboard short cut to bring the network configuration
menu to the foreground. Click on the popup
X

button at t
he upper right to exit the
network configuration.

Click
Forward

when complete.

5.

Firewall configuration menu:

The firewall is disabled by default; configure as appropriate for your installation.

Click
Forward

when complete.

6.

SELinux configuration menu:

SELinu
x is disabled by default; configure as appropriate for your installation.

Click
Forward

when complete.

NOTE
: if you enable SELinux, please see the SELinux FAQ at
www.conveysupport.com/protected/faqs

for more information about configuring SELinux
(required in order for the Convey compilers to operate correctly).

7.

Kdump configuration menu:

kdump is enabled by default; Convey recommends that it remain configured with 128MB
of reserved memory.

Click
Forwa
rd

when complete.

8.

Timezone configuration menu:

Select the appropriate timezone.

Click
Forward

when complete.

9.

Date and Time configuration menu:

Verify the current time and date; adjust as necessary. Use the NTP configuratioin tab to
enable and configure NTP
.

Click
Forward

when complete.

10.

Create User configuration menu:

Define additional local users. Select the
Use Network Login

button to configure NIS.

Click
Forward

when complete.

11.

Sound Card configuration menu:

Click
Forward

to skip.

12.

Additional CDs menu:

Clic
k
Finish

to complete the basic Convey / Centos server platform installation.

13.

After several seconds, the CentOS graphical login menu is displayed.

8.2

Fixing the Convey SW Repository Files

You need to add the Convey provided

username

and
password

(for web acces
s)
to
every

Convey
SW Repository file in
/etc/yum.repos.d
.

Edit every file in /etc/yum.repos.d, whose name begins with “convey
-
“, and insert your company‟s
Convey web access username and password into the URL entry. For example, if your
site‟
s
username /
password


was


abc / xyz
, then a URL line that initially looked like


baseurl=http://conveysupport.com/protected/install/hc1
-
1/convey_sdk/x86_64/RPMS/

would look like this after editing:



Convey System Administration Guide v1.6


40


baseurl=http://abc:xyz@conveysupport.com/protected/install/hc1
-
1/convey_sdk/x86_64/RPMS/

8.3

Adding
Convey

Provided
License Keys

After restoring the Convey OS, various optional products need to be installed, some of which
require a Convey provided license.

You may
r
equest a copy of the license
(s)

from the Convey support
group (email
support@conveycomputer.com

from an email address associated with the organization that
purchased the Convey server please).

8.4

Installing the Remaining Convey Software Products

See section
Installing
Convey Software on Cross
-
Development Systems
. This section
applies to Convey servers as well as cross
-
development systems.

The Convey
host system

should now be
fully configured
and
operational.

8.5

Checking the Management Processor

After following all the above steps, run the command:
/opt/convey/sbin/mpinfo

If the Management Processor (MP) is communicating with the host system, something similar to
the following will be
appear:

# /
opt/convey/sbin/mpinfo

Hostname :
your hostname

Active recipe : recipe.beta
-
0.9.3
-
09_07_08

Next recipe : none

Active firmware : firmware
-
0.0
-
0.0.1124
-
1

Committed firmware : firmware
-
0.0
-
0.0.1124
-
1

Trial firmware : n
one

U
-
Boot : 1.3.0
-
rc3 (May 13 2009
-

16:03:38)

Configured eth0 : static 192.168.22.182/24:

Current eth0 : 192.168.22.182/24:192.168.22.5 up

Console baud : 115200

Xinetd : on

Clock Frequency : 150.000 M
Hz


If the management processor appears to not be communicating with the host system, the
following steps should correct the problem:



Do
a yum update convey
-
\
*

(as root) to update all of Convey‟s packages to the latest
available (and known to be consistent
) versions. Alternatively, explicitly
install

the specific
versions of the convey
-
kernel (or convey
-
kernel_beta), the convey
-
mp_fw, and the
convey
-
uboot packages that you were using before the system was restored.


If convey
-
mp_fw or convey
-
uboot was one
of the packages updated by the above
command, you must also shutdown the host as shown below



Shutdown the host (
/sbin/shutdown
-
h now bye
), wait at least two minutes, and
power up the host via the RMM or the host systems power button.



Try the
mpinfo

com
mand again. If the MP is still not communicating with the host
system, please contact Convey support (
support@conveycomputer.com
).



Convey System Administration Guide v1.6


41

8.6

Updating the Server to the latest SW Versions

If you didn‟t do a yum update
as described in the preceeding sub
-
section, you may wish to
upgrade all the installed Convey SW to the latest released versions. The command


yum update convey
-
\
*

(logged in as
root
)

will

upgrade all installed Convey SW packages, and may require

a shutdown/reboot.

8.7

Restoring a Convey Server without a DVD drive

Contact the Convey support group (
support@conveycomputer.com
) for detailed instructions.



Convey System Administration Guide v1.6


42

9

Customer Support

Procedures

Please use email
or the toll free phone number for simple questions. The

issue tracker web
interface

is ideal for r
eport
ing

bugs and request
ing

enhancements to Convey products.

Email

support@conveycomputer.co
m

:
Convenient

for
quick simple questions.

Issue Tracker Web Interface

This is the preferred contact method when additional
information needs to be passed on to Convey, such as
source files,
screen dumps, etc.


If you already have a

Convey Issue T
racker
username
and password, browse to
rt.conveysupport.com
.


If you need an issue tracker username and password,
browse to
conveysupport.com/issuetracker.html

an
d
follow the directions there.

Call Convey‟s toll
-
晲eeumber
N
-
㠶U
-
㌳P
-
ㄷ㘸


Select the “Customer Support”

ex瑥n獩潮

“2”

when⁴hi猠
do捵men琠wa猠sri瑴enF


䥦⁹ u⁡re⁴ran獦srred⁴o
癯i捥ma楬
Ⱐplea獥 a癥⁡
me獳sge
wi瑨⁹ ur nameⰠ
your company‟s name,
祯ur
phone‣Ⱐand⁡ brie映
de獣siption o映瑨e problem.

䥦⁹ u⁰re晥r⁴o⁲ecei癥 an
e
-
mail⁲e獰on獥I⁰lea獥 lea癥⁹ ur⁥
-
ma楬⁡ddre獳s

qhi猠sumber⁷i汬⁵獵a汬礠be⁡n獷ered‹am


Rpm
䍥C瑲al⁔ime
啓AFⰠi映a⁣u獴smer⁳ r癩c

repre獥n瑡ti癥 i猠svai污lle.



捵獴smer⁳ r癩捥⁲epre獥n瑡ti癥⁷楬l⁴祰i捡汬礠y捫cowledge a汬 emailⰠ癯i捥ma楬Ⱐandew⁩獳 e猠
reported⁶ia⁴he i獳se⁴ra捫cr⁷i瑨楮iO4⁨our猬sjonday
-
crida礠yex捥p琠ho汩da祳F⸠

cor⁥x瑲emel礠yrgen琠i獳se猬sp汥l獥⁳ nd⁡n email⁴o
support@conveycomputer.com

and call
the toll
-
free #.



Convey System Administration Guide v1.6


43

10

Licensing

10.1

Convey Licensed Software

Beginning with the April 2010 release, all of the Convey products may be installed on a Convey
Server or customer provided c
ross
-
development system, but a license is required to use some of
the features and capabilities provided by those packages.

For a Convey server, the required licenses are typically installed before the server is delivered.

For a cross
-
development system
, and for an add
-
on order for new software for a Convey server,
the necessary license
(s)

are typically e
-
mailed to the designated contact, along with instructi
ons
for installing those licenses
.

The following are examples of what features and capabilities a
re licensed:



Using a Convey provided vector personality when executing
on the coprocessor



Use of the Convey coprocessor simulator



Use of the Convey compiler suite

10.2

Old Licensing Scheme

Prior to April 2010, t
he
packages containing the
Convey personality tool
s
,
the
FPGA personality
images,
the personality development kit,
some of software development utilities,
and the Convey
Math Libraries
were

licensed
. Those older packages

can
only be installed if a license

key

for th
at
package

is already
present

in the fi
le
/opt/convey/licenses
.

Usually, Convey Customer Support
would have

email
ed the license keys
to each customer

s designated contact person. Send e
-
mail
to
support@conveycomputer.com

for help with license ke
ys.


The is no need to remove the
/opt/convey/licenses

file when installing products that use the
new licensing mechanism described above. You may wish to install older versions of some
Convey packages someday, and the
/opt/convey/licenses
file may be required to install those
packages.





Convey System Administration Guide v1.6


44

Appendix A

-

Installing YUM on
older
SUSE™
Distributions


Start up
yast
, then



in the
Software

part, pick
Software Management
, then




in the
search bar
, type
yum

(and click search), then



select the package na
med
yum
, then




click on
accept

(you may have to
accept

other packages due to dependencies)


On Suse distributions that support
zypper

(openSUSE 10.3 and later), please use
zypper
, not
yum
.






Convey System Administration Guide v1.6


45

Index

.

31
-
31 inte
rleave

29

apt
-
get

12

binary interleave

29

boot options

20

booting

19

configuration files

24

Convey software distribution CD

17

Convey System Services

23

customer support

41

default base signature

33

default signatures

34

distributions, Linux

10

First Boot

19

installation, manual

16

Installing Convey Software

7

installing convey software without an internet
connection

17

Intel Remote Management Module

19

kernel configuration

23

licensing

42

linux distributions

10

memory interleaving

28

memory migration

31

NU
MA Memory Configuration

26

package version numbers

15

personalities

33

personality upgrading

33

Powering Up the Convey Server

19

Recovery DVDs

36

repository description file

7

RMM

19

SDK/Optional Products DVD

39

signature

33

software repository description

files

7

system memory management

26

System Recovery DVD

36

updating Convey software packages

14

variable page size

30

wget

16

yum

12

zypper

12