return [d_ServiceCounter *]<1.0 - CineSoft

musicincurableData Management

Jan 31, 2013 (4 years and 9 months ago)

121 views

Duma

Server side


Secretary



central

core of the
system, dispatcher for all jobs and
interaction manager w/ users.
You
don't need to keep running
dispatchers on each user’s
workstation
, “Fire and forget” :)


Comrade



remote launching
service

Client side


Deputat



main user GUI for management
all jobs and users


Televisor



tool for getting rendered
images over Internet


Scripts



ready to use way schedule render
from native application interface:
Renderman (MTOR/PRS), MR standalone
(MayaToMR), MayaBatch MR, MayaBatch
Software, Nuke, Shake, DF, After Effect
and others


DumaStart


tool for fast and easy creation
custom Alfred Scripts.


Rrun



console client for manual launching
command on remote Comrades. Useful in
testing purposes.

Compatibility with Alfred®


Duma has
full

Alfred’s
®
functionality, so you shouldn't change
your render
routine
. And of course we
offer

many new
abilities and fresh, user friendly GUI :)


Alfred Scripts for render job describing
through
full size TCL

pre
-
processor


100 % compatibility with all MTOR/PRS generated Alfred Scripts


Support distributed and deferred Mtor (or Houdini) Rib
generation with dynamic creation of subtasks branches



Alfred® and RenderMan® are registered trademarks of Pixar Animation Studios

Stability and speed


About 2 year's of VFX production on our humble ~100 CPUs
render farm


Wide hardware’s platform support Linux, Mac, Win


Duma central module


Secretary, takes advantages of modern
multi core CPU for it’s scalability. Also Secretary uses
proprietary database for interactive work w/ thousands
Comrad's, many Deputat's, Gigs of output logs


Tested for manage large render farms.

Tests was conducted on render farm which counted about 1500
virtual hosts

Users and jobs


In Deputat you may select several render
queues to merge theirs jobs for
collectively
manage (see video demo)


For jobs execution Duma use some kind of
Alfred
®’
s job spillover scheme


Every Queue/Job has
accumulating

Priority
property. Queues/Jobs with high priority can
consume more CPU’s than other


Maximum running task limit may be applied.

Separate limit values for queue and job may
be set


Inside queue Job Blocking Fence may be set.

Jobs beyond fence will not ran while all jobs
before become completed.

E.g. Job 1


textures gen, Job 2


render
which uses this textures


Security


Users


can manage only their private queue and
any public queues. Also any people can overview
any other queues/jobs/tasks


Supervisors


can do what they decide to do. E.g.
start/stop and even delete any jobs, change any
parameters. Also that’s people can boost Priority
factors in values greater than
1
(one)


Any activities on private queue from any person
except queue owner (i.e. supervisor) are logged on
Secretary side. So any people can investigate what
happened with their render tasks


For easy login, (w/o
credentials
) on office sites
trusted IP networks can be specified. User which
connecting from other than trusted sites must use
name/password to login


User accounts can be pooled from domain or local
computer database


Output and system Log's


Live delivery task’s stdout/err. Instant
browsing
regardless of

total output’s size


Out log for each run session store separated


Auxiliary log detail command execution
history (see it in video presentation)


Useful std out application/options:


Adjustable RegExp for live task
completion’s percent calculation


Output filters (w/ counters) for sweep
out hordes of duplicated warning/errors


Syntax highlighting (errors, warning and
etc.) may be set for particular task’s
classes


Working from anywhere


Lightweight network protocol to Deputat allow perfect
working over Internet. Most likely you will not feel any
deference while being working from home


Deputat sends requests describing desired to view
area, while Secretary accumulated that incremental
requests to know Deputat’s interests. When something
in desired area has changed, Secretary send
corresponding data bit to reflect changes


Secure login
-

user connecting from Internet must
provide name/password credential to login


Televisor


delivering pictures from your local net to
your home. By your request, on server side, images
will converted to jpeg with desired size and quality,
and then immediately sent to you directly through TCP
(see video demo)

Farm Stats


Runing task
-

list containing all running task on farm.

Multi selection to group
management of running task is available.


Hosts
-

list w/ attached to Duma computers. Live host CPU’s loading, free memory and
other stats available from here. Also, missing hosts highlighted to red colour, while hosts
excluded from render schedule marked with cross stroke


Selection in one of that’s tables lead to auto
-
selection corresponding rows in another. E.g.
selecting running task you also going to select host on which that task executing

Schedule reliable render in few clicks!


Send render job to Duma directly from your favourite application using one
of our ports


Auto testing render output files. Analysis performed by easy to customize
script. At this moment output file’s modify time and size similarity are
checked.

3ds Max

(
M
axScript)

Maya MR

(Mel)

Maya MTOR

(Mel)

Duma Start

(Standalone)

Nuke

(TCL)

AE

(Java Script)

Shake

(standalone

TCL)

DF


(DfScript)

Schedule file for Secretary


Single XML file describing various
aspects of rendering policy may be
modified and then automatically
reloaded by Secretary on the fly


“Rules”


much more flexible than
classical “slot” system. Using TCL you
may write own strategy how use
render frame.

E.g. “slot system” may be described by
single instruction

return [d_ServiceCounter *]<1.0


Job/Computers Variables


may be
used in Rule Expressions and easy
modified from Deputat GUI.


E.g. you may declare variable
“memory_demand”, then you should
reflect secretary’s rule expression by
adding compare that variable and
host’s memory.

After that/ you may easy sift out low
memory hosts for particular job


Statistic


After render task has completed, info
describing that render event send to
external SQL database (e.g. PostgreSQL).
In particular that info contain:


Project/Scene/Shot


Start Time + duration


Owner name, task type, render host,
exit code and many others…


You may collect render statistic from that
SQL database and format it into nice report
using any suitable software


Custom report solution may be developed
for you. Basically we provide

source codes
(c++) of SQL logging utility which can be
modified to reflect your specific demands

Administration

Secretary and Comrade:


Simple error statistic provided internally (
w/o
any SQL
), give easy way to discover buggy and
unstable render host


XML config files give very
fine control over
Secretary amd
Comrade


All auxiliary and trace info collected in batch of log
files. You may setup size limit and time of live
that's logs

Deputat config's:


User Config file. Store all GUI element settings, as
well as many other parameters and special config
file for new user


globals override for all site


Auxiliary configs purposed to define set of
viewer’s tools and setup highlighting of output log

Materials

and contacts

Video demo (on
-
line)

Video demo (zip 118mb)

Power Point

Open Office

Duma configs samples


Ivan Ivanchik
ivan@cinemateka.ru

Konstantin Kharitonov
cinesoft@cinemateka.ru




© 2008,Cinesoft