an example Oppy – an example Oppy - mague . com

cabbagepatchtapeInternet και Εφαρμογές Web

5 Φεβ 2013 (πριν από 4 χρόνια και 6 μήνες)

145 εμφανίσεις

Getting to Push Button Deploys



Moovweb

January 19, 2012

Things Everyone Wants

Reality

… is it done yet??? … is it done yet??? … is it done yet???

Gotta Have It!

PROS



Increase velocity of change/responsiveness


Get Operations out of the loop


Buzzword compliance


aka Shiny!!!

PROS

Increase velocity of change/responsiveness

PROS


Ops Gets Out of Loop


Less work for Operations


Developers feel more in control

Gotta Have It!

CONS



Increase velocity of change


Shared responsibility


Not always easy

Make It So...

Components

Components

Components

Components

Components
-

MoovStack

CI


Jenkins ( maiden name is Hudson )


Checks out a deploy branch from git


Runs a few sanity checks


Packages software


Uploads to software repo


Tags build in git

Jenkins

http://je
nkins
-
ci.org/

Jenkins

Jenkins

Rake

Packaging


But it's sooooo hard








save yourself first.....

Using FPM

https://github.com/jordansissel/fpm

Building a Gem

Right Tool Packages


Something that does not change often


Don't need to roll back quickly


Don't need more than 1 version at a time


You cache external repos or limit to just yours


Right Tool Gems


Fast roll back


gem cache


Easy setup of gem server


Need more than 1 version at a time


Bundler for dependencies


Bundle exec for peace of mind

Configuration Management


Puppet everything from bare metal/base image


Seperate data from manifests with an ENC


Max troubleshooting time is 20 minutes


Amazon free ”chaos monkey” service

Puppet


Start with a sane base

ENC


Use Groups!

http://docs.puppetlabs.c
om/guides/installing_das
hboard.html

ENC


Node Entries Inherit

ENC


Scripting Additions

ENC
-

Output

Manifests Contain ENC Variables

Templates Contain ENC Variables

Puppet
-

Tips


Execs hurt


try to avoid them


Recursive directories


use vcsrepo instead


Write your own types


just restart in between


Run mongrel or passenger


Use behind a load balancer for fun


Use reporting, but take the DB off of your
master


Write your own facts and save lots of time

Puppet
-

Scaling

Mcollective


Do It NOW!!


Orchestration framework


Crazy Parallel


Write more ruby code...


Clients and Agents

Mcollective


Crazy Parallel

6380 nrpe checks across 370 hosts in 3.8 seconds

https://github.com/mague
c/devops/tree/master/mco
llective/nrpe

Use Facts and Classes to Filter

Mcollective


Custom Agent

Mcollective


Custom Client

Mcollective
-

Tips


Remeber you have a machine gun pointed at
your foot


Wrap command in IO.popen and log everything


Log to syslog


Use ApacheMQ


For EC2 tune your keep alives (registerinterval)


Remember you have a machine gun pointed at
your foot

Mcollective
-

Scaling

Sinatra


Web app in 5 minutes


Much more minimal than rails


Written in Ruby so I can interface with
Mcollective easily

http://www
.sinatrarb.
com/

Sinatra


Addons


sinatra
-
respond_to
-

useful as API


Pony


send email from Sinatra


Twitter Bootstrap


make things pretty

Public Example

https://github.com/ma
guec/mco_dashboard
_example

Public Example

Really Simple

Sinatra
-

Tips


Run a few behind Nginx


Use helpers whenever possible


Log every push


Use it to integrate all your data

Actual Push Procedure

Dashboard Functionality


Push software


Audit software


Real
-
time monitoring check


Performance data integration


Log data integration


Troubleshooting tools for developers


Utilities for non
-
technical staff


Open trouble tickets


Cost information (in progress)


Userful links to other data sources

Oppy


an example

Oppy


an example

Oppy


an example

Oppy


an example

Oppy


an example

Oppy


an example

Thanks


Michael Catlin (organizer)


Tarun Desikan (giving me free reign/enough rope)


R.I. Pienaar ( Mcollective author and helpful guy)


Andrew Farmer ( for helping to get this going )

Links


http://blog.moovweb.com
/


http://www.devco.net
/


http://www.puppetlabs.com/


http://blog.mague.com/


http://www.sinatrarb.com/


http://jenkins
-
ci.org/


IRC #puppet,#mcollective,#devops

Author

Chris Mague

christianmague@gmail.com

IRC: bosszaru usually on #gaijin

Twitter: @maguec