Amazon EC2 and S3

viraginitysplashInternet and Web Development

Nov 10, 2013 (3 years and 5 months ago)

144 views

Amazon EC2 and S3



Presented by Denard Springle @ CFMeetup Dec 22
nd
, 2011


Sr. Software Developer @
CF
Webtools


Over 20 years IT & development experience


Java, Perl, PHP, C/C++


Hardware, network and storage engineering


CMMI process management & assessment


Over 10 years ColdFusion experience (v4.5)


Rich internet and mobile applications


Host of the Northern Virginia CFUG



denard.springle@gmail.com


http://www.nvcfug.org/


@
ddspringle

What is the ‘Cloud’?


Typically defined as a suite of services working
together to provide virtualization, scalability and
variable performance to hosted operating systems and
applications.


Allows full control of your hosting environment from
operating system to web server to database to network.


Allows for private networking, load balancing, instant
-
on provisioning, instant
-
on backups and more!


‘Cloud’ is often misrepresented by companies as a
name for their virtualization service.


Virtualization != Cloud.

How to get Amazon Web Services


Go to
http://aws.amazon.com

and click ‘Sign
-
Up Now’


Follow the processes to enter your information, email, etc.
asked for by Amazon


Provide payment information (you will not be billed until
you use the service)


AWS offers a Free Tier of service which provides one full
year of hours to run an EC2 micro Amazon Linux instance,
S3 storage and more for *new* clients only. If you already
have an AWS account, you do not qualify.


An EC2 micro Amazon Linux instance and ~10G S3 storage
costs about $15/mo total as of Dec 2011.

Amazon Elastic Cloud Computing (EC2)



Amazon’s computing platform, called EC2, provides nearly
instant on Linux and Windows cloud servers of many
different flavors and sizes


Highly secure environment with 256
-
bit AES encryption,
helps meet PCI compliance for managing financial or
sensitive information


Easily scale up or down, add or remove servers and load
balancers, to handle higher traffic (e.g. Black Friday)


EC2 servers run in their own memory space


unlike many
other cloud providers who share memory between cloud
servers.

Creating your first instance.


Launch a new Amazon EC2 instance


Specify the operating system you want to use (Amazon
Linux 32
-
bit in this case)


Specify the parameters of the cloud server to use


Specify the private keys you want to use, or create
them


Specify the firewall settings you want to use, or create
them

(NOTE: You must open up the SSH port on Linux or the RDP port on Windows
or you will not be able to access the server! You will also want to open
HTTP/HTTPS ports.)


Confirm your settings and launch the instance

Demo this now



Accessing EC2 from Windows


Download Putty.exe and PuttyGen.exe from the
Putty
Download Page


Download the security key you generated on Amazon Web
Services


KEEP A COPY OF THIS KEY IN A SAFE PLACE!


Use PuttyGen.exe to convert the key to putty format

(NOTE: the putty format also works with the
FileZilla

FTP client)


Launch Putty.exe and specify the key to use for the session
or use a .bat file consisting of:

X:
\
path
\
to
\
putty
-
i

mykey.ppk ec2
-
user@myhostname|myip


Copy and paste the connection information from Amazon
Web Services
(e.g.
ec2
-
user@myhostname|myip)


Login to the Amazon Linux EC2 server with Putty

Demo this now


Accessing EC2 from OSX/Linux


Open a terminal



Pass SSH the
keyfile

you downloaded from AWS:



>
ssh


i

mykey.pem ec2
-
user@myhostname|myip



Enjoy.



NOTE: SCP also uses this same convention, for copying files to the
server under OSX/Linux.


>
scp


i

mykey.pem ec2
-
user@myhostname|myip …

Sudo



Super User Do


AWS Linux (all variants) are configured to allow the
ec2
-
user
sudo

access to everything except
passwd

root


There is no ‘root’ user to log into. This is a safety
measure that mitigates hacking and
rootkit

attempts


Any time you want to make a change to the file system
(including copying, moving, downloading, installing
or removing files/applications, etc.) you will need to
use
sudo


The only exception is within the ec2
-
user’s home
directory, where you can perform any basic file system
operations within that directory.

Yummy
Yummy

Yum


Yum (
Yellowdog

Updater Modified) is the package
manager associated with numerous Linux
distro’s
,
including
CentOS
, upon which Amazon Linux is
based.


>
sudo

yum install <package to install>
[ENTER]
-

installs packages from the repository


>
sudo

yum update
[ENTER]


updates all installed
packages with available updates


> yum list <
wilcard

search string>
[ENTER]
-

lists
installed and available packages


Sudo

not required to list packages (no system changes)

First Steps with Amazon Linux


Amazon Linux comes as a fairly plain (locked down) server
deployment, so, the first thing to do is ensure you’ve got
the latest Apache build installed.
Sudo

is required.



>
sudo

yum install
httpd

[ENTER]


>
sudo

yum install
httpd
-
devel

[ENTER]


>
sudo

yum install
libstdc
++.so.5
[ENTER]


>
sudo

chkconfig

httpd

on
[ENTER]



httpd
-
devel

and
libstdc
++.so.5 are required by the ColdFusion installer


libstdc
++.so.5 should always locate to the most recent
libstdc
++ available, if
not, use: > yum find
libstdc
* to locate the correct package to install

Demo this now


Next steps


acquire ColdFusion


Download ColdFusion 9 and ColdFusion 9.0.1 update:


NVCFUG S3 Hosted 32
-
Bit Linux ColdFusion 9.0.1
Quick Start Package
(**For deployment in US
-
EAST Zone only**)


Use
wget

(or similar utility) to download the
gzip’d

and
tar’d

quick start archive:

>
wget

https://s3.amazonaws.com/nvcfug/dev/AWS.EC2.i386.cf901.QuickStart.tar.gz


Unpack the archive with tar:

>
tar
xvzf

AWS.EC2.i386.cf901.QuickStart.tar.gz
[ENTER]


>
cd

cf901QuickStart
[ENTER]

Demo this now


Install ColdFusion 9


>
sudo

ColdFusion_9_WWE_Linux.bin
[ENTER]


Hit [ENTER] a bunch to get through the license


Enter ‘y’ to agree to the license


Choose install type


developer, licensed or 30
-
day trial


Enter serial number, if applicable (licensed)


Choose ‘Server Configuration’ (other methods are
supported, but we won’t cover those in this demo)


Specify if there is already a CF9 server installed (no)


Select or deselect components as needed


Start demo now


Install ColdFusion 9, cont…


Choose the install folder. /opt/coldfusion9 is the
default


Specify if earlier versions of CF are installed (no)


Add Web Server Configuration (option 1)


Choose Apache (option 1)


Specify location of the
config

file: /etc/
httpd
/conf


Specify the location of the binary: /
usr
/
sbin
/
httpd


Specify the binary control file location:
/etc/
init.d
/
httpd

Install ColdFusion 9, cont…


Continue with installation (option 4)


Specify CF9 admin location (default /
var
/www/html)


Specify runtime user (nobody)


Skip
OpenOffice

configuration (unless needed)


Enter and confirm the admin password


Enable RDS and specify the RDS password


Confirm installation


Grab a beverage and wait, and wait, and wait


Once complete, DO NOT start the server… yet!

Configuring ColdFusion & Apache


CF *does not* properly update the apache_connector.sh
file that get’s executed the first time ColdFusion starts
when installing on Linux


Replace the CF created shell file with the one downloaded
in the cf901QuickStart


>
sudo

cp

f apache_connector.sh
/opt/coldfusion9/bin/connectors/
[ENTER]



Else,
sudo

vi
/opt/coldfusion9/bin/connectors/apache_connector.sh
[ENTER]
and manually change the paths for the conf, binary
and init files.

Starting ColdFusion


And start ColdFusion


>
sudo

/opt/coldfusion9/bin/
coldfusion

start
[ENTER]


Tail the cfserver.log file to watch CF start


> tail
-
f /opt/coldfusion9/logs/cfserver.log
[ENTER]


Hit CTRL
-
C to stop tailing the file when it’s done
loading


Open the CF Administrator for the first time in a
browser

Update to CF901


Stop Apache and ColdFusion


>
sudo

/etc/
init.d
/
httpd

stop
[ENTER]


>
sudo

/opt/coldfusion9/bin/
coldfusion

stop
[ENTER]


>
sudo

ColdFusion_update_901_WWEJ_linux.bin
[ENTER]


Runs exactly like the CF9 install, except all values will
already be filled in. *Do not add another connector for
Apache.*



Grab a beverage and wait, and wait, and wait



Restart Apache & CF


Restart Apache and ColdFusion


>
sudo

/etc/
init.d
/
httpd

start
[ENTER]


>
sudo

/opt/coldfusion9/bin/
coldfusion

start
[ENTER]


Tail the cfserver.log file to watch CF start


> tail
-
f /opt/coldfusion9/logs/cfserver.log
[ENTER]


Hit CTRL
-
C to stop tailing the file when ColdFusion is
done loading

Configure CF to start on boot



coldfusion
’ file is an init script, you only have to copy it
to the init directory and configure the system to start
it:



>
sudo

cp /opt/coldfusion9/bin/
coldfusion

/etc/
init.d
/
[ENTER]


>
sudo

chkconfig

--
add
coldfusion

[
ENTER]




That’s it. Your done. Grab a celebratory beverage!



Install
MySQL

(optional)


>
sudo

yum install
mysql

[ENTER]


>
sudo

yum install
mysql
-
server
[ENTER]


>
sudo

/etc/
init.d
/
mysqld

start
[ENTER]



Follow the instructions for adding a root user to
MySQL

shown on the console when
MySQL

starts the
first time


Use
mysqladmin

to manage the database(s) or install
PHP &
PHPMyAdmin

for a web based interface


Connect created database(s) through the ColdFusion
administrator as normal



Simple Storage Service (S3)


CF9.0.1 uses Apache VFS to facilitate access to Amazon
S3 buckets


You must use a script style Application.cfc to properly
instantiate the S3 settings (caveat #1)


DirectoryExists
() always returns false when trying to
access Amazon S3 buckets (caveat #2)


DirectoryCreate
() always works, even if there is already
a bucket in existence (caveat #3)


Directory and File functions include the bucket path
in the filename (caveat #4)

Application.cfc with S3 in CF

cfscript

style CFC works

CFML style CFC fails


component {

this.name="AmazonS3Demo";

//s3 info

this.s3.accessKeyid = "<
accessKey
>";

this.s3.awsSecretKey =
"<
secretKey
>";

}




<
cfcomponent
>

<
cfscript
>

this.name = "AmazonS3Demo";

<!
---

s3 info
---
>

this.s3.accessKeyId = "<
accessKey
>";

this.s3.awsSecretKey =
"<
secretKey
>";

</
cfscript
>

</
cfcomponent
>


Alternative method of directly accessing S3 within code w/o initializing it in the
Application.cfc:


DirectoryCreate
(“S3://<accessKey>:<secretKey>@bucket/path/”)

Using S3 within ColdFusion


All file and directory tags and functions in CF work
with Amazon S3


s3://
is the only difference between using these tags
and functions to call a file on Amazon S3 and calling a
local file (e.g. D:
\
, /
var
/www/)


Filenames will contain the full path to the file
excluding the bucket. (e.g. instead of ‘myfile.jpg’ you
will get ‘path/to/myfile.jpg’ as the name column and
‘s3://bucket/’ as the directory column).


As an aside, use
ram://
in CF9.0.1 to use the local RAM
drive with Apache VFS (configure VFS in CF Admin)

Demo this now



Additional Resources


Jeff Roberson’s CF & EC2 blog post



Willem
Redelijkheid's

blog post on installing CF on
CentOS


CFMeetup

"Replace Your Iron with a Cloud", with Barney
Boisvert



Ray Camden’s Amazon S3 guide


Adobe’s S3
LiveDocs


NVCFUG S3 Demo Page


Download this presentation at:

css.dvdmenubacks.com


The End