Python for database access and

gorgeousvassalΛογισμικό & κατασκευή λογ/κού

7 Νοε 2013 (πριν από 3 χρόνια και 7 μήνες)

52 εμφανίσεις

Python for database access and
Event driven programming in
Python

But first, this week …


This is Computer Science
Education Week


Date chosen to honor Grace
Murray Hopper, whose
birthday is December 9


Rear Admiral, U.S. Navy


Computer Science Pioneer


Credited with the term
“debugging”


Developed the first
compiler


Conceptualized the idea of
a machine independent
programming language


Explains
nano

seconds!

Database access in Python


A set of tutorials, including access to
MySQL

and
PostgreSQL

http://python.about.com/od/pythonanddatabases/Connecting_to_Databases_With_Python.htm



A brief introduction to SQLite3


Lightweight disk
-
based database


See http://docs.python.org/library/sqlite3.html

Example, using sqlite3


First, import sqlite3


Create an object that represents the database


Use standard
sql


This example from
Nakul

Rathod

import sqlite3

def main():


db = sqlite3.connect('test.db')


# connect to the database or create the database if it doesn't exist


db.row_factory

= sqlite3.Row


# the powerful Row Factory interface allows one to access information in the database like a dictionary


db.execute('drop

table if exists test')


db.execute('create

table test(t1 text, i1
int
)’) # the database object accepts standard
sql


db.execute('insert

into test (t1, i1) values(?,?)', ('one', 1))


# you can directly substitute the values in ?,? but variable names allow for code re
-
usability


db.execute('insert

into test (t1, i1) values(?,?)', ('two', 2))


db.execute('insert

into test (t1, i1) values(?,?)', ('three', 3))


db.commit
() # without a commit no changes to the database will be recorded


cursor =
db.execute('select

* from test order by i1')


for row in cursor:


print row['i1'], row['t1'] # Each record is fully indexed by the field heading due to Row Factory

main()

Spot check


Create a database using sqlite3


Each row in the database contains the
following information (Make about 5
entries)


ID (alphanumeric)


Last name (alpha)


First name (alpha)


Hire year (
int
)


Enter data in random order, then display all
in order of hire year

Event driven programming


This is another way to think about flow
control in a program.


An important consideration is parallelism
(or pseudo
-
parallelism)


If there is only one processor, and we write the
program to support more than one event at
once, then the one processor gives slices of
time to each event.


All modern computers have multi core
architecture


more than one processor
available. It is possible to have more than one
event process happening at the same time.

So far


Most of our programs are basically
sequential, with excursions into
functions and classes for special
purposes


There is a clearly defined sequence of
actions that the program will perform.



Event Driven Programs


In event
-
driven programs, the sequence of
actions is not known in advance


The program has the facility to respond to
external events and take appropriate
action.


Now, programming means


determining what events might happen


recognizing each event


taking appropriate action for the event


waiting for the next


sometimes accommodating simultaneous events

Error events


An event may be a special case of normal
programs


such as an error event


We have seen how to handle errors


This is a form of interrupt handling

number = 0

while

not 1 <= number <= 10:


try
:


number=
int
(
raw_input
('Enter

number from 1
to 10: '))



if
not

1 <= number <= 10:



print

'Your number must be from 1 to 10:'


except

ValueError
:



print

'That is not a valid integer.'

Servers, GUIs


Some programs do nothing except
respond to events


Servers listen for a request and respond


GUI interfaces present a pallet to the
user


The operating system responds to the
choices made by the user

Wait loops and threads


One way to implement event handlers is
with an endless loop that checks for an
indication of the event, then responds


If there is only one process running, the
event takes control and no other event can
be serviced until that one is complete


Threads allow multiple flows of execution
simultaneously


Same processor


Multiple processors

Network programming


Book introduces event driven
programming in the context of
drawing


Fine, but let’s look at networks


A brief introduction to networks to
start


Peer


to


Peer


Client
-
Server

Client
-
Server Computing


Much of our computing environment is
client
-
server based


Server:


provides a service


waits for a request and does something to
satisfy the request


Client:


The controlling side of the interaction


Connects to a server


Issues a request


Server must be connected to a network


Server software waits for requests,
does some processing, sends
responses

Server waits for
requests

Locating a server


Web as our primary communication method
to servers


URL (or URI) names the server where a
resource is provided


Class web page is located at

http://www.csc.villanova.edu/~cassel/8000/Syllabus.html


http: the protocol


www.csc.villanova.edu
: the domain


rest: the path to the resource


Domain Name Server (DNS) translates from
the domain name to the IP address of the
machine.


Multi
-
level


edu
, villanova,
csc

Network connections


Each machine on the network has a
location, called an IP address


think of it like a phone number


Many possible connection requests


Within the machine, a
port
identifies the
specific communication line


think of it like an extension in a phone system


A client requests a connection on a particular
port to reach the service of interest


Now, messages flow between the client process and
the desired server process

Protocols


How does the client request the desired
service?


A protocol is a standard set of messages
to exchange between cooperating
entities to achieve a desired result


What protocols do you know?

Spot check


You know protocols


How do you introduce two people?


Suppose person A is senior to person B


Can you think of any other standard
message exchanges?


Anyone remember Lawrence
Welk





And a 1 and a 2 and a 3 …


Network protocols


There are many that we will not consider


For Client
-
server computing


Server listens on the port (end point of the
socket)


Client directs a message to the known port


Message content tells the server what service
is wanted


Server responds with a standard message, in
an agreed format


Each side understands only the pre
-
defined
messages!

Python Socket class

Syntax

Semantics

s.connect(host, port)

connects socket s to
remote host at specified
port.

s.send(data)

sends data to connected
remote host.

s.recv(maxNumChar
s)

receives a
maxNumChars from the
remote host.

s.close
( )

Disconnects socket
s

from remote host.

Simple client


Get time from the time server at NIST


No need to send anything because the
server only does one thing.


As soon as the connection is requested, the
desired service is identified

>>> from socket import socket

>>> connection = socket()

>>> connection.connect(('time.nist.gov',13))

>>> print connection.recv(1024)


55902 11
-
12
-
07 16:41:46 00 0 0 380.0 UTC(NIST) *

Process the returned data


We know the exact format of the data,
because that is part of the protocol


We can now process that, format it, do
anything we like with it.

>>> from socket import socket

>>> connection = socket()

>>> server = '
time.nist.gov
'

>>> connection.connect((server,13))

>>> fields= connection.recv(1024).split()

>>> date = fields[1]

>>>

>>> time = fields[2]

>>> print 'Date (YY
-
MM
-
DD) is %
s
, time is %
s

(
UTC)'%(date,time
)

Date (YY
-
MM
-
DD) is 11
-
12
-
07, time is 16:46:50 (UTC)

>>>

Spot check


Connect to the NIST time server


Print out the current date and time in
this format:

It is now <time> on <date>



How would you make the date be in the
form


December <day>, 2011


(or, if you prefer) <day> December 2011

Terry Scott University of Northern Colorado
2007 Prentice Hall

24

recv Method


There are four possibilities for the
outcome of a recv command

1.
Receive string of length equal to maximum
length specified. May have further data that
can be retrieved by another call to recv.

2.
Receive less data than maximum. That is the
end of that batch of data.

3.
No data was sent so recv waits indefinitely.

4.
Empty string is received. Connection was
closed by sender or network was disrupted.

Client side


Nothing particularly event driven on the
client side


This is where the event originates


The event driven software is on the
server side

Writing a server


event driven
programming


A server is a basic event
-
driven program


Server


listen on a particular port for incoming
connections


create a socket to manage each connection


follow the chosen protocol for communication
with the client


From the server side


send means from server to client


recv

means from client to server


A bit about network protocols


References to TCP


Transmission control protocol


Connection
-
oriented protocol for
establishing and maintaining a connection
between communicating entities on the
Internet


handles things like corrupted transmissions,
dropped calls, etc.

TCPServer

class


Hides all the details of establishing a
reliable connection and making sure all
the data is delivered correctly, and in
order.


Customize by defining a class to handle
each connection


A simple server


Echo server


Receives a message from a client and sends
the same message back.


Actually useful for testing that a connection is
working, and the round trip time for a message


# Example from
Goldwasser

book

from
SocketServer

import
TCPServer
,
BaseRequestHandler

class
EchoHandler(BaseRequestHandler
):


def
handle(self
):


message = self.request.recv(1024)


self.request.send(message
)


# may need to customize
localhost

and port for your machine

echoServer

=
TCPServer
( ('
localhost
', 9128),
EchoHandler
)

echoServer.serve_forever
()

The client side of echo


Connect to the server


Send the message


Get the message back

>>> from socket import socket

>>> echo = socket()

>>> echo.connect(('localhost',9128))

>>>
echo.send('This

is a test')

14

>>> print echo.recv(1024)

This is a test

>>>

Spot check


Get the echo server running on your
machine.


Make the echo client connect to the
server and echo a message.

Basic web server


Similar to the echo server, except return
a requested file instead of echoing the
message


Overall outline


Accept the connection


Receive the file request


Search for the file in the path provided


If not there, return error message (404!)


If there, return the file


Web server

from
SocketServer

import
TCPServer
,
BaseRequestHandler


class
WebHandler(BaseRequestHandler
):


def
handle(self
):


command = self.request.recv(1024)


if command[:3] == 'GET':


pagename

= command.split()[1][1:] # remove leading '/' for filename


try:


requestedFile

=
file(pagename
, '
r
')


content =
requestedFile.read
()


requestedFile.close
()


header = 'HTTP/1.0 200 OK
\
r
\
n
'


header += 'Content
-
Length: %
d
\
r
\
n
\
r
\
n
' %
len(content
)


self.request.send(header
)


self.request.send(content
)


except
IOError
: # could not open the file


self.request.send('HTTP/1.0 404 Not Found
\
r
\
n
\
r
\
n
')


webServer

=
TCPServer
( ('
localhost
', 8080),
WebHandler
)

webServer.serve_forever
()

Very simple.
Assume file in local
directory

Note creation of file
header information
sent back to
requestor with the
file

Note: runs on local
host, not accessible
over the Internet

Spot check


If not now, then later




Write the corresponding client to
request a file and receive it back.

Persistence


So far, connection made, something
happens, connection ends.


Server is then free to handle another
connection.


Client is disconnected entirely


Some applications require persistence


continuing connection between client
and server

Terry Scott University of Northern Colorado
2007 Prentice Hall

36

Case Study:

Network Chat
-
Room


Develop our own protocol for chat room.


Connection has permanence unlike previous
examples


persistent connection.


Client must monitor keyboard and also listen to
the socket.


Use multithreading to accomplish this. Without
multithreading can get hung
-
up on listening to
socket or the keyboard.


Following two slides show different messages
sent from client to server and server to client.

Terry Scott University of Northern Colorado
2007 Prentice Hall

37

Client to Server Messages

Message Type

Format

Join the room
using given
identity.

'ADD %s
\
n' % screenName

Broadcast the
message to
everyone.

'MESSAGE %s
\
n'%content

Send a private
message.

'PRIVATE %s
\
n%s
\
n'
%(recipient, content)

Quit the chat
room.

'QUIT
\
n
'

Terry Scott University of Northern Colorado
2007 Prentice Hall

38

Server to Client Messages

Message Type

Format

New user has joined.

'NEW %s
\
n' % screenname

Message was
broadcast to
everyone.

'MESSAGE %s
\
n%s
\
n' %

(sender, content)

Private message
was sent to user.

'PRIVATE %s
\
n%s
\
n' %
(sender, content)

Someone has left the
room.

'LEFT %s
\
n' % screenName

Acknowleges
request to quit.

'GOODBYE
\
n
'

Terry Scott University of Northern Colorado
2007 Prentice Hall

39

Chat Server


Uses
ThreadingTCPServer

rather than
TCPServer
. Needed since can be multiple
people in the chat room.


_broadcast function used to send to
clients in the
chatroom
.


Each person that joins the chat
-
room is
given a new socket that is accessed via a
dictionary with the person screen name
as the key.


Code for the server is on the next three
slides.

Terry Scott University of Northern Colorado
2007 Prentice Hall

40

Chat Server Code

from

SocketServer

import
ThreadingTCPServer
,
BaseRequestHandler

_
socketLookup

=
dict
()


def

_
broadcast(announcement
):


for

connection
in

_
socketLookup.values
():



connection.send(announcement
)


class

ChatHandler(BaseRequestHandler
):


def

handle(
self
):



username = 'Unknown'



active =
True

Terry Scott University of Northern Colorado
2007 Prentice Hall

41

Chat Server Code (continued)



while

active:




transmission
=
self
.request.recv(1024)




if

transmission:




command = transmission.split()[0]




data = transmission[1+len(command):]




if

command == 'ADD':




username =
data.strip
()





_
socketLookup[username
] =
self
.request





_
broadcast('NEW

%
s
\
n
' % username)




elif

command == 'MESSAGE':





_
broadcast('MESSAGE

%
s
\
n%s
\
n
' %






(
username,data
))

Terry Scott University of Northern Colorado
2007 Prentice Hall

42

Chat Server Code (continued)




elif

command == 'PRIVATE':





rcpt = data.split('
\
n')[0]





if
rcpt
in
_
socketLookup
:





content = data.split('
\
n')[1]






_
socketLookup[rcpt].send
(





'PRIVATE %
s
\
n%s
\
n
' % (
username,content
))




elif

command == 'QUIT':





active =
FALSE





self
.request.send('GOODBYE
\
n
')




else:





active =
False



self
.request.close
()



_
socketLookup.pop(username
)



_
broadcast('LEFT

%
s
\
n
' %username)

myServer
=ThreadingTCPServer('localhost',9000),
ChatHandler
)

myServer.serve_forever
()


Sufficient unto the day


or the semester


Hopefully, we have accomplished a good
bit in the last four months


Python, object
-
oriented programming,


Some natural language processing


as examples of language modules and
algorithms


A bit of networking even.