Windows System Programming using Python

gorgeousvassalSoftware and s/w Development

Nov 7, 2013 (3 years and 9 months ago)

56 views

Windows System
Programming using
Python

Mark Hammond

mhammond@skippinet.com.au



OReilly Open Source Python Conference

August 1999, Monterey, CA

O’Reilly Python Conference

Aug 24, 1999

Windows System Programming using Python

Slide
2

About this Presentation


Most content taken directly from
upcoming book for O’Reilly


Python Programming on Win32

By Mark Hammond and Andy Robinson



http://www.ora.com/catalog/pythonwin32/

O’Reilly Python Conference

Aug 24, 1999

Windows System Programming using Python

Slide
3

Who this talk is for?


Existing Python programmers


Even those without any Windows experience
should follow this without problem.


Although there is not enough time to explain the
relevant Windows APIs


Existing Windows Programmers


Even without Python experience, you should
immediately see the similarities between your
existing language.

O’Reilly Python Conference

Aug 24, 1999

Windows System Programming using Python

Slide
4

Python and Windows


Each Python version comes with an
installer package for Windows.


Standard Python port contains all cross
-
platform Python features, but very few
Windows specific features.


Python for Windows extensions contains
many useful Windows extensions for
Python.


http://www.python.org/windows

O’Reilly Python Conference

Aug 24, 1999

Windows System Programming using Python

Slide
5

Python Windows Extensions


Includes:


Pythonwin: MFC based GUI environment and
IDE/Debugger


win32com: Interfaces Python and COM


win32 Extensions: Interfaces to native Win32 API.


Official releases can be found at
http://www.python.org/windows


Extensions home is at
http://starship.python.net/crew/mhammond


O’Reilly Python Conference

Aug 24, 1999

Windows System Programming using Python

Slide
6

System Level Programming?


For this talk, we define system level
programming as working with low
-
level
features of Windows Files, Pipes,
Processes, Threads, Services, Event Log
and so forth.


Python and similar languages really not
suitable for device
-
driver type
development, and other more system
-
like
Systems Programming!

O’Reilly Python Conference

Aug 24, 1999

Windows System Programming using Python

Slide
7

Why not just use Python?


Python has excellent native support for
files, processes, threads etc.


These features are typically limited to
those defined by ANSI C.


Many advanced Windows features are not
exposed using these interfaces.


Standard implementation of some of the standard
library functions leaves a little to be desired in a
Windows environment.

O’Reilly Python Conference

Aug 24, 1999

Windows System Programming using Python

Slide
8

Portable Process Control

(1 of 3)


Standard Python functions all work


Just often not quite how we would like!


os.system()

import os

os.system(“notepad C:
\
\
autoexec.bat”)

Problems


Creates a new console window when run from a
GUI.


Waits for process to terminate.

O’Reilly Python Conference

Aug 24, 1999

Windows System Programming using Python

Slide
9

Portable Process Control

(2 of 3)


os.execv family


Doesn’t search system path, and doesn’t parse
command lines

os.execv("c:
\
\
Winnt
\
\
notepad.exe",
\


("c:
\
\
autoexec.bat",) )


Does clobber your existing process
-

the call to
os.execv() never returns!

O’Reilly Python Conference

Aug 24, 1999

Windows System Programming using Python

Slide
10

Portable Process Control

(3 of 3)


os.popen()

>>> file = os.popen("echo Hello")

>>> file.read()

'Hello
\
012'


Works fine from Windows NT console programs,
but fails miserably from a GUI!


win32pipe module in the Win32 extensions
provides a working replacement.

O’Reilly Python Conference

Aug 24, 1999

Windows System Programming using Python

Slide
11

Better Process Control

(1 of )


win32api module provides some high
-
level, Windows specific functions.


win32api.WinExec()


Very similar to os.system(), but overcomes
limitations.


>>> import win32api


>>> win32api.WinExec("notepad")


Optional parameter allows you to specify the
Window’s initial state (eg, minimized)

O’Reilly Python Conference

Aug 24, 1999

Windows System Programming using Python

Slide
12

Better Process Control

(2 of 2)


win32api.ShellExecute()


Typically opens “documents”
-

eg, execute
“foo.doc”, and (typically) Word will open.


Finer control over the new process.


Can also execute arbitrary executables
-

not
limited to documents.


For example, to print a specific document:

win32api.ShellExecute(0, "print",
\


"MyDocument.doc", None, "", 1)

O’Reilly Python Conference

Aug 24, 1999

Windows System Programming using Python

Slide
13

Ultimate Process Control

(1 of 2)


win32process module exposes the low
level Win32 API.


Full support for CreateProcess,
CreateProcessAsUser, CreateThread etc.


Full support for Windows Handles


Files can be passed as stdin/out/err


Process and thread handles are waitable using
the win32event module.

O’Reilly Python Conference

Aug 24, 1999

Windows System Programming using Python

Slide
14

Ultimate Process Control

(2 of 2)


Able to set thread and process priority and
affinity levels


Very handy for bugs that only appear in multi
-
processor machines.


Able to do all this for both existing and
new processes.


Process usage samples included in
distribution.

O’Reilly Python Conference

Aug 24, 1999

Windows System Programming using Python

Slide
15

Introduction to our Sample


Full sample Windows NT Service


Provides event log and performance monitor
information. Clients connect using Named Pipes


Less than 75 lines of code for all this functionality.


Still too big to present in one hit


Selected excerpts included in slides


Full code on CD, and at

http://starship.python.net/crew/mhammond/conferences/
ora99

O’Reilly Python Conference

Aug 24, 1999

Windows System Programming using Python

Slide
16

Portable Files and Pipes


Python has excellent built
-
in file support


Inherits platform stdio support
-

wont bother
discussing them here.


Native Windows files only useful for
highly
-
advanced features, such as:


Overlapped IO


IO Completion Ports


Named Pipes


NT Security requirements

O’Reilly Python Conference

Aug 24, 1999

Windows System Programming using Python

Slide
17

Native Files

(1 of 4)


win32file.CreateFile() used for most file
operations


Create and open regular files, memory mapped
files, etc.


Takes seven parameters
-

c.f. open()’s two!


Returns a PyHANDLE object


Can be passed to any Python API wrapper expecting a
handle.


Auto
-
closed when last reference removed.

O’Reilly Python Conference

Aug 24, 1999

Windows System Programming using Python

Slide
18

Native Files

(2 of 4)


Overlapped IO for asynchronous
operations


File opened for overlapped IO requires a
Windows event object.


All IO operations return immediately.


Event object signalled when IO complete


Great for high
-
performance servers


Simple to support multiple concurrent file
operations per thread.

O’Reilly Python Conference

Aug 24, 1999

Windows System Programming using Python

Slide
19

Native Files

(3 of 4)



NT Completion ports for even better
asynchronous control


Windows manages associating the completed IO
operation with a connection


More complex to use.


Often requires a state
-
machine implementation.


Offers excellent performance
-

Microsoft’s
recommended architecture for scalable, high
-
performance servers.

O’Reilly Python Conference

Aug 24, 1999

Windows System Programming using Python

Slide
20

Native Files

(4 of 4)



Full support for NT Security


Default security can be used by passing None


Many real
-
world applications require explicit
security configuration


Full support for Windows Impersonation


Processes can automatically impersonate the
remote pipe client.


Impersonate any user given their password.

O’Reilly Python Conference

Aug 24, 1999

Windows System Programming using Python

Slide
21

File Sample

(1 of 2)


Overlapped IO is used


self.overlapped =
\


pywintypes.OVERLAPPED()

# create the event to be used.

self.overlapped.hEvent =
\


win32event.CreateEvent(None,0,0,None)



Special security for pipes is needed for
services


sa = win32security.SECURITY_ATTRIBUTES()

# Allow full access!

sa.SetSecurityDescriptorDacl ( 1, None, 0 )

O’Reilly Python Conference

Aug 24, 1999

Windows System Programming using Python

Slide
22

File Sample

(2 of 2)


Create pipe and connect to client


pipeHandle =
\


win32pipe.CreateNamedPipe(pipeName,


openMode,


pipeMode,


win32pipe.PIPE_UNLIMITED_INSTANCES,


0, 0, 6000, # 6 second timeout.


sa)

...

hr = win32pipe.ConnectNamedPipe(pipeHandle,
\


self.overlapped)


O’Reilly Python Conference

Aug 24, 1999

Windows System Programming using Python

Slide
23

Windows NT Services


Similar concept to a Unix daemon


A few special requirements


Must respond to asynchronous commands from
NT to (e.g.) Shutdown


Must be capable of reporting status to NT


NT has built
-
in UI for configuring and
controlling services.

O’Reilly Python Conference

Aug 24, 1999

Windows System Programming using Python

Slide
24

Python controlling Services


Full exposure of the Windows NT Service
Control Manager API


Start, Stop, Pause Services, Install or Remove
services, etc


win32serviceutil module makes it simple


>>> win32serviceutil.StopService("Messenger")

(32, 3, 0, 0, 0, 6, 20000)

>>> win32serviceutil.StartService(...)

>>>

O’Reilly Python Conference

Aug 24, 1999

Windows System Programming using Python

Slide
25

Implementing Services


Simple to implement Services in Python


Smallest service is around 16 lines of code!


Includes debug support, self
-
installation, and fully
controllable from Windows NT


Few more lines needed to make it something useful :
-
)


Simply sub
-
class Service base
-
class, and
implement control features


Minimum required is StopService


Trivial to implement most controls

O’Reilly Python Conference

Aug 24, 1999

Windows System Programming using Python

Slide
26

Windows NT Event Log


Central place for programs to log
information.


Ideal for Services
-

can not present effective GUIs


Benefits to programmer


Built in transaction and thread safety, maximum
size ability, etc.


Benefits to Administrator


Central log of messages, and third party tools to
help analysis.

O’Reilly Python Conference

Aug 24, 1999

Windows System Programming using Python

Slide
27

Python reading the Event Log


Complex native API from win32evtlog


Simpler interface from win32evtlogutil


Define Feeder function

def DumpRecord(record):


print ”Got event ID”, record.EventID


And feed it!

win32evtlogutil.FeedEventLogRecords(
\


DumpRecord)

Got Event ID
-
2147483645

Got Event ID
-
2147483645

O’Reilly Python Conference

Aug 24, 1999

Windows System Programming using Python

Slide
28

Python writing the Event Log


More complex than reading
-

event
sources must be registered.


win32evtlog and win32evtlogutil used here
too


Python Service framework also supports
simple event log writing

O’Reilly Python Conference

Aug 24, 1999

Windows System Programming using Python

Slide
29

Writing the Event Log Sample


Our sample uses the Service Framework
functions which makes it trivial


Events are logged by ID, rather than
explicit text.


Our sample uses a built
-
in ID to log a “service
starting” message


import servicemanager

servicemanager.LogMsg(


servicemanager.EVENTLOG_INFORMATION_TYPE,


servicemanager.PYS_SERVICE_STARTED,


(self._svc_name_, ''))

O’Reilly Python Conference

Aug 24, 1999

Windows System Programming using Python

Slide
30

NT Performance Monitor


Built
-
in NT tool for monitoring an
applications performance.


Application must be written to provide this
data.


API designed for smallest impact on
program supplying data


Moves the burden to the collecting application.


Not trivial to work with

O’Reilly Python Conference

Aug 24, 1999

Windows System Programming using Python

Slide
31

Python reading from PerfMon


Useful for overcoming limitations in built
-
in tool


For example, sample the data hourly and log to a
longer
-
term database.


Useful for reading standard information about
another process.


Process ID, Memory usage, etc.


win32pdhutil module for reading data


Good sample code in the source file

O’Reilly Python Conference

Aug 24, 1999

Windows System Programming using Python

Slide
32

Python supplying PerfMon


Complex installation and setup procedure.


C .h file and custom .ini file necessary at
application installation


Supported by the Python Service framework


Quite trivial to use once running


Simply increment a counter
-

Performance
Monitor handles conversions to required units.

O’Reilly Python Conference

Aug 24, 1999

Windows System Programming using Python

Slide
33

PerfMon Sample


Installation more than we can cover here


See InitPerfMon() method


Working with counters trivial


Increment our connections counter each
connection


self.counterConnections.Increment()


Perfmon manages converting to
connections
-
per
-
second automatically.

O’Reilly Python Conference

Aug 24, 1999

Windows System Programming using Python

Slide
34

Our Sample in Detail

(1 of 8)


PipeService2.py


The service implementation.


PipeService2_install.h

PipeService2_install.ini


Required for performance monitor installation


Most services don’t need this!


PipeServiceClient.py


Sample client to connect to our service.

O’Reilly Python Conference

Aug 24, 1999

Windows System Programming using Python

Slide
35

Our Sample in Detail

(2 of 8)


Service functionality in PipeService class


Base class handles most of the grunt


We simply supply service name and other
optional attributes


class PipeService(
\


win32serviceutil.ServiceFramework):


_svc_name_ = "PythonPipeService"


_svc_display_name_ = "A sample Python ... "


O’Reilly Python Conference

Aug 24, 1999

Windows System Programming using Python

Slide
36

Our Sample in Detail

(3 of 8)


We respond to an NT ServiceStop request
by telling NT we are stopping, and setting
a Windows Event


def SvcStop(self):


# Before we do anything, tell the


# SCM we are starting the stop process.


self.ReportServiceStatus(
\


win32service.SERVICE_STOP_PENDING)


# And set my event.


win32event.SetEvent(self.hWaitStop)


O’Reilly Python Conference

Aug 24, 1999

Windows System Programming using Python

Slide
37

Our Sample in Detail

(4 of 8)


Overlapped IO means we can wait for
either a connection, or our Stop request


win32pipe.ConnectNamedPipe(pipeHandle,
\


self.overlapped)


...


# Wait for either a connection, or


# a service stop request.


waitHandles = self.hWaitStop,
\


self.overlapped.hEvent


rc = win32event.WaitForMultipleObjects(
\


waitHandles, 0, timeout)


O’Reilly Python Conference

Aug 24, 1999

Windows System Programming using Python

Slide
38

Our Sample in Detail

(5 of 8)


Install our service simply by executing the
service script


If not for Performance Monitor, the command
-
line
would be simple:

C:
\
Scripts> PipeService2.py install


But PerfMon needs an extra arg

C:
\
Scripts> PipeService2.py
\

--
perfmonini=PipeService2_install.ini install

Installing service PythonPipeService to ...

Service installed


O’Reilly Python Conference

Aug 24, 1999

Windows System Programming using Python

Slide
39

Our Sample in Detail

(6 of 8)


Once installed, we can start the service


Start from Control Panel, or using the script itself

C:
\
Scripts> python.exe PipeService2.py start


And start a client test session

C:
\
Scripts> python.exe PipeServiceClient.py
\


Hi there

The service sent back:

You sent me:Hi there


O’Reilly Python Conference

Aug 24, 1999

Windows System Programming using Python

Slide
40

Our Sample in Detail

(7 of 8)


We will have Event
Log records...

O’Reilly Python Conference

Aug 24, 1999

Windows System Programming using Python

Slide
41

Our Sample in Detail

(8 of 8)



And Performance
Monitor Data.

O’Reilly Python Conference

Aug 24, 1999

Windows System Programming using Python

Slide
42

Summary


Python library flexible, rich and portable
enough for many tasks


Low
-
level Windows programming
achieved by using extensions that expose
the raw Windows API


This talk should have shown:


An overview of how this programming is done in
Python.


How simple it is to do complex Windows tasks.

O’Reilly Python Conference

Aug 24, 1999

Windows System Programming using Python

Slide
43

More Information


Python and Python Documentation


http://www.python.org


http://www.python.org/doc


Python for Windows Extensions


http://starship.python.net/crew/mhammond


http://www.python.org/windows


Reference Manuals and Samples included with
the distributions

O’Reilly Python Conference

Aug 24, 1999

Windows System Programming using Python

Slide
44

Thanks for coming


Mark Hammond


mhammond@skippinet.com.au


http://starship.python.net/crew/mhammond