Practical_4x

mangledcobwebSoftware and s/w Development

Dec 14, 2013 (3 years and 5 months ago)

95 views

Advanced operating systems course, The University of Tokyo

Process working set and page
replacement in WinNT

Practical session #4

バリ

ゲローフィ

Outline

Process working set

Working set implementation

Aging, trimming and replacement

Modifying the replacement algorithm



Priority inversion user
-
land


2

3

Process Working
Set

Working set:

All
the physical pages “owned” by a process

Essentially, all the pages the process can reference
without incurring a page fault

Working set limit:

The
maximum pages the process
can

own

When limit is reached, a page must be released for
every page that’s brought in (“working set replacement”)

Default upper limit on size for each process

System
-
wide maximum calculated & stored in
MmMaximumWorkingSetSize

Approximately
RAM minus 512 pages (2 MB on x86)
minus min size of system working set (1.5 MB on x86)

Interesting to view (gives you an idea how much memory
you’ve “lost” to the OS)

True upper limit: 2 GB minus 64 MB for 32
-
bit Windows

4

Working Set List

A process always starts with an empty working set

It then incurs
page faults
when referencing a page that isn’t in its
working set

Many page faults may be resolved from memory (to be described
later
)

Just an array of entries maintaining some statistics for each page
mapped in to the process’ address space


PerfMon

Process “WorkingSet”

newer pages

older pages

5

Working Set Replacement

When working set max reached (or working set trim occurs), must give up
pages to make room for new pages

Local page replacement policy (most Unix systems implement global
replacement)

Means that a single process cannot take over all of
the physical
memory unless other processes aren’t using it

Page replacement algorithm is
an approximation of the least
recently
used
(LRU) method

Pages
are
aged based on access bit

PerfMon

Process “WorkingSet”

to standby


or modified


page list

Working set implementation:

data structures

6

MMSUPPORT

MinimumWorkingSetSize
: minimum size of
working set

MaximumWorkingSetSize
: maximum size of
working set

WorkingSetSize
: current size of working set

VmWorkingSetList
: pointer to the working set list

NextAgingSlot
: index from where aging
algorithm starts next time

7

_MMWSLE


Working set list

FirstDynamic
: the first working set index that
could be removed

LastEntry
: the last valid entry

NextSlot
: index from where trimming starts (i.e.
the last scanned index + 1)

FirstFree
: the first free index (if there is any)

Wsle
: the actual array of working set entries


8

Working set manager source files

File name

Description

ps.h

Process data structures and

part of the working set

mi.h

Memory management data structures,

also includes
some part of the working set

wslist.c

Operations

on working sets

wsmanage.c

Working set management thread

9

Where is the decision made which
pages to remove?

Global method (
BalanceSetManager
):

Iterates a list of working sets (processes) and calls
MiProcessWorkingSet
()
-
>
MiTrimWorkingSet
()
-

removes multiple pages

Per
-
process method during page fault:

MmAccessFault
()
-
>
MiAddValidPageToWorkingSet
()
-
>
MiAllocateWsle
()
-
>
MiDoReplacement
()
-
>
MiReplaceWorkingSetEntry
()


drops one page

10

MiTrimWorkingSet
()

Iterates the working set and looks for pages:

That are older than a certain age (LRU!)

Collects them to a list (called
WsleFlushList
)

Calls
MiRemoveWorkingSetPages
() for actual
removal



11

MiReplaceWorkingSetEntry
()

Called in the page fault handler

Looks for a page in the working set that can be
removed, only one page:

It scans working set and tries to find a page that is
older than the specified limit (also keeps track the
oldest during the search)

it doesn’t check more than a predefined limit
(MM_WORKING_SET_LIST_SEARCH = 17) so
that the time spent on search is reasonable

It frees the entry


12

How is age maintained?
(the foundation of LRU)

MiAgeWorkingSet
() function:

Walks trough a portion of a
working set

Checks PTE access bit

If not set:

Increases the age field of the
WSLENTRY

13

VOID

MiAgeWorkingSet

(


IN PMMSUPPORT
VmSupport
,


IN LOGICAL
DoAging
,


IN PWSLE_NUMBER
WslesScanned
,


IN OUT PPFN_NUMBER
TotalClaim
,


IN OUT PPFN_NUMBER
TotalEstimatedAvailable


)


Experiment with page removal

Test scenario:

Use
SetProcessWorkingSetSizeEx
() to set working
set size to max 256 pages

Allocate more memory than 256 pages

Refer the pages periodically and see which ones
get removed during page faults


14

Let’s modify the paging algorithm

Modify
MiReplaceWorkingSetEntry
() so that it
decides which page to drop based on some
other policy (not the pages’ age), for instance:

FIFO

LIFO (does it make sense?)

Random

Page out modified pages first

Clock algorithm

Suggestions?

15

Log from default policy:

16

MiFreeWsle
: (pid:1604) drops
wset

ID: 6, address: 0x318

MiFreeWsle
: (pid:1604) drops
wset

ID: 7, address: 0x319

MiFreeWsle
: (pid:1604) drops
wset

ID: 8, address: 0x31a

MiFreeWsle
: (pid:1604) drops
wset

ID: 9, address: 0x31b

MiFreeWsle
: (pid:1604) drops
wset

ID: 10, address: 0x31c

MiFreeWsle
: (pid:1604) drops
wset

ID: 11, address: 0x31d

MiFreeWsle
: (pid:1604) drops
wset

ID: 12, address: 0x7c81f

MiFreeWsle
: (pid:1604) drops
wset

ID: 13, address: 0x7c812

MiFreeWsle
: (pid:1604) drops
wset

ID: 15, address: 0x31e

MiFreeWsle
: (pid:1604) drops
wset

ID: 16, address: 0x7c82e

MiFreeWsle
: (pid:1604) drops
wset

ID: 17, address: 0x7ffdf

MiFreeWsle
: (pid:1604) drops
wset

ID: 18, address: 0x31f

MiFreeWsle
: (pid:1604) drops
wset

ID: 19, address: 0x430

MiFreeWsle
: (pid:1604) drops
wset

ID: 20, address: 0x431


Same scenario (referring same
addresses) with FIFO:

17

MiFreeWsle
: (pid:1952) drops
wset

ID: 582, address: 0x316

MiFreeWsle
: (pid:1952) drops
wset

ID: 504, address: 0x317

MiFreeWsle
: (pid:1952) drops
wset

ID: 503, address: 0x77e6f

MiFreeWsle
: (pid:1952) drops
wset

ID: 455, address: 0x318

MiFreeWsle
: (pid:1952) drops
wset

ID: 413, address: 0x319

MiFreeWsle
: (pid:1952) drops
wset

ID: 409, address: 0x31a

MiFreeWsle
: (pid:1952) drops
wset

ID: 402, address: 0x31b

MiFreeWsle
: (pid:1952) drops
wset

ID: 396, address: 0x31c

MiFreeWsle
: (pid:1952) drops
wset

ID: 362, address: 0x77eaa

MiFreeWsle
: (pid:1952) drops
wset

ID: 360, address: 0x7c81f

MiFreeWsle
: (pid:1952) drops
wset

ID: 356, address: 0x7c812

MiFreeWsle
: (pid:1952) drops
wset

ID: 331, address: 0x31e

Let’s see the code…


18

Assignment #3

Pick a method and change the page
replacement algorithm!

Print out the page replacements and explain
how the log corresponds to your method!



19

Priority inversion user
-
space
application


Anybody implemented already?

Problems?

Question?


20