Modeling the effects of International Interventions

runmidgeΤεχνίτη Νοημοσύνη και Ρομποτική

20 Οκτ 2013 (πριν από 3 χρόνια και 8 μήνες)

77 εμφανίσεις





Modeling the effects of International Interventions

with Nexus Network Learner

Dr. Deborah Duong

Agent Based Learning Systems

AAAI12 Spring Symposium, Stanford University


Purpose: To show the benefits of Symbolic
Interactionist

Simulation for the Simulation of Social Aggregation

Purpose and Agenda


Agenda

2


Coevolution and
Symbolic
Interactionist

Simulation


The Socio
logical Dynamical System
Simulation


SISTER: Symbolic
Interactionist

Simulation
of Trade and Emergent Roles


Nexus Network Learner


Modeling Corruption



Social Impact Model

Coevolving Agents


Genetic Algorithms, Neural Networks, or Reinforcement Learning
Algorithms in Agents can co
-
evolve


Agents learn to optimize a function in an environment composed
mostly of agents also learning to optimize a function.


Moving fitness landscapes


Agents apply selective pressure upon each other.


Selective pressure causes the optimal way to achieve the goal to
change over time.


Evolutionary Stable Strategies


Maynard
-
Smith’s theory that a co
-
evolutionary system
converges when no species (or inducing agent) can make a
change that will make it better off (Nash Equilibrium)


Agents Differentiate into a System


Species (or inducing agents) can be both cooperative and
competitive

3

Social Impact Model

Symbolic
Interactionist

Simulation


Coevolutionary

reinforcement learning algorithm.


Autonomous agents each have an inductive mechanism.


For example, an entire Genetic Algorithm or Neural
Network.


Agents only experience through their senses (no direct
knowledge transfers from other agent minds).


Agents choose to interact with other agents based on
signs that the other agents display.


Agents induce both the signs to display and the signs that
they read.


Social system emerges


Signs come to mean behaviors.


Behaviors interlock into a system of expectations.


Value function becomes reward function as society
evolves (Adam Smith’s invisible hand).

4

Social Impact Model

The First Symbolic
Interactionist

Simulation: The
Sociological Dynamical System Simulation


A System of IAC Networks as the Basis for Self Organization
in a Sociological Dynamical System Simulation


-

Duong, 1991; Duong and Reilly, Behavioral Science 1995.


Employer agents hire from a pool of employee agents, periodically laying
off employees.


Employees who are less talented are laid off in greater proportions than
those who are more talented.


Employees seek employment.


Employees can choose to wear three different kinds of signs


One of two signs they can not change. This is their “skin color”.


One of three signs they have to pay for with money from
employment. This is their “suit”.


One of three signs they can change arbitrarily. This is their “fad”.


Employees induce what sign they should wear based on what has gotten
them employed in the past.


5

Social Impact Model

The Sociological Dynamical System Simulation
(SDSS)


Employees have a hidden, unchanging, talent level that
employers can not see until after the employee is employed.


Employers seek talented employees.


Employers hire based on the signs the employees wear,
inducing how much talent they have from their signs.


The two “races” of employees are equally talented, but the
employers do not know this.


Employers and Employee Agents both have the same kind of
induction mechanism.


Each Agent has their own Interactive Activation and
Competition (IAC) Neural Network to induce the signs they
read and display.


6

Social Impact Model

Interactive Activation and Competition


An Employer’s IAC


7

Social Impact Model


The IAC is a Constraint
Satisfaction (Hopfield) Neural
Network.


Nodes within each pool are
mutually inhibited.


A central pool contains memory
instances.


Each instance has positive
connections to its features.


To guess the talent of an
applicant, the employer “turns
on” the applicants features and
sees which talent node turns
on.


Simulates schema, or mental

groupings of features that go
together


SDSS: Emergent Phenomena

8

Social Impact Model


Status Symbols


Employee agents learn to buy expensive suits and Employer
agents learn to seek expensive suits.


Because the less talented get laid off more, they have less
money, and talented employees learn to differentiate
themselves.



Racism and Social Class


One race gets into a vicious cycle: because of schema, by
accident one race gets associated with less talent (even though
they are equally talented).


Many talented in a race could not afford suits because they were
never given opportunity.


One race would have less money than the others as a result



Meaning attributed to Fads



SISTER: Symbolic
Interactionist

Simulation of
Trade and Emergent Roles


SISTER: A Symbolic
Interactionist

Simulation of Trade and Emergent
Roles”
-

Duong 1995, Duong and
Grefenstette
,
JASSS

1/2005.


Trade is good for “farmer” agents.


Agents need each of four food groups, and as much as they
can get of each.


Agents can make more food if they concentrate their efforts on
fewer of them.


Agents have efforts to spend on making or trading food as
they wish.


Agents can trade if they have corresponding trade plans.


Agents have a sign to display to attract trade.


To learn to trade, agents induce what signs to display and
what signs to trade with based on whatever gets them the
most of each food in the four food groups at the end of the
day.


9

Social Impact Model

Coevolving Genetic Algorithms


Each agent has an entire
genetic algorithm that
tells it:


Where to place efforts


What sign to trade with,
what and how much to
trade


What sign to display

10

Social Impact Model

SISTER: Emergent Phenomena

11

Social Impact Model


Division of Labor


As

agents learn to trade,
their utility increases, and
their sign comes to mean
a role


Price


Goods become valued at
standard ratios


Money


In a third of the runs, one
good is traded for the
purpose of trading again


Different types of stores


Central bargain stores
and local convenience
stores


SISTER: Results

12

Social Impact Model

Nexus


SISTER is a “theoretical” symbolic
interactionist

simulation


SISTER embodies the formation of social patterns of
behavior


Its scenarios are not realistic



Nexus is a “data
-
driven” symbolic
interactionist

simulation


In order to do analysis, we must start from a scenario in
the real world.


Being realistic and theoretically correct at the same time
is difficult.


Nexus attempts to mirror the virtuous and vicious cycles
of the real world that created its input data.


13

Social Impact Model

What is Nexus Network Learner ?

14


One of the two Nexus Cognitive Agent models that Debbie
Duong wrote at the OSD/CAPE/Simulation Analysis Center.


Nexus Network Learner models Social Role Networks


Nexus Schema Learner models Cognitive Dissonance


A Simulation of Social Role Networks in which Agents learn:


To choose role partners to perform transactions with:

-
Choice based on signs, social markers and communications
on past transaction behaviors.


Transaction behaviors and signs.

-
Choice based on signs and social markers.


Based on Cultural Values.


Social markers, roles, transaction behaviors, signs, role
-
based communications and cultural values are all input to
the program.


Population data determines the initial population tendencies.


Utility and motivation determines how they change.

How Does Nexus Network Learner Work?

15


Artificial Intelligence Technologies represent Cognition.


Rule Based.

-
An ontology of roles with crisp rules for roles.


Represents general social structures, that can be used in many
scenarios.


Defines utility of transactions.


Machine Learning.

-
Bayesian networks initialize social markers , signs/transaction
behaviors, and role choice behaviors.

-
The Bayesian Optimization Algorithm (BOA) changes those
behaviors based on the utility of transactions.


BOA can be seeded with initialization data and injected data.


A form of Evolutionary Computation using reinforcement
Learning optimizes (
satisfices
) utility.


As conditionals change, the equilibrium point moves (in accord
with the New Institutional Economics).




What Happens in Nexus Network Learner?

16


Individual Agents Choose Network Partners.


Ontology tells who may choose and how many.

-
Example: an “Employer” may choose an average of 5 employees with a standard
deviation of three.


Bayesian network tells how the choices are ranked.


Passive role may have an option to reject offer.

-
Example: an “Employee” may reject an employer because a role relation has told
her he steals paychecks.


Ontology may include a chance occurrence of natural attrition.


Individual Agents engage in transactions.


Account distributions send funds through networks according to rules in
ontology and transaction behaviors in Bayesian networks.


Probability of observing, reporting, and knowing about behaviors are role
-
based.


Agents may go to jail, and not be allowed to participate in transactions for a time.


Every N cycles, they judge their learned strategies by utility based on
transactions that their valued role partners engaged in.


Ontology determines culturally valued individuals and transactions.


After testing all strategies agents recombine them.




Performing Tests with Nexus Network Learner

17


A wide variety of tests relevant to Irregular Warfare (IW) may be
performed.


For example, new network formations and behaviors may be tested
based on many different things…


The effect of different utility functions.

-
For example, make agents care only for self rather than larger social
network.


The effect of different penalties.

-
For example, a penalty attribute that encodes different jail terms or
different chances of getting caught.


The effect of different exogenous resources.

-
For example, test resource rents or foreign aid.


The effect of different abilities to observe.

-
For example, the effect of a media agent.


The effect of removing different agents.

-
For example, measure how long it takes to replace a terrorist leader


Monte Carlo methods reveal if new structures are the result of different
CONOPS.


Bayesian Networks make Nexus Stochastic


How Nexus Agents Learn

18


As each agent learns, all the agents
coevolve
, making them very
adaptive.


Every agent has its own private learning algorithm.


Their behaviors effect the larger social structure and the larger
social structure effects their behaviors.

-
Micro
-
Macro Integration is modeled.


They can adapt to data from other simulations and to initial
country data as well.


The learning algorithm in each agent makes the adaptation to
data flexible.


BOA (Bayesian Optimization Algorithm) can start learning from
initial data.

-
In the calibration phase. agents to adapt to initial data, so that
they generate it though their perceptions and motivations.

-
Thus they “explain” the data, going from correlation to cause.


This greater ability to ingest data also allows them to meld with
other simulations in a composition.

-
Together, composed simulations create a coherent picture of
the social environment.

-
Conflicts are resolved through mutual adaptation.

Use Case: Modeling Corruption with

Nexus Network Learner

19

Social Impact Model

Assumptions


A role perspective is appropriate for examining corruption


With
Nexus,
we may explore how the patron
-
client role relationships in traditional
African societies interact with the bureaucratic relationships made necessary by
globalization


People adjust their behaviors based not only on policies but on other peoples reactions to
policies


With
Nexus,
we can explore how agents adjust their behaviors to meet their cultural
goals, given that other agents are doing the same thing


Corruption is a social process, a vicious cycle


People typically do not participate because they like it, but because they feel they
have to


People take into account both legal penalties and social penalties in adjusting their
behavior


With
Nexus,
we can explore the effects of legal penalties on eight specific corrupt
behaviors, and how they interact with social penalties


The ability to hide what you are doing (bounded knowledge) is important


The chances that one person will know about another’s corrupt behaviors is based
on the role relation between the two


The more people know about a corrupt behavior, the more likely the perpetrator is to
get caught


With
Nexus,
we can explore the effects of transparency programs on the chances of
getting caught for a corrupt behavior, based on social relations



Perspective Orientation


The foundation for the Nexus Network
Learner is built upon a rich literature in
social constructivism and social
emergence where methodological
individualism (
only thing that matters
is an individual
) is rejected.


Nexus embraces the study of both
individuals and institutions:
endogenously (within the model)
modeling the institution
-
individual
linkage simultaneously with the
individual
-
institution links.

[1]

For an in
-
depth view see Bourdieu, P. (1977).
Outline of a Theory of Practice
. (R Nice, Trans.).
Cambridge: Cambridge University Press. (Original work published in 1972) and Giddens, A. (1984).
The
Constitution of Society
. Berkeley: University of California Press. Another potential source is Sawyer, RK.
(2005).
Social Emergence: Societies as complex systems
. Cambridge: Cambridge University Press.

Interpretive Social Science Used in Nexus


From economics: The New Institutional Economics (NIE) (North)


Institutions (Social and Legal Norms and Rules) underlie economic
activity and constitute economic incentive structures


Institutions come from the efforts of agents to understand their
environment, so as to reduce uncertainty, given their limited perception


When some uncertainties are reduced, others arise, causing economic
change


To find the leverages to corruption, NIE would look at actor’s definition of
their environment, and how this changes incentives and thus institutions


From sociology: Symbolic Interactionism (Mead)


Roles and Role Relations (such as in trade roles and trade relations) are
learned, created during the display and interpretation of signs (such as
gender, ethnicity, and other demographic characteristics)


Institutions (social and legal norms and rules) are commonly accepted
interpretations of symbols, that start out as a subjective perception and
engrained in society as an objective rule

[1]

See
http://coase.org/niereadinglist.htm

for an extended reading list

[2]

See Duong, Deborah Vakas, “The Generative Power of Signs: Tags for Cultural. Reproduction” Handbook of
Research on Agent
-
Based Societies: Social and Cultural Interactions, Goran Trajkovsky and Samuel Collins,
eds., 2008.
http://www.scs.gmu.edu/~dduong/GenerativePowerOfSigns.pdf

and also
Blumer, Herbert (1969).
Symbolic Interactionism: Perspective and Method. Berkeley: University of California Press.

[3]

Duong, Deborah Vakas and John Grefenstette. “SISTER: A Symbolic Interactionist Simulation of Trade and
Emergent Roles”. Journal of Artificial Societies and Social Simulation, January 2005.
http://jasss.soc.surrey.ac.uk/8/1/1.html
.

Conceptual Model


Nexus is a model of corruption based on the theory that corruption is a
result of globalization.


Many social scientists assert that corruption is the result of conflict
between the roles and role relations of the kin network and the trade and
bureaucratic networks, separate social structures with their own
institutions forced together because of globalization.

[1]

Smith, Daniel Jordan. 2007. A Culture of Corruption. Princeton: Princeton University Press.

Incentives

Trade

Network

Bureaucratic

Network

Kinship

Network

Transaction
-
based Utility

Strategy (Behavior)

Nexus Main Components


Nexus models individuals and
their interactions.


Individuals have various roles on
the three different networks and
dynamically interact with other
agents through these roles.


Retailer


Customer


Government Employer


Government
Employee


Head Of Household


Dependent


Role Networks are Input to Nexus



Only commodity is Money



Agents pass money to other agents’
accounts


External support is in the form of
injections of funds to certain individuals
(that have certain roles)


Utility of agents (their “happiness”) is
raised when they spend the money on
things they need



Trade

Network

Bureaucratic

Network

Kinship

Network

Role Interactions

Individual

Determined Traits

Situational Traits

Behavioral Traits

Transaction
-
based Utility

Experience

Cognition

Institutions

Emerged Learned Attributes

External Control

User
-
defined Policies

External

Support

Nexus Main Components:
Individuals



They want their kin to be happy, and can think about how to adjust their behaviors
towards that goal, based on experience of what met that goal in the past


They have the demographic characteristics, both determined and situational, of the
modeled country


Determined Traits: Gender, Ethnicity, Age, etc.


Behavioral Traits: Tendency to Steal or Bribe, based on other traits and on
learning during the run


Situational Traits: Employment, Are they under penalty, etc.


They actively seek role relationships, following socio
-
cultural rules about who
proposes the relationship, what sort of person is chosen


For example, a husband chooses a wife, or an employer chooses an employee


They judge others based on characteristics they can see or they have heard
rumors about


There role responsibilities include the distribution of funds to accounts they are
responsible for


They are happy when funds flow through certain accounts, for example, from the
household budget to the grocery store income.


They have differing length legal penalties, as well as social stigma



Nexus Main Components:
Role and Role Relations


There are eight types of corruption relations possible in the three networks
(example actions provided):


Nepotism
: Hiring Kin/ Trade Network


Commission for Illicit Services
: Bribing/Government Network


Misappropriation
: Stealing/ Trade or
Gov

Network


Rig Election
: Elected Officials bribing for Employment


Gratuity
: Bribing/ Trade Network


Unwarranted Payment
: Accepting Bribes/Government Network


Levy Toll Sidelining
: Stealing/Government Network


Scam
: Stealing From Customer in Trade Sector.


There are many other types of role and role relations (64) in the model:


Each role has a corresponding role


Roles are dynamic (such as an agent can move from a government employee to unemployed)

Bureaucratic Network

Gov Employee

Gov Employer

Attempts to Bribe

Accepts/Rejects
Bribe

Observer

Nexus Main Components:
Kinship Network

Active Roles

Corresponding Passive Role

Father

Child

Head Of Household

Home Receiver

Husband

Wife

Brother

Dependent (Provider)

MaternalAunt

MaternalCousin

MaternalGrandparent

Mother

Parent

PaternalCousin

PaternalGrandparent

PaternalUncle

Sibling

Sister

Spouse

Provider (Dependent)


Three main active roles, from
which 14 more are derived


Derived Roles are used to model
Residence


Utility (Satisfaction)
calculated based on
Residence in Anthropology


Matrilineal, Patrilineal of
Neolocal


Support account goes from
Provider to Dependent


Derived Roles

Nexus Main Components:
Trade Network


Derived Roles are ten different income levels


Accounts include personal salary, employee salaries, money for
office purchases



Active Roles

Corresponding Passive Role

HeadOfCorporation

CorporateReceiver

Customer

Retailer

Employer

Employee

Purchaser

Vendor

Service
Providee

Service Provider

Nexus Main Components:

Bureaucratic Network

Active Roles

Corresponding Passive Role

Taxpayer

Taxman

GovernmentEmployer

GovernmentEmployee

GovernmentPurchaser

GovernmentVendor

HeadOfGovernment

GovernmentReceiver

Service
Providee

Service Provider


Derived Roles are ten different pay grades


Accounts include corporate and income taxes, government salaries,
government office money, government money for purchases


Lets talk about the Agents


Agents are able to learn and adapt to new role behaviors through the use of
evolutionary computation techniques of artificial intelligence, also known as
genetic algorithms. (cognitive agents)


They learn other behaviors based on utility.



Utility is in the trade interactions (transaction
-
based utility) of the themselves and the kin that an
agent cares about.


Agents learn how to navigate their environment according to their individual traits and experience
through their own Bayesian Optimization Algorithm.


Agents learn the type of persons to include in their social network, including kinship, ethnicity,
and bribing behavior.


They also learn whether to divert funds across networks through bribing and stealing. Agents
have money in accounts (which is a situational attribute)



Corruption behavior changes through synchronous individual interaction, driven
by new incentive structures created from government policies, agent’s reactions to
those policies, and agent interaction.


Determined

Situational

Behavior

Traits

Behavior

Cognitive

Mechanism

Utility

Experience

Computational Model


Nexus was created with the REPAST
Simphony
, a free and open source
agent
-
based modeling toolkit


REPAST makes use of the social
network software Jung, shown below


Dots along the circle are agents’


Different colored arrows
represent relations of different
networks

-
Bureaucratic is Blue, Trade
is Green, Kin is Red


All the analysis functions of Jung and
data mining software
Weka

are
available for Nexus


Weka

displays number of agents
that did each type


Jung can describe characteristics
of the network like centrality,
reach, etc.

-
With Jung, you can tell who
are the important actors


So what happens first? (Initialization)


Demographic Data Input


The input to Nexus are the demographic characteristics for an entire population.

-
Physical characteristics, social categories, and behavioral traits that are based on these
physical characteristics and social categories.

-
Variables we are trying to explain are used to calibrate the simulation in the beginning.

-
Example: We know that a subset of the population in this region who have characteristics
A and B, have a greater propensity of corruption than you would expect by chance.


Application of Bayesian Network Algorithm (Data Interpretation)


Initial Population Representation


Describes characteristics that agents cannot change, for example, social markers such as
ethnicity or gender. (Determined Traits)


Describes characteristics that agents can change on an individual basis during the simulation, for
example, behavioral characteristics, such as bribing or stealing, or preferences for choices of
others in social networks. (Situational and Behavioral Traits)


Describes demographic characteristics which individual agents do not learn, but are rather the
output of the computations made during the simulation, such as unemployment statistics.
(Aggregated)

Demographic Data

Application of Bayesian
Network Algorithm

Initial Population

Representation

Application of Bayesian Network Algorithm


A Bayesian network is a graphical model that encodes probabilistic relationships
among variables of interest. When used in conjunction with statistical techniques,
the graphical model has several advantages for data analysis:



Because the model encodes dependencies among all variables, it readily handles situations
where some data entries are missing.


A Bayesian network can be used to learn causal relationships, and hence can be used to gain
understanding about a problem domain and to predict the consequences of intervention.


Because the model has both a causal and probabilistic semantics, it is an ideal representation for
combining prior knowledge (which often comes in causal form) and data
.


Bayesian statistical methods in conjunction with Bayesian networks offer an efficient and
principled approach for avoiding the over
-
fitting of data.


Methods to construct Bayesian networks include using prior knowledge and implementing the
Bayesian statistical methods for using data to improve the models. This includes both
parameters and the structure of a Bayesian network, and techniques for learning with incomplete
data.


Source: Neural Network Learning and Expert Systems, Stephen I Gallant, MIT Press, Cambridge, MA,
1993

Initial Population

Representation

Bayesian Logic

Observed Correlations

Example: We know bribing employers is

common practice in Ethnic Group A


Real World Data

Example: % of Ethnic Population in a
Region

How does the Cognitive Mechanism work?


In this model, the cognitive mechanism is the Bayesian optimization algorithm
(BOA).


The output of this algorithm is an agent’s strategy (how to distribute funds to
maximize utility


network choices and behaviors)


Recall at model initialization, the raw data is converted by a Bayesian network
into model input. Among other things, this process generates the individual
agent traits.


Determined (or fixed traits) are set per agent for the simulation run.


There are also two categories of behavioral traits (or learned behaviors):

-
Learned network choice behaviors (Initialized Random)


You choose a network partner when you choose a wife or
employee


You may choose a wife or employee based on ethnicity,


You may choose an employee based on whether he bribes you


You may reject an employer if your kin tells you he steals

-
Simple learned behaviors (set by Bayesian network during
initialization)


You may learn not to steal even if you started out that way, if it
harms your dependents

So what happens when you hit go? (
Simulation)


Remember that Agents are initialized with money in their accounts.


When the money reaches a certain threshold they will distribute it.

-
Distribution includes trade, for example, you distribute your money to a
grocery store, and gain utility for yourself in doing so


Exogenous funds are then pumped in on a regular basis (if there
are any like diamond revenue)

-
Wages, Taxes, Accounts Receivable are internal


All those with active roles seek relationships. (an active
govt

employee
will seek a new
govt

employer)


Both active and passive agents have thresholds for initiating or
accepting a partnership.


There are distance thresholds associated with the role choice, that
make them have to match by a certain amount or no partner will be
chosen.


As required, agents update their strategies (based on experience, traits,
and evaluation of past strategies) to maximize utility.

Experiment

36

Social Impact Model

Experiment: Stuck in Stealing Mode

37


Comparison

of

the

evolution of a society which
initiates in a strong
vicious cycle of stealing to one with more moderate levels of stealing.


If they started out stealing excessively, they never learned not to, never
attempted to find service providers who wouldn’t steal from them


If

the stealing is in more moderate amounts, agents learn to find service
providers that do not steal from them within two years


Agents in a stealing vicious cycle never use bribing to accomplish
goals


Agents with moderate stealing used bribes, but after fifteen years,
employers and service providers stopped bribing


Convergence occurs at the fifteen year mark


After
15 years in both the excessive stealing
scenar
-
io

and the
‘normal’ stealing scenario, we see ho
-
mogeneous

responses
across strategies. For in
-
stance, agents generally provided the
same re
-
sponse

for a particular parameter (say ‘Bribe
-
ForServices
’)
after 15 years as opposed to more heterogeneous responses for
the same parameter after two years.


Implication: Diversity of Behavior is needed for flexibility

Summary

38

Social Impact Model

Symbolic
Interactionist

Simulation

39


Sy
mbolic

Interactionist

Simulation is a form of Reinforcement
Learning by
Coevolution
.


Agents

learn associated rules in the form of actions to take with
other agents based on signs displayed and read.


Symbolic

Interactionist

Simulation can be Theoretical (SISTER)
or Data
-
Driven (Nexus).


Symbolic

Interactionist

Simulation can model the motivation
based vicious and virtuous cycles of behavior that determine
social structure

Questions and Comments

40

Social Impact Model

POC: Deborah Duong

dduong@agentBasedLearningSystems.com