The Human Performance Assessment and Modeling (HPAM) Laboratory

geographertonguesAI and Robotics

Nov 30, 2013 (3 years and 10 months ago)

95 views

Human Performance Assessment and Modeling Laboratory

The Harold & Inge Marcus Department of Industrial and Manufacturing Engineering

1

The Human Performance
Assessment and Modeling (HPAM)
Laboratory

Director: Ling Rothrock, Ph.D

Associate Professor,

The Harold and Inge Marcus Department of Industrial and Manufacturing Engineering

The Pennsylvania State University

210 Leonhard Building

University Park, PA 16802

Phone: 814
-
865
-
7241

Email: lxr28@engr.psu.edu

Human Performance Assessment and Modeling Laboratory

The Harold & Inge Marcus Department of Industrial and Manufacturing Engineering

2

Laboratory Mission

To form a mutually
-
supporting triad
with
human
-
in
-
the
-
loop simulations




informing the development of
inductive learning models

and



testing hypotheses grounded in

cognitive science


Human Performance Assessment and Modeling Laboratory

The Harold & Inge Marcus Department of Industrial and Manufacturing Engineering

3

Investigating Team Performance Using a Human
-
in
-
the
-
loop Discrete
-
Event Simulation System

Construction and utilization of the Team Aegis Simulation
Platform (TASP) to investigate Modeling for Effective Team
Training (METT)


Sponsor: US Navy

Human Performance Assessment and Modeling Laboratory

The Harold & Inge Marcus Department of Industrial and Manufacturing Engineering

4

HITL Test Bed

TASP

Roles

Interaction

Scenario

Pool

User

Model

Hard

Hard

Hard

Avg

Avg

Avg

Easy

Easy

Easy

Scenarios

Performance

Human Performance Assessment and Modeling Laboratory

The Harold & Inge Marcus Department of Industrial and Manufacturing Engineering

5

Investigating Compensatory and Noncompensatory
Decision Making Strategies in Dynamic Task
Environments

Research Team

Ling Rothrock, Yin Jing (Penn State)

Alex Kirlik (UIUC)


Sponsor: NSF DRMS

Human Performance Assessment and Modeling Laboratory

The Harold & Inge Marcus Department of Industrial and Manufacturing Engineering

6

Compensatory Lens Model (CLM)

Rule
-
based Lens Model (RLM)

Systematic Analysis of Operator Strategies in
Missile Defense: Implications toward Visualization
Design

Sponsor: Alion Science and Technology

(MicroAnalysis and Design)

Human Performance Assessment and Modeling Laboratory

The Harold & Inge Marcus Department of Industrial and Manufacturing Engineering

7

Human Performance Assessment and Modeling Laboratory

The Harold & Inge Marcus Department of Industrial and Manufacturing Engineering

8


Experiment 1

Experiment 2

Experiment 3

Task

Binary choice

Pairwise choice

Interactive Simulation

Task Surface Problem

Indicate preference of
missile defense plans in
terms of lives

Indicate preference

of
missile defense plans in
terms of GBIs

Create missile defense
plan in terms of GBIs

Task Depth Problem

binary

comparison of
expected lives lost and
saved

pairwise comparison of
expected lives lost and
saved based on static GBI
allocation

multiple co
mparison of
expected lives lost and
saved based on dynamic
GBI allocation

Performance measure

Lives lost

Lives Saved

Preference score

Rank correlation


coefficient

Population lost score

GBI allocation score

Impact of Positive
Framing

on Task
Performance

Promotes risk averse
behavior

(confirm H
1
)

Increases preference
score and rank
correlation coefficient
(confirm H
2
)

No significant impact on
performance

(disconfirm H
3
)

(confirm H
4
)

Impact of Negative
Framing

on Task
Performance

Promotes risk seeking
be
havior


(confirm H
1
)

Decreases preference
score and rank
correlation coefficient
(confirm H
2
)


Question 1. The country is currently under attack and the following two
defense plans have been proposed:


If Plan A is adopted,


there is a 0.33 probability that 4,600,000 people will be saved,


there is a 0.66 probability that 4,400,000 people will be saved, and


there is a 0.01 probability that no one will be saved


If Plan B is adopted,


4,400,000 people will be saved


Experiment 1 Format

Experiment 2 Format

Experiment 3 Format

Development of VHMS Prognostic Capability for
the Heavy Brigade Combat Team (ongoing)

Sponsor: US Army

Human Performance Assessment and Modeling Laboratory

The Harold & Inge Marcus Department of Industrial and Manufacturing Engineering

9

Investigating Refinery Control Room Display
Visualizations (ongoing)

Sponsor: Abnormal Situation Management Consortium
(Honeywell)

Human Performance Assessment and Modeling Laboratory

The Harold & Inge Marcus Department of Industrial and Manufacturing Engineering

11

Heat Map

Visual Thesaurus

Parallel Coordinates

Publications

Altuntas
, B., Wysk, R. A., & Rothrock, L. (2004).
A Formal Approach to Include A Human Material Handler in A
CIM System. Paper presented at the Institute of Industrial Engineers Annual Conference, Houston,
TX.


Altuntas
, B., Wysk, R. A., & Rothrock, L. (2007). A Formal Approach to Include a Human Material Handler in a
CIM System.
International Journal of Production Research, 45(9), 1953
-
1971.


Barron
, K., Simpson, T. W., Rothrock, L.,
Frecker
, M., Barton, R. R., &
Ligetti
, C. (2004, Sep 28
-
Oct 2 2004).
Graphical user interfaces for engineering design: Impact of response delay and training on user performance.
Paper presented at the 2004 ASME Design Engineering Technical Conferences and Computers and Information in
Engineering Conference, Salt Lake City, UT, United States.



Bejarano
, D., Rothrock, L., & Warner, J. (2001, August 10, 2001).
Using Time Windows to Aid Situation
Comprehension in a Highly Automated Control Environment. Paper presented at the 9th International Conference
on Human
-
Computer Interaction, New Orleans, LA.


Bhandarkar
, D.,
Thiruvengada
, H., Rothrock, L., Campbell, G. E., & Bolton, A. E. (2004, May 15
-
18, 2004).
TASP: A Toolkit to Analyze Team Performance in a Complex Task Environment. Paper presented at the Annual
Conference of the Institute of Industrial Engineers, Houston, TX.


Bolton
, A. E., Campbell, G. E., Buff, W. L., Rothrock, L.,
Bhandarkar
, D., &
Kukreja
, U. (2004).
TASP: A
Flexible Alternative for Team Training and Performance Research. Paper presented at the
Interservice
/Industry
Training, Simulation & Education Conference (I/ITSEC), Orlando, FL
.

Human Performance Assessment and Modeling Laboratory

The Harold & Inge Marcus Department of Industrial and Manufacturing Engineering

13

Publications

(cont.)

Harvey, C. M., Daigle, T., Darisipudi, A., Rothrock, L., & Nabatilan, L. (2006). Multidisciplinary Systems Teams.
In
Handbook of Industrial and Systems Engineering (pp. 11
-
11 to 11
-
19). Boca Raton, FL: CRC Press.


Harvey, C. M., Koubek, R. J., Rothrock, L., Darisipudi, A., Kim, J. W., & Munch, J. (2005). Cognitive
Ergonomics. In R. Proctor & K.
-
P. L. Vu (Eds.),
Handbook of Human Factors in Web Design (pp. 70
-
89).
Mahwah, NJ: Lawrence Erlbaum Associates.


Hodge, K. A., Rothrock, L., Kirlik, A. C., Walker, N., Fisk, A. D., Phipps, D. A., et al. (1995).
Training for
Tactical Decision Making Under Stress: Towards Automatization of Component Skills (Human Attention
Performance Laboratory Technical Report
-
9501: Georgia Institute of Technology, School of Psychology.



Hodge, K. A., Rothrock, L., Krosnick, W., Nagel, K. L., Kirlik, A. C., Walker, N., et al. (1993).
Training for
Tactical Decision Making Under Stress: An Empirical Investigation of Part
-
Task Training for Perceptual and
Rule
-
based Decision Making (Human Attention Performance Laboratory Technical Report
-
9301: Georgia
Institute of Technology, School of Psychology.


Jin, J., & Rothrock, L. (2005).
A Visualization Framework for Bounding Physical Activities
--

Towards a
Quantification of Gibsonian
-
based Fields. Paper presented at the 49th Annual Meeting of the Human Factors and
Ergonomics Society, Orlando, Florida.

Human Performance Assessment and Modeling Laboratory

The Harold & Inge Marcus Department of Industrial and Manufacturing Engineering

14

Publications
(cont.)

Jin, J., Rothrock, L., McDermott, P., & Barnes, M. (2010). Using Analytic Hierarchy Process to Examine
Judgment Consistency in a Complex Multi
-
Attribute Task.
IEEE Transactions on Systems, Man and Cybernetics,
Part A, 40(5), 1105
-
1115.


Kim, N., Rothrock, L., Wysk, R. A., & Joo, J. (2010, December 5
-
8).
An Affordance
-
based Formalism for
Modeling Human
-
involvement in Complex Systems for Prospective Control. Paper presented at the Winter
Simulation Conference, Baltimore, Maryland.


Kim, N., Shin, D., Wysk, R. A., & Rothrock, L. (2008, May 18
-
20, 2008).
The Hierarchical Modeling of Human
Planning Activities with Affordances Using Finite State Automata (FSA). Paper presented at the Annual
Conference of the Institute of Industrial Engineers, Vancouver, BC.


Kim, N., Shin, D., Wysk, R. A., & Rothrock, L. (2009). Using finite state automata (FSA) for formal modelling of
affordances in human
-
machine cooperative manufacturing systems.
International Journal of Production Research,
48(5), 1303
-
1320.


Kirlik, A., Fisk, A. D., Walker, N., & Rothrock, L. (1998). Feedback Augmentation and Part
-
Task Practice in
Training Dynamic Decision
-
Making Skills. In J. A. Cannon
-
Bowers & E. Salas (Eds.),
Making Decisions Under
Stress: Implications for Individual and Team Training (pp. 91
-
113). Washington, D.C.: American Psychological
Association.

Human Performance Assessment and Modeling Laboratory

The Harold & Inge Marcus Department of Industrial and Manufacturing Engineering

15

Publications
(cont.)

Ma
, J., Kim, N., & Rothrock, L. (2011). Performance Assessment in an Interactive Call Center Workforce
Simulation.
Simulation
Modelling

Practice and Theory, 19, 227
-
238.


McDermott
, P., Hutchins, S., Barnes, M.,
Koenecke
, C.,
Gillan
, D., & Rothrock, L. (2003).
The Presentation of
Risk and Uncertainty in the Context of National Missile Defense Simulations. Paper presented at the Human
Factors and Ergonomics Society 47th Annual Conference, Denver, Colorado.


Munch
, J., & Rothrock, L. (2003).
Modeling Human Performance in Supervisory Control: Informing Adaptive Aid
Design. Paper presented at the Annual Conference of the Institute of Industrial Engineers, Portland, OR.


Oyewole
, S. A., Haight, J. M., Freivalds, A., Cannon, D. J., & Rothrock, L. (2010). Statistical evaluation and
analysis of safety intervention in the determination of an effective resource allocation strategy.
Journal of Loss
Prevention in the Process Industries, 23(5), 585
-
593.


Park
, S., Bolton, G. E., Rothrock, L., &
Brosig
, J. (2010). Toward an Interdisciplinary Perspective of Training
Intervention for Negotiations: Develop Strategic Negotiation Support Contents.
Decision Support Systems, 49,
213
-
221.


Park
, S., & Rothrock, L. (2007). Systematic Analysis of Framing Bias in Missile Defense: Implications toward
Visualization Design.
European Journal of Operational Research, 182, 1383
-
1398
.

Human Performance Assessment and Modeling Laboratory

The Harold & Inge Marcus Department of Industrial and Manufacturing Engineering

16

Publications
(cont.)

Repperger, D. W., & Rothrock, L. (2002).
A Dual Haptic Interface Investigation for Improved Human
-
Computer
Interaction (Technical Report AFRL
-
HE
-
WP
-
TR
-
2002
-
0189). Wright
-
Patterson AFB, OH: United States Air
Force Research Laboratory.


Repperger, D. W., & Rothrock, L. (2002, March 5, 2002).
Using Genetic Algorithms to Help Design Human
Machine Interface Systems. Paper presented at the Dayton
-
Cincinnati Aerospace Science Symposium, Dayton,
OH.


Repperger, D. W., & Rothrock, L. (2005). Using GA
-
based Intelligent Control Means to Enhance Human
-
Machine Interfaces.
The International Journal of Intelligent Automation and Soft Computing (AutoSoft), 11(2),
123
-
140.


Rothrock, L. (1999).
Genetics
-
Based Inductive Inference Model to Represent Human Decision Strategies in a
Supervisory Control Task. Paper presented at the Artificial Neural Networks in Engineering Conference, St.
Louis, MO.


Rothrock, L. (2000).
Developing Performance Metrics in Pre
-
Deployment Systems Using the Abstraction
Hierarchy. Paper presented at the Military, Government and Aerospace Simulation Symposium of the 2000
Advanced Simulation Technologies Conference, Washington, D.C.


Human Performance Assessment and Modeling Laboratory

The Harold & Inge Marcus Department of Industrial and Manufacturing Engineering

17

Publications
(cont.)

Rothrock, L. (2000).
Domain
-
relevant Performance Analysis in Human
-
in
-
the
-
loop Discrete
-
event Simulations.
Paper presented at the 2000 Summer Computer Simulation Conference, Vancouver, B.C.


Rothrock, L. (2001). Using Time Windows to Evaluate Operator Performance.
International Journal of Cognitive
Ergonomics, 5(1), 1
-
21.


Rothrock, L. (2002, 10 April 2002).
Adaptive Aiding Using Physiological Operator Functional State Assessment.
Paper presented at the Third Annual Joint AFRL/DAGSI Research Symposium, Dayton, OH.


Rothrock, L., Barron, K., Simpson, T. W., Frecker, M., Ligetti, C., & Barton, R. R. (2004, Sept. 20
-
24).
Applying
the Proximity Compatibility and the Control
-
Display Compatibility Principles toward Engineering Design Using
Configural Interfaces. Paper presented at the 48th Annual Meeting of the Human Factors and Ergonomics
Society, New Orleans, LA.


Rothrock, L., Barron, K., Simpson, T. W., Frecker, M., Ligetti, C., & Barton, R. R. (2005). Applying the proximity
compatibility and the control
-
display compatibility principles to engineering design interfaces.
Journal of Human
Factors and Ergonomics in Manufacturing, 16(1), 61
-
81.


Rothrock, L., Cohen, A., Yin, J., Thiruvengada, H., & Nahum
-
Shani, I. (2009). Analyses of Team Performance in
a Dynamic Task Environment.
Applied Ergonomics, 40(4), 699
-
706.

Human Performance Assessment and Modeling Laboratory

The Harold & Inge Marcus Department of Industrial and Manufacturing Engineering

18

Publications
(cont.)

Rothrock, L., & Harvey, C. (2002, 27
-
31 January, 2002).
Synthetic Task Environments for the study of
collaboration in supervisory control teams. Paper presented at the 2002 Western Multiconference, Collaborative
Technologies Symposium, San Antonio, TX.


Rothrock, L., Harvey, C. M., & Burns, J. (2005). A Theoretical Framework and Quantitative Architecture to
Assess Team Task Complexity in Dynamic Environments.
Theoretical Issues in Ergonomics Science, 6(2), 157
-
172.


Rothrock, L., & Kirlik, A. (2000).
The Use of Inductive Inference Models to Understand Human Performance in
Supervisory Control Domains. Paper presented at the 2000 IEEE International Conference on Systems, Man, and
Cybernetics, Nashville, TN.


Rothrock, L., & Kirlik, A. (2003, June 22
-
27).
Explorations in Modeling Human Decision Making in Dynamic
Contexts. Paper presented at the 5th International Conference on Engineering Psychology and Cognitive
Ergonomics, Crete, Greece.


Rothrock, L., & Kirlik, A. (2003). Inferring rule
-
based strategies in dynamic judgment tasks: toward a
noncompensatory formulation of the lens model.
IEEE Transactions on Systems, Man, and Cybernetics, Part A,
33(1), 58
-
72.

Human Performance Assessment and Modeling Laboratory

The Harold & Inge Marcus Department of Industrial and Manufacturing Engineering

19

Publications
(cont.)

Rothrock, L., & Kirlik, A. (2006). Inferring Fast and Frugal Heuristics from Human Judgment Data. In A. Kirlik
(Ed.),
Adaptive Perspectives on Human
-
Technology Interaction: Methods and Models for Cognitive Engineering
and Human
-
Computer Interaction (pp. 131
-
148). Oxford: Oxford University Press.


Rothrock, L., Koubek, R., Fuchs, F., Haas, M., & Salvendy, G. (2002). Review and Reappraisal of Adaptive
Interfaces: Toward Biologically
-
Inspired Paradigms.
Theoretical Issues in Ergonomics Science, 3(1), 47
-
84.


Rothrock, L., & Nembhard, D. A. (2006). Team
-
in
-
the
-
loop Simulations: Advances in the Study of Collaboration
and Conflict. In W. Karwowski (Ed.),
International Encyclopedia of Ergonomics and Human Factors (2nd ed., pp.
2407
-
2414). Boca Raton: CRC/Taylor & Francis.


Rothrock, L., Park, S., Barnes, M., McDermott, P., Hutchins, S., & Gillan, D. (2003).
Systematic Analysis of Risk
Visualization Strategies for Homeland Defense. Paper presented at the IEEE International Conference on
Systems, Man & Cybernetics, Washington, D.C.


Rothrock, L., Prasad, K. S., Harvey, C., & Narayanan, S. (2004). A configurable telerobotics system for human
factors education and research.
International Journal of Modelling and Simulation, 24(4), 239
-
249.


Rothrock, L., & Ramachandran, K. (2002, July 15
-
19, 2001).
Use of a Configurable Telerobotics System to
Measure and Model Human Supervisory Control Performance. Paper presented at the Summer Computer
Simulation Conference 2002, San Diego, CA.

Human Performance Assessment and Modeling Laboratory

The Harold & Inge Marcus Department of Industrial and Manufacturing Engineering

20

Publications
(cont.)

Rothrock, L., Ventura, J., & Park, S. (2003).
An Optimization Methodology to Investigate Operator Impact on
Quality of Service. Paper presented at the Interservice/Industry Training, Simulation, and Education Conference
(I/ITSEC), Orlando, FL.


Rothrock, L., Wysk, R. A., Kim, N., Shin, D., & Son, Y.
-
J. (2011). 20th ICPR: A Formalism for Human
-
Machine
Cooperative Systems.
International Journal of Production Research, 49(14), 4263
-
4273.


Rothrock, L., & Yin, J. (2008). Integrating Compensatory and Noncompensatory Decision Making Strategies in
Dynamic Task Environments. In T. Kugler, J. C. Smith, T. Connolly & Y. J. Son (Eds.),
Decision Modeling and
Behavior in Uncertain and Complex Environments (pp. 125
-
141). New York: Springer.


Shin, D., & Rothrock, L. (2005, August 13
-
15).
A Formal Model of Human
-
Automation Interaction. Paper
presented at the International Workshop on Robot and Human Interactive Communication, Nashville.


Shin, D., Wysk, R. A., & Rothrock, L. (2004, IMECE 2004, Nov. 13
-
19 2004).
Modeling and analysis of human
task
-
performing process in manufacturing systems with human task and error classifications. Paper presented at
the 2004 ASME International Mechanical Engineering Congress and Exposition, Anaheim, CA.


Shin, D., Wysk, R. A., & Rothrock, L. (2006). A Formal Control
-
Theoretic Model of Human
-
Automation
Interactive Manufacturing System Control.
International Journal of Production Research, 44(20), 4273
-
4295.

Human Performance Assessment and Modeling Laboratory

The Harold & Inge Marcus Department of Industrial and Manufacturing Engineering

21

Publications
(cont.)

Shin, D., Wysk, R. A., & Rothrock, L. (2006). Formal Model of Human Material
-
Handling Tasks for Control of
Manufacturing Systems.
IEEE Transactions on Systems, Man and Cybernetics, Part A, 36(4), 685
-
696.


Shin, D., Wysk, R. A., & Rothrock, L. (2006). An Investigation of a Human Material Handler on Part Flow in
Automated Manufacturing.
IEEE Transactions on Systems, Man, and Cybernetics, Part A, 36(1, Special Issue on
Advances in Heterogeneous and Complex System Integration), 123
-
135.


Simpson, T. W., Barron, K., Rothrock, L., Frecker, M., Barton, R. R., & Ligetti, C. (2007). Impact of Response
Delay and Training on User Performance with Text
-
Based and Graphical User Interfaces for Engineering Design.
Research in Engineering Design, 18, 49
-
65.


Simpson, T. W., Frecker, M., Barton, R. R., & Rothrock, L. (2007). Graphical and Text
-
based Design Interfaces
for Parameter Design of an I
-
Beam, Desk Lamp, Aircraft Wing, and Job Shop Manufacturing System.
Engineering
with Computers, 23, 93
-
107.


Simpson, T. W., Iyer, P., Rothrock, L., Frecker, M., Barton, R. R., Barron, K., et al. (2005, April 18
-
21).
Metamodel
-
Driven Interfaces for Engineering Design: Impact of Delay and Problem Size on User Performance.
Paper presented at the 46th AIAA/ASME/ASCE/AHS/ASC Structures, Structural Dynamics & Materials
Conference, Austin, TX.

Human Performance Assessment and Modeling Laboratory

The Harold & Inge Marcus Department of Industrial and Manufacturing Engineering

22

Publications
(cont.)

Thiruvengada, H., Bhandarkar, D., Kukreja, U., & Rothrock, L. (2004, May 17
-
20, 2004).
TASP: A toolkit for
human experimentation in a synthetic environment with a hybrid human
-
agent team. Paper presented at the
Behavior Representation in Modeling and Simulation, Crystal City, VA.


Thiruvengada, H., Bhandarkar, D., & Rothrock, L. (2006).
Time Windows
-
based Team Performance Measures:
Design and Implementation. Paper presented at the 2006 IEEE International Conference on Systems, Man, and
Cybernetics, Taipei, Taiwan.


Thiruvengada, H., Kim, N., Rothrock, L., & Wysk, R. A. (2008, May 18
-
20, 2008).
A Systematic Approach for
Developing a Computable Model for Gibson's Affordance Using Finite State Automata. Paper presented at the
Annual Conference of the Institute of Industrial Engineers, Vancouver, BC.


Thiruvengada, H., & Rothrock, L. (2006, May 21
-
24, 2006).
HCI Implications of a Colored Petri Net Model for
Gibson's Affordance. Paper presented at the Annual Conference of the Institute of Industrial Engineers, Orlando,
FL.


Thiruvengada, H., & Rothrock, L. (2007).
Affordance
-
based Computational Model of Driver Behavior on
Highways: A Colored Petri Net Approach. Paper presented at the 2007 IEEE International Conference on
Systems, Man and Cybernetics, Montreal.

Human Performance Assessment and Modeling Laboratory

The Harold & Inge Marcus Department of Industrial and Manufacturing Engineering

23

Publications
(cont.)

Thiruvengada, H., & Rothrock, L. (2007). Team Training in Complex Environments. In D. A. Nembhard (Ed.),
Workforce Cross Training (pp. 181
-
207). New York: CRC Press.


Thiruvengada, H., & Rothrock, L. (2007). Time Window
-
based Performance Measures: A Framework to Measure
Team Performance in Dynamic Environments.
Cognition, Technology & Work, 9(2), 99
-
108.


Wheatly, M., Narayanan, S., Koubek, R. J., Harvey, C., Rothrock, L., Smith, P., et al. (2003).
Biologically
Inspired Analysis of Complex Systems: Back to Nature. Paper presented at the 2003 Human
-
Computer Interaction
Conference, Crete, Greece.


Yin, J., & Rothrock, L. (2006). A Rule
-
based Lens Model.
International Journal of Industrial Ergonomics, 36(5),
499
-
509.


Yin, J., Rothrock, L., & Kirlik, A. (2005).
Using the Lens Model Framework to Characterize Compensatory and
Noncompensatory Decision Making Strategies. Paper presented at the 11th International Conference on Human
-
Computer Interaction, Las Vegas, NV.

Human Performance Assessment and Modeling Laboratory

The Harold & Inge Marcus Department of Industrial and Manufacturing Engineering

24