IMRC Social Research Methods

ghostslimIA et Robotique

23 févr. 2014 (il y a 3 années et 7 mois)

268 vue(s)

Innovative Design & Manufacturing Research Centre

University of Bath


World
-
leading research in engineering
design and manufacture.”

I
d
MRC Social Research Methods

Autumn Lecture
-
Workshop Series


Science


Aim?


When is knowledge
scientific
knowledge?


Criteria?


Knowledge sources?


When is research
scientific
research?

Henri Christiaans

Science


Realism


What we observe is real



Instrumentalism


What we observe doesn’t need to be real



Social constructivism


Theories only get meaning through social and political context

What is Knowledge?


Justified true belief (Plato’s Theaetetus)


The Greeks classify knowledge into 2 types:


Doxa (believed to be true)


Episteme (known to be true)


Doxa


Epistime


Through Scientific process of inquiry


How do we know what we know?


Define knowledge alternatively


Supported by evidence (usually empirical)


Conceive knowledge claims in a probabilistic sense


Knowledge is a matter of societal acceptance


How is Knowledge Acquired?


Role of science, where science is a convention, related to
societal norms, expectations, values, etc.


Thus, is science equals any scholarly attempt at acquiring
knowledge


Science requires conventions to be followed

How is Knowledge Acquired?


Role of science, where science is a convention, related to
societal norms, expectations, values, etc.


Thus, is science equals any scholarly attempt at acquiring
knowledge


Science requires conventions to be followed

Knowledge in design


Implicit prioritisation of the (language
-
based mode of)
propositional knowledge (justified true beliefs) seems to
exclude certain kinds or formats of knowledge associated
with practice, which are often called practical, experiential,
personal, or tacit knowledge and which evade verbal
articulation.

Knowledge sources


Observation


Experiments/measurements


The Reason


Mathematics/logical reasoning


Intuition


Authority


(Divine) Revelation

Science based on empirism

Empirism:

Knowledge derived from how the world is experienced.
Scientific statements are controlled by and derived from our
experiences and observations. en


Scientific theories

developed and tested by experiments and
observations through empirical methods

Questions to be asked

1.
Which methods do we plan to use?

2.
Which methodology defines the use of methods?

3.
Which theoretical perspective do we start from in order to
apply the right methodology?

4.
Which epistemology feeds this theoretical perspective?

Ontology

1. A systematic account of Existence. Nature of the world
around us.

2. (From philosophy) An explicit formal specification of how to
represent the objects, concepts and other entities that are
assumed to exist in some area of interest and the
relationships that hold among them.

3. The hierarchical structuring of knowledge about things by
subcategorising them according to their essential (or at least
relevant and/or cognitive) qualities.

Epistemology and ontology

The way of understanding and interpreting how we
know what we know.


Particular methodologies tend to entail (subscribe to)
particular epistemologies and, in their turn, particular
forms of ontology




Ontology in Computing Terms


For
AI

systems, what "exists" is that which can be
represented.


We can describe the ontology of a program by defining a set
of representational terms. Definitions associate the names of
entities in the
universe of discourse

(e.g. classes, relations,
functions or other objects) with human
-
readable text
describing what the names mean, and formal
axioms

that
constrain the interpretation and well
-
formed use of these
terms. Formally, an ontology is the statement of a
logical
theory
.


A set of
agents

that share the same ontology will be able to
communicate about a domain of discourse without
necessarily operating on a globally shared theory. The idea of
ontological commitment is based on the
Knowledge
-
Level

perspective.

Epistemology


From the Greek words episteme (knowledge) and logos
(word/speech) is the branch of
philosophy

that deals with the
nature, origin and scope of knowledge.


Refers to our theory of knowledge, in particular, how we
acquire knowledge (Hirschheim, 1992).

Research background

Epistemology


objectivism

subjectivism


Theoretical
perspective

positivism

Interpretativism

symbolic
interactionism

phenomenology

hermeneutics

feminism

(post)modernism

Social
-
constructivism

Methodology


experimental

descriptive

survey

ethnography

heuristic

action research

discourse anal.

evaluation


Methods


scaling

questionnaires

observation

interview

focus group

case study

narratives

ethnographic

stat analysis

data reduction

cognitive mapping

interpretative meth

document analysis

content analysis

conversation anal.

Crotty, 1998

Research background

Epistemology


objectivism

subjectivism


Theoretical
perspective

positivism

Interpretativism

symbolic
interactionism

phenomenology

hermeneutics

feminism

(post)modernism

Social
-
constructivism


Methodology


experimental

descriptive

survey

ethnography

heuristic

action research

discourse anal.

evaluation


Methods


scaling

questionnaires

observation

interview

focus group

case study

narratives

ethnographic

stat analysis

data reduction

cognitive mapping

interpretative meth

document analysis

content analysis

conversation anal.

Theoretical perspective

Philosophical point of view which feeds the methodology and
offers a context for the process and the logics, and gives our
criteria a basis.


Cultural differences play a role


Research background

Epistemology


objectivism

subjectivism


Theoretical
perspective

positivism

Interpretivism

symbolic
interactionism

phenomenology

hermeneutics

feminism

(post)modernism

Social
-
constructivism

Methodology


experimental

descriptive

survey

ethnography

heuristic

action research

discourse anal.

evaluation


Methods


scaling

questionnaires

observation

interview

focus group

case study

narratives

ethnographic

statistical analysis

data reduction

cognitive mapping

interpretative meth

document analysis

content analysis

conversation anal.

Crotty, 1998

Three Main Epistemologies

Positivist

Interpretivist

Critical

Interpretivism

Interpretivism rests upon idealism:



the world is interpreted through the mind; e.g., classificatory
schemes of species;


the social world cannot be described without investigating how
people use language and symbols to construct what social practices;
i.e., understand their experience;


the social world becomes the creation of the purposeful actions of
conscious agents; and


no social explanation was complete unless it could adequately
describe the role of meanings in human actions


Actions are not governed by discrete patterns of cause and effect
(as in positivism), but by rules that social actors use to interpret the
world

Positivist Science


5 Pillars


Unity of scientific method


Causal Relationships


Empiricism


Science and its process is Value
-
Free


Foundation of science is based on logic and maths

Ontology of Positivism


Realism


Universe comprised of objectively given, immutable
objects and structures, existing as empirical
entities, on their own, independent of the
observer’s appreciation of them.


Contrasts with relativism or instrumentalism, where
reality is a subjective construction of the mind, thus
varying with different languages and cultures.


While hugely successful in physical sciences, it is
not as successful for social science.

Anti
-
Positivism


Latter part of 19
th

century


Man as an actor could not be studied through the methods of
natural sciences that focus on establishing general laws. In
the cultural sphere man is free (Burrell and Morgan, 1979)

Post
-
Positivism


Based on the concept of
critical realism
, that
there is a real world out there independent of our
perception of it and that the objective of science is
to try and understand it,


combined with
triangulation
, i.e., the recognition
that observations and measurements are
inherently imperfect and hence the need to
measure phenomena in many ways.


The post
-
positivist epistemology regards the
acquisition of knowledge as a process that is more
than mere deduction. Knowledge is acquired
through both deduction and induction.

23
-
2
-
2014

Objective Analysis

design
Problem

design
Solution

Designer

objective Analysis

Rational Solving Problem Paradigm

Design Task

(= problem +
situation+ teime)

design
Solution

Designer

subjective Interpretation

Reflection in Action Paradigm

POSITIVISM

PHENOMENOLOGY

Constructivist Root

Rationalist Root

Simon


versus



Schon

Methodology

Our strategy and action plans, the design process which
defines what specific methods we will choose

Research background

Epistemology


objectivism

subjectivism


Theoretical
perspective

positivism

Interpretativism

symbolic
interactionism

phenomenology

hermeneutics

feminism

(post)modernism

Social
-
constructivism

Methodology


experimental

descriptive

survey

ethnography

heuristic

action research

discourse anal.

evaluation


Methods


scaling

questionnaires

observation

interview

focus group

case study

narratives

ethnographic

statistical analysis

data reduction

cognitive mapping

interpretative meth

document analysis

content analysis

conversation anal.

Crotty, 1998

Types of Research

Analytical


Historical



Philosophical




Literature study


Meta
-
analysis

-------------------------------------------------------------------------------------------------------

Descriptive


Survey

(questionnaire, interview)




Case study


Task analysis




Document analysis


Correlation anal.




Observation


Etnographics

-------------------------------------------------------------------------------------------------------

Explorative


Survey



Correlational




Case study


Experimental

-------------------------------------------------------------------------------------------------------

Experimental


Pre
-
experimental




True
-
experimental




Quasi
-
experimental

-------------------------------------------------------------------------------------------------------





Ethnographics

Types of research methods

idiographic


nomothetic

rational

empirical

unbiased


participatory

qualitative


quantitative


inductive


deductive


prescriptive

descriptive


knowledge problem

Fundamental Research: the Empirical cycle

‘t Hart c.s.

testing

deduction

prediction

theory

generalising

modelling

Explaining/

interpreting

modelling

testing

evaluating

describing/

interpreting

evaluation

specifying

induction

hypotheses

Practice oriented Research: The regulative cycle

problem from practice

evaluation

intervention

plan

(problem solving)

generalising

modelling

designing

deciding

process evaluation

describing/

interpreting

action
-
process supporting

observing

evaluating

diagnosis

‘t Hart c.s.

Method

The technique to gather data, related to the research question.

Research background

Epistemology


objectivism

subjectivism


Theoretical
perspective

positivism

Interpretativism

symbolic
interactionism

phenomenology

hermeneutics

feminism

(post)modernism


Methodology


experimental

descriptive

survey

ethnography

heuristic

action research

discourse anal.

evaluation


Methods


scaling

questionnaires

observation

interview

focus group

case study

narratives

ethnographic

stat analysis

data reduction

cognitive mapping

interpretative meth

document analysis

content analysis

conversation anal.

Crotty, 1998

35

Qualitative Positivist Research versus
Non
-
Qualitative Positivist Research

QPR Methods

Non
-
QPR Methods

Field experiment

Math Modeling

(analytical
modeling)

Lab experiment

Group feedback

Free simulation experiment

Participative research

Experimental simulation

Case study

Adaptive experiment

Philosophical research

Field study



Opinion research



Archival research



Table 1. QPR versus Non
-
QPR Methods (Click on the method for its definition)

Type of Research, General Research Approaches,

Data Collection Techniques, & Data Analysis Techniques

Research
-
Led

Design
-
Led

Participatory

Design

generative

tools

Design and
Emotion

Critical

Design

User
-
centered

Design

contextual
enquiry

Lead
-
user
inovation

applied

ethnography

Usability
testing

Human factors
and ergonomics

Dutch/Scandi
navian design

Research
-
Led

Participatory mindset

Expert mindset

Probes

Sanders, 2002

Design
-
Led

Research
-
Led

Design
-
Led

Participatory

Design

generative

tools

Design and
Emotion

Critical

Design

User
-
centered

Design

contextual
enquiry

Lead
-
user
inovation

applied

ethnography

Usability
testing

Human factors
and ergonomics

Dutch/Scandi
navian design

Research
-
Led

Participatory mindset

Expert mindset

Probes

Sanders, 2002

Design
-
Led

Research background

Epistemology


objectivism

subjectivism


Theoretical
perspective

positivism

Interpretativism

symbolic
interactionism

phenomenology

hermeneutics

feminism

(post)modernism


Methodology


experimental

descriptive

survey

ethnography

heuristic

action research

discourse anal.

evaluation


Methods


scaling

questionnaires

observation

interview

focus group

case study

narratives

ethnographic

statistic. analysis

data reduction

cognitive mapping

interpretative meth

document analysis

content analysis

conversation anal.

Crotty, 1998

Definitions


‘Research’ = the systematic inquiry to the end of gaining new
knowledge


a ‘researcher’ = a person who pursues research (e.g., in
design).


Practice’ = professional practice (e.g., in design) or to
processes usually used in professional practice to produce
professional work for any purpose other than the (deliberate)
acquisition of knowledge.


‘Practitioner’ = anyone who works in professional practice.

Process (
design methodology
)

product

people

Design Knowledge

designers

Design knowledge


Design knowledge resides firstly in
people
: in designers especially.
Therefore, we study human ability
-

of how people design. This
suggests, for example, empirical studies of design behaviour, but it
also includes theoretical deliberation and reflection on the nature of
design ability. It also relates strongly to considerations of how people
learn to design

Design knowledge


Design knowledge resides firstly in
people
: in designers especially.
Therefore, we study of human ability
-

of how people design. This
suggests, for example, empirical studies of design behaviour, but it
also includes theoretical deliberation and reflection on the nature of
design ability. It also relates strongly to considerations of how people
learn to design.


Design knowledge resides secondly in
processes
: in the tactics and
strategies of designing. A major area of design research is
methodology: the study of the processes of design, and the
development and application of techniques which aid the designer.


Design knowledge


Design knowledge resides firstly in
people
: in designers especially.
Therefore, we study of human ability
-

of how people design. This
suggests, for example, empirical studies of design behaviour, but it
also includes theoretical deliberation and reflection on the nature of
design ability. It also relates strongly to considerations of how people
learn to design


Design knowledge resides secondly in
processes
: in the tactics and
strategies of designing. A major area of design research is
methodology: the study of the processes of design, and the
development and application of techniques which aid the designer.


The
product

dimension asks for forms and materials, and finishes
with the embodiment of design attributes: both the intentional world
(teleological and functional

wishes and needs

) in relation with the
principal, partial and elementary function and the man’s connection
with the systemic formal and material part (structure, organization,
parts and connections).

Scientific

Interpretive

Theoretical
perspective


Scientific, usually based on physics

Interpretive, focusing on
individuals’ experiences, their
construction of understanding,
perceptions and interpretation of
reality. Often centres on
individual creativity and
subjective perceptions relating to
being creative.

Focus

Empirical realities of the design
processes, design objects, design
brief and contexts.

The core concept of ‘design’ is
defined in terms of these activities.


Experiences of designers and
other design constituents. Tries
to identify form of internal
creative design activities from
observation of externalities.

Typically defines design in terms
of creativity, art, individual genius
and socio
-
cultural influences

Design Research

Scientific

Interpretive

View of Design

Design is a process.

May or may not include
creativity.

Intuitive, involving hidden aspects of
human subjective thinking and
affective activity.

View of
creativity

‘Something, or a
specification for something,
is “created”’.


Creation can be achieved
mechanically, by automation
or intuitively.

Human internal activity that results in
ideas for new, unusual, highly valued,
never before created things,
emerging ‘magically’ from the genius
of designers.

Focus on ‘individual creativity’
attributed to specific ‘designers’ and
socio
-
cultural influences.


Design Research

Scientific

Interpretive

Data collection

Similar to physics and
natural sciences.

Drawn from various qualitative
traditions, e.g. anthropology,
ethnography, history, includes self
reporting data collection.

Analysis
methods

Similar to physics and
natural sciences.


Drawn from various qualitative
traditions, e.g. anthropology,
ethnography, history, includes
reflective analysis of self reports and
self perception.

Knowledge
focus

Discipline specific empirical
information (along with)

elicited representations of
tacit information and data
that designers
use
.

Tacit and embodied skills of designers
and users.

Culturally
-
determined knowledge.

Embedded meanings.

Design Research

Scientific

Interpretive

Strengths

1. Techniques to investigate
phenomena in ways that are
transparent, repeatable, testable,
and verifiable.

2. Research methods are
expressed in a formal language
that enables precise critique of
the data collection techniques,
methods of analysis, processes
that lead to abstractions, and the
theory abstractions and
conclusions.

3. Correspondence between
characteristics of phenomena
and the formal defined symbolic
language of concepts and
operations in which
mathematically theories and
representations of the
phenomena are expressed.

1. Focus on human considerations,
such as the human creative aspects of
design, and how users and other
interpret designed outcomes.

2. Interpretive methods give space for
designers and users to explain, in their
own words, and from their own
perspectives, how they design and
use designed outcomes and how they
communicate with others about
designs.

3. Interpretive methods also allow
exploration of opinions of users about
cultural aspects of particular designs.

4. The interpretive approach can be
extended to draw strength from the
use of large data sets by which
correlations and measures of
confidence in them can be established
between individuals’ ‘stories’ and the
phenomena being studied.

Scientific

Interpretive

Weaknesses

Scientific empirical method does
not adequately address human
subjective, interpretive and
experiential phenomena except
via physiological substrates.

Main weakness is lack of reliability of
individuals’ evidence, perceptions and
interpretations i.e. lack of correlation
between what people say and reality.

Evidence of this problem in studies of
e.g. witness testimony, reliability of
memory, relationships between
reported thoughts and physiological
evidence, influence of subconscious
‘thinking’, mental illusions and
delusions in normal people.

‘False consciousness’: people’s
representations of themselves are
inaccurate or simply wrong.

Extends to individuals descriptions of
processes, and the social activities
that they undertake.

Scientific

Interpretive

Contradictions

There is an incompatibility
between scientific modelling of
design process and inclusion of
a process element ‘create a
new solution’ as a subjective
human activity.


Claims that all sub
-
fields of
design are incommensurate as
they use different knowledge
(and that the broader field of
design is fundamentally
fragmented) is at odds with
scientific representation of
designers working across
disciplines and in multi
-
cross
-

and trans
-
disciplinary teams.

There is tension between interpretive
approaches that focus on experiential
subjective phenomenological aspects
of human creative design activity and
the frequent shift of emphasis onto
aspects of design and creative activity
that are more accessible empirically
using a physical science approach.


There is an epistemological
inconsistency in claims that Design
exists of itself as a phenomena
capable of creative agency and action.

Love’s proposal:

a unified basis for design theory bridging these two
incompatible approaches.


Advantages


It provides a coherent epistemological basis for new theories


It recasts prior research and theory within a justified
integrated framework with a clear epistemology and ontology.


This in turn provides the basis for developing a design field.


Design Research



Designs (i.e. the specification for creating or doing something)


Designed outcomes (after they are manufactured/actualised)


Design activity


Design processes


The skills of designers


The role of design activity


Cognitive design processes


Behaviour of designers as individuals and in social groups


Combinations of the above


Foundations for a unified basis

Epistemologies Assumptions for
Qualitative and Quantitative Research

Researcher tests or verifies a theory

Researcher tests hypotheses

or research questions

Researcher defines and operationalizes

variables derived from the theory

Researcher measures or observes

variables using an instrument

to obtain scores

Creswell, 2003

Deductive logic of quantitative research

Generalizations or theories

to past experiences and literature

Researcher looks for broad patterns.

Generalizations or Theories from

Themes or Categories

Researcher analyze data to

form themes or categories

Researcher asks open
-
ended questions

of participants or records field
-
notes

Researcher gathers information

e.g. interviews, observations

Creswell, 2003

Inductive logic of qualitative research

Qualitative vs Quantitative

Purpose




Perspective


Procedures



Quantitative

General Laws

Test Hypotheses

Predict behavior


Outsider
-
Objective


Structured


formal measures


probability samples


statistical analysis

Qualitative

Unique/Individual case

Understanding

Meanings/Intentions


Insider
-
Subjective


Unstructured


open ended measures


judgement samples


interpretation of data

Qualitative Research

Triangulation

By using several data collecting methods


field
notes, interviews, narratives


a complete picture
of the phenomenon can be provided

Interpretation:

observation of species


-

-

-

-

-

-

-

-

-

-

-

-

-

-

a

Interpretation


-

-

-

-

-

-

-

-

-

-

-

-

-

-

-

-

-

-

-

-

-

-

-

-

-

-

-

-

-

-

-

-

-

-

-

-

-

-

-

-

-

-

-

-

-

-

-

-

-

-

a

b

c

d

Interpretation:

observation of discourse

J
(reading) pack is firmly attached to the bike positioning of the
backpack was alright fact that the centre of gravity of the
backpack is placed rather far to the back of the bike (inaudible)

I

do we have any … em...

J

there's a problem with potholes .. the backpack tends to slide up
and down which adversely influences stability I guess when you
hit bumps

I

isn't that in the negative?

J

mm yeah well the product was considered ugly well that's
solvable (laughter) we can fix that one if nothing else ... it takes a
while to get used to cycling with this weight; mistakes are made
attaching the fastening device to the bike so it has to be easy to
attach

K

with only one yeah gotta be fool proof so that's part of our

J

yeah that should be in our spec

K

functional spec

The role of interpretation

Gap between objects and our representations, in
3 forms (
'methodological horrors'
, Woolgar '88)
:


1. Indexicality

2. Inconcludability

3. Reflexivity


THANK YOU!