Mar13.Research 2.0_1x

darkfryingpanΚινητά – Ασύρματες Τεχνολογίες

10 Δεκ 2013 (πριν από 3 χρόνια και 4 μήνες)

121 εμφανίσεις

March 13, 2012: I. Sim

Tech Research Methods

Epi


206 Medical Informatics

Technology Mediated Research
Methods

Ida Sim, MD, PhD


March 13, 2012



Division of General Internal Medicine, and

Graduate Group in Biological and Medical Informatics

UCSF

Copyright Ida Sim, 2012. All federal and state rights reserved for all original material presented in this course
through any medium, including lecture or print.

March 6, 2012: I. Sim

What Next?

Epi 206


Medical Informatics

Outline


So you
wanna

build an app…


app design and development


validation methodology


Beyond the hype


Class summary

March 6, 2012: I. Sim

What Next?

Epi 206


Medical Informatics

Development Decisions


The “specs” (specifications)


what is your system supposed to do? for whom?


what specific functions will it have?


do use case scenarios (step
-
by
-
step, storyboards,
etc
)


Basic technical choices



form factor

: kiosk, desktop, laptop, notebook, tablet, phone…


operating
system (e.g.,
Mac
vs

PC, or Android
vs

iPhone
vs

mobile
web)


check
some browser and platform usage statistics sources


Find a developer


internal: UCSF Mobile Services, DMG, see
mHealth Innovators
Forum


external: contact
Tuhin.Sinha@ucsf.edu

in ITA for referrals and
standard contracts


March 6, 2012: I. Sim

What Next?

Epi 206


Medical Informatics

Agile Design


Traditional approach


design a website or system; pilot
it on a few users,
improve it;
run RCT; analyze data; publish (over 2
-
3 years)


Agile design approach


user interaction design


rapid cycle iteration with
qualitative
and quantitative user
studies


Why wait till end of RCT to find out that

the system


didn’t work or wasn’t used, and have little idea why?

User Interaction Design


The practice of designing interactive digital products


more than user interface, is user experience


User interaction design expertise


http://www.visualizing.org
/


http://www.coroflot.com/public/individual_search_results.asp?
k
eywords


contact Beth
Berrean

BerreanB
@
MEDSCH.UCSF.EDU
,
UCSF
Mobile Services


Participatory design


bring in your target users, sit them down with developers, build
and test prototypes over a day


like code
-
a
-
thons, e.g.,
http
://
hackinghealthuc.eventbrite.com
/

March 6, 2012: I. Sim

What Next?

Epi 206


Medical Informatics

Rapid Cycle Field Testing


Qualitative


surveys, interviews, video recordings, “talk
-
aloud
protocols”


Quantitative


user analytics


track

app
launches, clicks, dwells, scrolls, links to web and back…e.g.,
Flurry.com


Google launched 450 search engine improvements in 2007,
each one
tested rigorously

through real
-
time user analytics


embedded RCTs


March 6, 2012: I. Sim

What Next?

Epi 206


Medical Informatics

Embedded RCTs


Identify part of the system for testing (e.g., videos vs.
pictures of inhaler use)


Randomly assign each person landing on the screen
to videos or pictures


Test for comprehension with short survey


Compare % correct answers, set

stopping rule,


run
study until answer clear or test is over


Can run multiple embedded user interaction RCTs to
optimize design features in parallel


March 6, 2012: I. Sim

What Next?

Epi 206


Medical Informatics

March 15, 2011: I. Sim

Clinical Research 2.0

Epi


206 Medical Informatics

Validation Methodology


Participant
recruitment and sample size


selection bias, sampling error


Outcomes assessment


measurement error and bias


Clinical research 2.0

March 15, 2011: I. Sim

Clinical Research 2.0

Epi


206 Medical Informatics

Subject Recruitment


Recruitment is biggest bottleneck of clinical research


30
-
40% of clinical trial costs


>80% of trials have recruitment delays


1/20 recruited patients actually enroll



Web
-
based recruitment can be international, cheap,
fast


e.g.,
www.stopsmoking.ucsf.edu

Dec 05
-

Feb 07


350,000 hits, 60,000 entered data, 20,000 enrolled


2/3 Spanish
-
speaking, 1/3 English


131,517 visits from 121 countries
1/12/05
to
4/5/06

March 15, 2011: I. Sim

Clinical Research 2.0

Epi


206 Medical Informatics

V
i
s
i
t
s
0
=
>
1
=
>
1
0
0
=
>
1
,
0
0
0
=
>
1
0
,
0
0
0
Distribution of Visits to
www.stopsmoking.ucsf.edu


Jan 12, 2005 to April 5, 2006

(131,517 visits from 121 countries)

March 15, 2011: I. Sim

Clinical Research 2.0

Epi


206 Medical Informatics

Reducing Sampling Error


Social sciences and marketing are most advanced in
population
-
level recruitment methodology


e.g., Joint Statistical Meetings of the American Statistical
Association


http://www.knowledgenetworks.com/


Two major methods


recruit a representative sample


use a pre
-
assembled representative cohort

Disclosure: I have no relationship with KnowledgeNetworks

March 15, 2011: I. Sim

Clinical Research 2.0

Epi


206 Medical Informatics

Recruit Representative Sample


Random digit dialing (RDD) with web
supplementation equally representative as (land
-
line)
telephone RDD


RDD sampling (landline and cellular)


if respondent agrees, provide them with free Internet access
(via MSN TV, formerly WebTV) or other necessary
hardware for duration of participation


e.g.,http://knowledgenetworks.com/

March 15, 2011: I. Sim

Clinical Research 2.0

Epi


206 Medical Informatics

Representative Cohorts


Maintained by e.g., large survey and marketing firms


http://www.knowledgenetworks.com/knpanel/index.html



KnowledgePanel is representative of US


can target specific respondents,

survey response rates of 65
-
75%, abandonment rate <2%



www.surveysampling.com


~6 m people from 72 countries, incl. health
-
specific panels


http://www.experimentcentral.org/



NSF
-
funded representative panel for social science survey
research

March 15, 2011: I. Sim

Clinical Research 2.0

Epi


206 Medical Informatics

Other Recruitment Methods


With higher risk of sampling bias


search engines, with search engine
optimization (SEO) techniques (e.g.,
Google
adwords
)


links from related pages


listservs
, Facebook, etc.



Can combine radio, TV, or print w/


website (URL, Uniform Resource
Locator)


text messaging Common Short Codes


QR code (or matrix bar code) launches
URL, vCard, displays text, etc.

March 15, 2011: I. Sim

Clinical Research 2.0

Epi


206 Medical Informatics

March 15, 2011: I. Sim

Clinical Research 2.0

Epi


206 Medical Informatics

Search Engine Ranking


Search engines have their own (secret) algorithm for ranking
pages


Google uses >100 factors, esp. how many pages link
into

a
page


Google
AdWords



put in your keywords,
see cost
-
per
-
click


pay
only if someone clicks, more if the keyword historically
brings more
traffic


More on recruitment strategies using technology


Recruitment: What Methods Are Appropriate?


Friday, May 4, 1
-
3 pm,
HSW
-
303, Laurie
Herraiz

March 15, 2011: I. Sim

Clinical Research 2.0

Epi


206 Medical Informatics

Note on Sample Size


Estimating sample size


e.g., Google provides traffic history for various keywords
(adwords.google.com)


Since incremental cost of web surveys/ICT
interventions are often negligible, less pressure to
minimize sample size


not unusal to get large samples (>10,000)


But high sample size = high accuracy!


may be precise (p < 0.05) but inaccurate if sample is non
-
representative

March 15, 2011: I. Sim

Clinical Research 2.0

Epi


206 Medical Informatics

Validation Methodology


Participant
recruitment and sample size


selection bias, sampling error


Outcomes assessment


measurement error and bias


Clinical research 2.0

March 15, 2011: I. Sim

Clinical Research 2.0

Epi


206 Medical Informatics

Measurement Error/Bias


What you designed may not be what respondent
sees


Client

s browser/phone displays the intervention
based on


platform, browser, monitor, screen/window size


different users see different survey, e.g.,


small screen/window size makes

Next


button not visible


text doesn

t fit on small window, or requires scrolling for some
respondents and not others


colors, graphics (e.g., visual analog scales) may appear
differently

March 15, 2011: I. Sim

Clinical Research 2.0

Epi


206 Medical Informatics

Loss to Follow
-
up


Influenced by


respondent familiarity with functions (e.g., using
short codes)


technical design of survey


disability
accesibilty

(Section 503 compliance)


bandwidth


convenience (can interrupt survey?)


Can use mixed
-
mode surveys/interventions to
increase completion rates/follow up, e.g.,


combined web/voice, web/mail surveys


combined desktop/laptop and tablet/mobile (I.e., use
different form factors for initiating vs. sustaining intervention
effect)

ICT Research Methods


International Society for Research on Internet
Interventions


http://www.isrii.org


Health
UnBound

Evaluation Commons


http://www.healthunbound.org/content/evaluations
-
commons


k4health mHealth Toolkit


http://www.k4health.org/toolkits/mhealth


mHealth data worldwide


http://
mobileactive.org
/
areaofpractice
/Health


March 15, 2011: I. Sim

Clinical Research 2.0

Epi


206 Medical Informatics

March 15, 2011: I. Sim

Clinical Research 2.0

Epi


206 Medical Informatics

Validation Methodology


Participant
recruitment and sample size


selection bias, sampling error


Outcomes assessment


measurement error and bias


Clinical research 2.0

March 6, 2012: I. Sim

What Next?

Epi 206


Medical Informatics

What Learning Occurs Now?


Studies are expensive, difficult to conduct, 30
-
40% of
studies never accrue enough patients


Studies take years to answer limited questions in limited
populations


Study designs and results are
heterogenous
, limiting ability
to pool findings or make summary interpretations


Research questions don’t address combination treatments
(e.g., ACEI and amlodipine)


Research questions answer front
-
line clinical needs


little data on mid
-

to long
-
term effectiveness of
antidepressants


Overall lack of relevance, generalizability, and sustainability


Moss, et al. NEJM 2011; 364(9):789
-
761

Crowley, et al. JAMA 2004; 291(9):1120
-
6 etc
.

March 6, 2012: I. Sim

What Next?

Epi 206


Medical Informatics

What Learning is Needed?


Population
-
level efficacy and effectiveness


for endpoints (e.g., through secondary analysis of EHR data,
not
only intermediate
outcomes


for patient
-
centered outcomes (symptoms, side effects)


Therapeutic precision (best therapy for
this

patient)


informed by, but not limited to, genomic treatment markers


learning from experience (e.g., N
-
of
-
1 trials)


How to promote and sustain behavior change


What are individualized predictors (e.g., depression, IBD,
asthma)


What are prevalence, natural
hx
, etc. even of rare diseases?


How to enable patients, families, communities, and clinicians to
maintain wellness and manage chronic illness together? etc. etc
.

March 6, 2012: I. Sim

What Next?

Epi 206


Medical Informatics

Opportunity: CRC Without Walls


Traditional clinical research centers (CRCs)


patients came in to campus for
assessment


Make UCSF a CRC without
walls (e.g.,
Mytrus
)


intervene and assess where
health and disease are


get
at
richer, real
-
time
phenome

and
exposome


Formulate broader notion of clinical research


build in relevance, generalizability, sustainability


exploit Web 2.0 principles


Generate, share, and build upon evidence
-
based best practices with rapid feedback

Anti
-
depressant Efficacy

In 2005,

27 million
Americans were prescribed
anti
-
depressants
1


…data often come from short
-
term (6
-

to 12
-
week) efficacy trials that cannot show whether
treatments are effective over the medium
-

and
long
-
term

2






1
Olfson, et al. Arch Gen Psych 2009;66(8):848
-
856

2
APA Depression Guideline 2010

Learning Healthcare System

RCT of long
-
term
comparative effectiveness
of antidepressants in
primary care

We invite you to
participate in a study on
the effectiveness of …

Xing Xu

10/4/2010

7/21/1932

7/21/1932

7/21/1932

427 King Rd. SF

Prozac 20 mg, 1 tab PO daily, #30

Consent to Being
Contacted for Studies





Yes

APEX

Watch this YouTube
video for informed
consent…real
-
time chat
for questions…secure
sign
-
up for enrollment

Or at a website

Every Clinic is a Study Site

Randomization

ePharmacy

AntiD Study

Masked drug(s)

Dec 13, 2009

Guilt

Child care

Worse after school drop
-
off


AT&T

Data Collection App

Wakemate Sleep Monitor

ePharmacy

AntiD Study

e
-
coupon

Study DB

Dec 13, 2009

Guilt

Child care

Worst after school drop
-
off


AT&T

covariates

Anonymization

APEX

Large
-
Scale Research

In 2005, 27 million Americans were prescribed anti
-
depressants
1


…data often come from short
-
term (6
-

to 12
-
week) efficacy trials
that cannot show whether treatments are effective over the
medium
-

and long
-
term

2

Since 2005, # of subjects in all antidepressant drug trials
worldwide total <100,000 (<0.4% of 27 million)

If only

1 out of 250

antidepressant patients in the US enrolls,
would exceed total number of participants in all antidepressant
trials worldwide in last 5 years




1
Olfson, et al. Arch Gen Psych 2009;66(8):848
-
856

2
APA Depression Guideline 2010

rephrasing
‘does it work?’

(Complexes of)
Exposures

Outcome

strength of association?

individual

population

depression

duloxetine

‘does it work on average?’

50 people

population

100 people

E
ffexor

depression, PHQ
-
9

depression, PHQ
-
9

50 people

Z
oloft

March 13, 2012: I. Sim

What Next?

Epi


206 Medical Informatics

But…


Latest research indicates that Effexor’s 8
-
week
remission rate (68%) is greater than Zoloft’s
(45%)
1


Among your past 20 patients in APEX


6 Effexor patients, 3 remitted (50%, not 68%)


14 Zoloft patients, 12 remitted (86%, not 45%)


What should you do for Patient X?

1
Mehtonen OP, et al. Randomized, double
-
blind comparison of venlafaxine and sertraline in outpatients
with major depressive disorder. Venlafaxine 631 Study Group. J
Clin

Psychiatry. 2000 Feb;61(2):95
-
100.


does it work for
Patient X?


PHQ
-
9*

Effexor

Zoloft

Zoloft

Effexor

Effexor

Zoloft

individual

me

N
-
of
-
1 study design

PHQ
-
9*

1
Kravitz, et al.
Contemp

Clin

Trials 2009; 30:436
-
445

*PHQ
-
9, PROMIS sleep, PROMIS fatigue, daily mood,
actigraphy
, voice stress,
Facebook activity, sleep, custom outcomes, etc.

March 13, 2012: I. Sim

What Next?

Epi


206 Medical Informatics

Empirical Bayes Approach


Empirical Bayes method balances between the two
parcels of evidence (borrow from strength)



T
i,EB

= (1


B
i
) * T
GLOBAL

+ B
i

*
T
i,LOCAL



B
i

= s
BETWEEN
2

/ (s
BETWEEN
2

+ s
WITHIN,i
2
)




The greater the between
-
group variance relative to
the total variance, the more weight is placed on local
evidence


Local evidence accumulates over time

flipping direction of
research inference

PTSD


sx

PTSD Coach

Usual care

Usual care

PTSD Coach

PTSD Coach

Usual care

PTSD sx

individual

me

population

PTSD


sx

PTSD Coach

Usual care

Usual care

PTSD Coach

PTSD Coach

Usual care

PTSD sx

individual

me

PTSD


sx

PTSD Coach

Usual care

Usual care

PTSD Coach

PTSD Coach

Usual care

PTSD sx

individual

me

PTSD


sx

PTSD Coach

Usual care

Usual care

PTSD Coach

PTSD Coach

Usual care

PTSD sx

individual

me

PTSD


sx

PTSD Coach

Usual care

Usual care

PTSD Coach

PTSD Coach

Usual care

PTSD sx

individual

me

March 6, 2012: I. Sim

What Next?

Epi 206


Medical Informatics

Outline


So you
wanna

build an app…


app design and development


validation methodology


Beyond the hype


Class summary

IDA SIM, CO
-
FOUNDER

DEBORAH ESTRIN, CO
-
FOUNDER


DAVID HADDAD, PROGRAM MANAGER

JOSH SELSKY, CHIEF SW ARCHITECT

#
openmhealth


Funded by the Robert Wood Johnson Foundation and
the California Health Care Foundation

A project of the Tides Center

data driven feedback loops

2

without better sensemaking to
drive these feedback loops…

Plateau of Diminished Promise

h
ow can we foster an mHealth
ecosystem that creates meaningful
care innovation and evidence?


…open architecture and community

open architecture

Decentralized, innovative, co
-
development community
needs architecture to ensure mix and match


Architecture is a small set of common principles/practices
by which reusable modules can be described and interface
to one another

open modules

InfoVis
: reusable modules for data collection, analysis,
visualization and feedback


Personal Evidence Architecture: reusable modules for


shared measures


context metadata


scripting and execution of n
-
of
-
1 and other studies



Informatics @ UCSF


Large campus
-
level biomedical informatics
initiative pending


new faculty, training programs,
etc


Needs to span bioinformatics to traditional
medical informatics to digital health


CTSI BMI is current home for med informatics


mHealth group

and
listserv


Digital Health Innovators Forum


RAP mHealth grants for
Research

and
Translational

Projects

March 6, 2012: I. Sim

What Next?

Epi 206


Medical Informatics

March 6, 2012: I. Sim

What Next?

Epi 206


Medical Informatics

Outline


So you
wanna

build an app…


app design and development


validation methodology


Beyond the hype


Class summary

State of Health IT


EHR adoption still low


barriers include finances, lack of organizational change
expertise, fragmentation of health care system, misaligned
incentives


Recovery Act is spur EHR adoption, for good or ill


EHR and data warehouses can but don’t always help
research


Limited success of decision support systems


March 6, 2012: I. Sim

What Next?

Epi 206


Medical Informatics

The Uneven Future


A “flat” world, blending health and healthcare
unbounded by walls or time


Lots of data, lots of things, everyone is “in”


many non
-
traditional health players


Challenges will be


aggregating smaller data into Big Data


analyzing Big Data in appropriate clinical, social,
environmental, etc. context


drawing scientific conclusions, showing validity


March 6, 2012: I. Sim

What Next?

Epi 206


Medical Informatics

New Picture
of
Health(care)

Virtual
Patient



Transactions



Raw data


Medical
knowledge


Clinical
research
transactions


Raw
research
data


Decision
support

Medical logic

PATIENT CARE /
WELLNESS

RESEARCH

Interaction design, workflow
modeling and
support,
cognitive
support
, policies and
mechansims

for sharing, privacy

Patient Transactions

. .

Patient

. .

. .

. .

Big (Open) Data

. .

Doctor

. .

. .

. .

. .

Citizen

Scientists

. .

. .

. .

Open Discussion


How to balance standardization and comparability
(e.g., of EHR notes, of research variables) with
flexibility and innovation?


Medicine and biomedical research is conservative


will all this web 2.0/3.0 stuff pass right by us?


How will this change what you do/how you think, if at
all?


What would you like to see from academia/UCSF to
help you stay as competitive in research as possible?


???


March 6, 2012: I. Sim

What Next?

Epi 206


Medical Informatics

Summary


Informatics helps make sense of data and knowledge


is necessary for better care and research


Today

s technologies promise transactional support


major barriers are economic, policy, and workflow related


Need brand new technologies for other 3/4 of Big
Picture


Disruptive change to clinical research
is coming (fast)


as mobile technologies break down time and space barriers


as social computing

takes off


as open data becomes more
common


March 6, 2012: I. Sim

What Next?

Epi 206


Medical Informatics

Take Home Message


Bold new era of opportunities in informatics


To guard against hype, know as much as
possible of the underlying informatics


Bring all your other expertise to informatics
and vice versa


Go lead!


Course
evaluation


http://rds.epi
-
ucsf.org/ticr/CourseEvaluations/ticreval.asp?id
=478

March 6, 2012: I. Sim

What Next?

Epi 206


Medical Informatics