Defining Information Systems from an Academic Perspective

earthblurtingAI and Robotics

Nov 14, 2013 (3 years and 7 months ago)

107 views


Page
1

Defining Information Systems from a
n

Academic

Perspective

Authors: Devipsita Bhattacharya, Samuel Birk, John Gastreich, Justin Giboney,
Chenhui Guo, Shan Jiang, YuKai Lin, Jeff
Proudfoot, Ryan Schuetzler, Jaebong Son,
Xing Wan,
Justin Williams



Page
2

Contents

What is MIS?
................................
................................
................................
..........................
4

How can MIS be identified within academia?

................................
................................
.............

4

What differentiates high and low quality MIS research?

................................
............................

5

Method

................................
................................
................................
................................
.
6

Data for Article Citations and Characteristics

................................
................................
..............

6

Measures
................................
................................
................................
................................
......

6

Dependent variable.

................................
................................
................................
.................

6

Contribution.

................................
................................
................................
............................

7

Article Coding

................................
................................
................................
...............................

8

Cluster analysis of
Research Paper Abstracts

................................
................................
..........
8

Approach

................................
................................
................................
................................
......

8

Dataset

................................
................................
................................
................................
.....

9

Process for Analyzing Abstract

................................
................................
...............................

10

Five Naturally Formed Clusters

................................
................................
................................
..

11

IS for Decision Support Cluster

................................
................................
...............................

11

Organizational Behavior Cluster

................................
................................
............................

12

Electrical Engineering & Healthcare Cluster

................................
................................
..........

13

Economics & Accounting Cluster

................................
................................
............................

14

What MIS is
NOT Cluster

................................
................................
................................
........

14

Conclusions from Clustering Analysis

................................
................................
........................

15

Keyword Analysis

................................
................................
................................
.................

16

Vector Space

Model

................................
................................
................................
...................

16

Cosine Similarity

................................
................................
................................
.........................

17

Implications

................................
................................
................................
................................

21

Limitations

................................
................................
................................
................................
.

22

Discipline Correlation by Citations

................................
................................
........................

22

References Analysis

................................
................................
................................
...................

25

Number of citations received by a discipline
................................
................................
..........

26

Number of references given by a discipline

................................
................................
...........

27

Number of Self citations made by each discipline
................................
................................
..

29

Number of citations received Vs number of references made

................................
...............

31

Market share of citations received by discipline

................................
................................
....

32

Market share of references given by discipline

................................
................................
......

33


Page
3

Interaction Between MIS and Other Disciplines

................................
................................
........

35

What Makes a MIS Article a High
-
Quality Article?

................................
................................
.

42

Overview

................................
................................
................................
................................
....

42

Advanced statistical analysis of “high
-
quality” MIS articles

................................
......................

44

The 6 Basic Variables

................................
................................
................................
.................

44

The
Generation of Textual Variables

................................
................................
.........................

45

The results of Logistic Regression Analyses

................................
................................
...............

46

Analysis to Identify the Determinants of a Highly
-
Cited MIS Paper

................................
........

51

Introduction

................................
................................
................................
...............................

51

SPSS Logistic Re
gression

................................
................................
................................
............

51

Analysis 1

................................
................................
................................
...............................

51

Analysis 2

................................
................................
................................
...............................

53

Analysis 3

................................
................................
................................
...............................

55

R


Matched Pair Logistic Regression

................................
................................
........................

58

Using coded variables

................................
................................
................................
............

58

Using other variables

................................
................................
................................
.............

59

Discussion

................................
................................
................................
................................
..

60

Future Research

................................
................................
................................
.........................

61

General Conclusions

................................
................................
................................
.............

62

Works Cited

................................
................................
................................
.........................

63

Appendix 1
-

C2MIS Evolutio
n
................................
................................
...............................

64

Appendix 2


Highly Cited MIS Articles

................................
................................
.................

71

Appendix 3


Motion Chart

................................
................................
................................
..

73







Page
4

What is MIS?

Previous MIS696a projects and the MIS literature in general make it evident that this is not an
easy question to answer. When approached with this question, our class decided to approach it from
two directions. First, considering the importance of MIS as a

multidisciplinary field, we were interested
in determining the similarities, differences, and relationships between MIS and several related research
fields (e.g. psychology, management, economics, computer science, etc). The hope was that this
examination

would provide insight into how MIS can be identified within academia. Second, we were
interested in determining not only what MIS is in general, but what attributes can be used to identify
high quality MIS research, which we consider to be the core of MIS
.

How can MIS be identified within academia?

The project conducted by MIS 696a in 2009 concluded that future projects should investigate
the relationship between MIS and related disciplines. In order to explore these relationships, we first
determined 12
related disciplines identified by (
Katerattankul, Han, & Rea, 2006
). Several of these
disciplines were also identified by previous projects. We then identified 6 to 9 journals high quality
journals for each of the 12 disciplines using either ISI’s discipli
ne search or articles citing quality journals
within the discipline
i
.

Next, article and citation d
ata for this study
was

obtained from the Institute for
Scientific Information
’s

(ISI)
Web of Knowledge website. ISI is the

major source of
academic research
c
itation information
, reporting citation and article information for over 8,500 scientific journals.

Citation
and article information was obtained for articles published in 13 identified journals
. Because we were
interested in studying articles that have an

impact on the
ir

field through their empirical and theoretical
research, we limited our scope to

publications that ISI classified as

articles, note
s, or

reviews

(here after
called articles)
. In total
,

citation

and article

information was
obtained
for
102,3
88 articles. We will refer
to this as the overall citation database.


Page
5

This data was then cleaned and mined to determine relationships between MIS and other
disciplines based on frequently used words in abstracts, commonly used keywords, and the number of
ci
tations to other disciplines. More information about these analyses will be provided later.

What differentiates high and low quality MIS research?



We felt that beyond understanding how to identify MIS in academia at large, it was
necessary to determine how to identify MIS research that is widely recognized and appreciated for its
contribution to the field. These articles might be considered the core
of MIS, and as such we felt it was
necessary to determine how to differentiate this research from general MIS publications.

Further, one of the most important indicators of a scholars’ success is the number and quality of
the articles s/he publishes throu
ghout a career. These publications are used to inform promotion and
salary decisions for professors. Therefore, it seemed practical for us, as future MIS scholars, to
understand the factors that influence the quality of MIS publications.

A

recent study by

Judge et al. (2007) has begun to identify some of the important factors directly
influencing article citations, such as the quality of the journal the article was published in, whether the
article explored new theoretical paradigms, the number of referenc
es the article cited, and type of
research design employed. In addition, these authors also found that the clarity of an article’s
presentation, the prestige of author affiliations, and whether the study included independent data
sources also indirectly af
fected the citations an empirical article received.

We therefore sought to investigate the impact of these factors on citations in the field of MIS.
We also attempted to extend
the Judge et al. (2007) study
by specifically identifying

the characteristics
that distinguish highly
-
cited
MIS

articles from those that are less highly
-
cited. This is important because
our research indicates that although “citation classics” (i.e., articles that have received at least 100

Page
6

citations) represent a relatively small pro
portion of the total number of articles published
,

they
account

for
large proportion

of the citations attributed to journals
.

Given the disproportionately large effect of
these “classics,” we believe it is critical to understand
those
factors
that
predict whether an article will
have a disproportionate level of impact, or not.

W
e
also
include
d

additional measures of theoretical contribution (i.e., theory
building and theory
testing; Colquitt & Zapata
-
Phelan, 2007
; Newman & Cooper, 1993
) that were

excluded from th
e Judge et
al. study. Finally,
we examine a list of citation classics (and matched pair articles) published

between
197
0 and 2010
.

Method

Data for Article Citations and Characteristics

Using the overall citation dataset, we identified a

subset of “citation classics.” We defined a
citation classic as an article in our database having 100 or more citations. 50 of these articles were
randomly selected to be coded. We also identified 50 non
-
citation classics (articles having fewer than
100 c
itations) that were matched on publication year, journal to control for article age and journal
source, factors which have been shown to have substantial effects on citations in the field of
management (Colquitt & Zapata
-
Phelan, 2007; Judge et al. 2007; Po
dsakoff et al., 2005). Therefore, we
identified a set of 50 matched pairs, for a total of 100 articles. We will refer to this data as the matched
pair database in future discussions.

Measures

Dependent variable.

Article impact was measured as the total nu
mber of citations that each
article in our sample had accrued in
ISI’s

Social Science Citation Index (SSCI) from the time it was
published until
October

31
st
, 2010
. Using the total number of citations an article has received is the most

Page
7

frequently used mea
sure of article impact (Bergh et al., 2006; Colquitt & Zapata
-
Phelan, 2007; Judge et
al., 2007; Kacmar & Whitfield, 2000; Newman & Cooper, 1993; Podsakoff et al., 2005, 2008).


Contribution.

We employed two measures to assess universalistic attributes rela
ted to the
ideas presented in each article. First, we coded the “research plot” of each article according to the
criteria described by Newman and Cooper (1993). Articles having
refinement plots

were those that
focused on increasing the accuracy of the scope of known things by replicating previous studies in
different settings or with different statistical/methodological techniques. Articles having
extension plots

were those that focused on “arti
culating an existing paradigm” (Newman & Cooper, 1993, p. 520) by
examining relationships between previously examined dependent variables and previously unexamined
independent variables, examining the relationships between multiple independent variables (p
reviously
examined in isolation) with a dependent variable, or proposing new mediators/moderators to better
explain existing links between independent and dependent variables. Finally, articles with
exploration
plots

tried to carry a paradigm into unknown
territory by examining a traditional dependent variable as
an independent variable, examining variables at a new level of analysis, or introducing a new casual
network or foundations for new theory. Raters coded each article based on the highest level rese
arch
plot present in an article.

Second, we also coded articles based on the “theory building” and “theory testing” criteria
developed by Colquitt and Zapata
-
Phelan (2007). Colquitt and Zapata
-
Phelan developed 5
-
point scales
designed to rate the extent to

which empirical articles build new theory (1 = “attempts to replicate
previous findings” to 5 = “introduces a new construct [or significantly reconceptualizes an existing one]”)
and test existing theory (1 = “is inductive or grounds predictions with logic
al speculation” to 5 =
“grounds predictions with existing theory”). These authors reported that both the theory building and
theory testing variables explained significant variance in empirical article impact, even after controlling

Page
8

for the year the articl
e was published. Therefore, we coded both dimensions for all empirical articles in
our sample.

Article Coding

First, each student independently coded a set of 3 randomly selected articles from the matched
pairs database. After completing their codes, the
two raters compared codes and discussed all
discrepancies, which led to the refinement of several coding criteria. Following this, students again
independently coded a set of 3 randomly selected articles from the matched pairs dataset. Comparison
of the in
dividual codes and discussion of differences in ratings, led to a few additional refinements to
the criterion coding scheme. In the final step of this process, each student was assigned a set of
randomly selected articles representing approximately a third

of the remaining dataset. Each student
coded the articles on his/her own and then met with a group of 1 or 2 other students who had coded
the same articles. The groups changed depending on the particular articles ensuring cross
-
validation of
coding decisi
ons.

Cluster analysis of Research Paper Abstracts

Our assumption is that similar disciplines use similar words, and the similarity among disciplines
may be reflected in the abstracts from research papers. Therefore, text mining was leveraged to analyze
and

compare the abstracts form research papers to determine disciplines similar to MIS and disciplines
dissimilar to MIS.

Approach

To conduct text mining, two main analyses were required: term extraction and cluster analysis.
Both analyses were performed usin
g Microsoft SQL Server 2000. One of the main analysis steps is the
extraction of terms. SQL Text Mining uses a Markov chain
-
based grammar model to detect terms in

Page
9

addition to performing a normal stemming technique.

Terms extracted were used as input val
u
es

for
the
cluster analysis. EM clustering algorithm of SQL Server 2008 is a probabilistic
-
based technique that
iteratively refines an initial cluster model to fit the data and determines the probability which a data
point exists in a
cluster
. Therefore, the

results are probabilistic.

Dataset

38,642 out of our complete dataset of 102,388 records were chosen for this analysis because
our dataset only has the abstracts from research papers for those 38,642 records; the other records in
our dataset do not have

abstracts and therefore could not be included.

Figure

1

shows the proportion of disciplines that were used in the abstract analysis. MIS, and
Economics, and Electrical Engineering disciplines had a large volume of research papers with
approximately 5,000

each. In contrast, the Accounting and Communication disciplines had a relatively
small number of research papers with about 1,000 each.

To prevent the unbalanced data volume from causing biased results, the number of terms to
represent each discipline wa
s limited to 150.


Figure
1

-

Proportion of Discipline in Analysis



Page
10

Process for Analyzing Abstract

We followed four steps to analyze abstracts. Through the analysis, we expected that similar
disciplines would be clustered
together.


Figure
2

-

Abstract Analysis Process

Step 1: Nouns and noun phrases (terms) are extracted from all thirteen disciplines by the Term
Frequency (TF) method, which is commonly used in text mining tasks.

Step 2: Using the
previously extracted terms from Step 1, a global vocabulary was created,
representing all terms from all disciplines. We obtained 817 terms, after the elimination of redundant
terms.

Step 3: The global vocabulary was used to create bag
-
of
-
words model by in
dexing all extracted
terms. The bag
-
of
-
model is a simple assumption for natural language processing, text mining, and
information retrieval systems.

Step 4: Cluster algorithm used this bag
-
of
-
words model as input variables to form clusters.



Page
11


Five Naturall
y Formed Clusters

From the analysis, five naturally formed clustered were generated

(
Figure 3
)
. Clusters were then
labeled by their
top ten keywords and their
most predominant disc
iplines; the labels given were
IS
for
Decision Support

(5,524 articles)
,

Organizational Behavior

(6,196 articles)
,

Electrical Engineering &
Healthcare

(9,270 articles)
,

Economics & Accounting

(9,555 articles)
, and
What MIS is NOT

(8,097
articles)
.



Figure
3

-

Cluster Proportions

IS
for Decision Suppor
t

Cluster

The first cluster (
Figure 4
) shows the

high
est

concentration
s of MIS (30.3%) and Library Science
(24
.
1
%)

with a
rather
large segment of Communications (14.4%).
All of the other disciplines in the

cluster represent less than ten

percen
t each. This

cluster shows a high overlapping of MIS, Library
Science, and Communications
.

On this cluster, Psychology, Economics, and Electrical Engineering have
very low representations.


IS for Decision
Support, 5524
Organizational
Behavior,
6196
Elec Eng &
Healthcare,
9270
Econ & Acct,
9555
What MIS
is NOT,
8097

Page
12


Figure
4

-

IS for Decision Support Cluster

Terms in this cluster include:
decision support system (DSS), information, system, software,
organization, database, Web, collaboration, knowledge, and information retrieval.

From these
keywords, it can be seen that the abstracts from the articles in this
cluster are highly related to such
topics
such
as collaboration, information systems, and decision making.

Organizational Behavior

Cluster

The second cluster (
Figure 5
) was labeled
Organizational Behavior

due to its diverse number of
discipline
s related t
o human behavior and its keywords

related to the human side
. This cluster was more
evenly distributed than
the previous clusters with several disciplines representing six

percent or more. It
is represented by Management (20
.2%),
Computer Science (12.9%),
M
arketin
g (11.8%), and Sociology
(9.8%), Library Science (8.0%),

and
Psychol
ogy (6.6%)
. It appears that MIS

also

is
somewhat related to
the articles in this cluster with a 6.6% representation
.


Page
13


Figure
5

-

Organizational Behavior
Cluster

Terms in this cluster include:
transformational leadership, leader
-
member exchange (LMX),
relational uncertainty, organizational citizenship behavior (OCB), organizational commitment,
leadership, satisfaction, culture, meta
-
analysis, and social mov
ement.

Electrical
Engineering
& Healt
hcare

Cluster

The third

cluster (
Figure 6
) was labeled the
Electrical
Engineering

& Healthcare

Cluster due to
the domination by the discipli
ne of Electrical Engineering (30.2
%)

and Healthcare (20.5%)
.
Again, MIS is
represented with a 6.6% share of the articles in this cluster. This

indicate
s

that MIS is somewhat related
to
the
Electrical Engineering
& Healthcare discipline
.



Figure
6

-

Electrical Engineering & Healthcare Cluster

Terms in th
is cluster include:
inverter, induction motor, sensor, topology, mobile robot, neural
network, architecture, system, support vector machine (SVM), and genetic algorithm (GA).


Page
14

Economics & Accounting

Cluster

The fourth

cluster (
Figure 7
) was labeled the Econ
omics
& Accounting
Cluster due to the
dominance of the Economics
and Accounting
discipline
s with both representing about 27% of the
cluster.

Another strongly represented discipline is Marketing (16.2%). Most other disciplines
represent
five percent or less

of the cluster.

This cluster is very financial
-
based with an emphasis on numbers.


Figure
7

-

Economics & Accounting Cluster

Terms in this cluster include:
earnings announcement, Financial Accounting Standard Board
(FASB), Sarban
es
-
Oxley Act (SOX), audit fee, equilibrium, valuation, private information, bidder, earnings
forecast, and incentive.

What MIS is NOT

C
luster

The fifth cluster (
Figure 8
) was labe
led What
MIS is NOT

due
its very limited splice of the cluster.
This cluster
is dominated by Psychology (23.4%), Sociology (17.3
%),
Communication (16.1%), Education
(16.5
%), and
Healthcare (9.8
%). All of disciplines represented five percent or less of the cluster.
Interes
tingly, MIS only represented 1.2
% of the cluster which would
indicate that MIS is ve
ry dissimilar
to the types of papers this cluster represents
.


Page
15


Figure
8

-

What MIS is NOT Cluster

Terms in this cluster include:
somatic symptom, body mass index, bipolar disorder, anxiety
disorder, (major)
depression, (psychiatric, mental) disorder, physical activity, medication, blood pressure,
and competitive intelligence (CI).

Conclusions from Clustering Analysis

The results from this cluster analysis show that MIS is more similar to some disciplines and
less
similar to others

(
Figure 9
)
. MIS is more similar to Library Science

and Communication

with common
terms such as
information, system, software, organization, database, Web, collaboration, knowledge,
and information retrieval
. On the other extreme
,

MIS appears to be very dissimilar to P
sychology &
Social Sciences which have

key terms such as
body mass index, physica
l activity, blood pressure,
bipolar
disorder
, and anxiety disorder
.


Figure
9

-

Conclusions from Clustering A
nalysis


Page
16

Keyword Analysis

In order to compare the relationships among MIS and the other
twelve

disciplines with keyword
analysis, there are three q
uestions to be asked and addressed

in this section
:

1.

How to represent a discipline

with keywords
?

2.

Based on the
representation, how to compare the relations/similarities among different
disciplines?

3.

How’s the relations/similarities between MIS and the other disciplines evolve over time?

W
e propose to utilize vector space model

(Raghavan & S. K. M. Wong, 1986; Salton

et al., 1975)

to represent each discipline, and compare them with cosine similarity, a typical measurement of
similarity in text mining.
I
n the following, we will explain the exact processes on how the keyword
analysis is conducted. I
n the end of this sec
tion
,

we provide implications as well as limitations of this
keyword

analysis.

Vector Space Model

In a vector space mode, target items are represented as vectors.
T
he elements inside a vector
are called

features.


In our context, we try to use paper keywords as features to represent a disciple.
T
he use of keyword to represent content of a text file has been used for years in the information
retrieval and text mining studies. In the beginning, there is a process to
transform a
discipline

with its
representative keywords used in that field (Figure 1
0
).



Page
17

MIS
Computer
Science
MIS
Computer
Science
K
1
K
x
K
2
w
11
w
12
w
1
x
w
21
w
22
w
2
x
D
n
D
1
D
1
D
2
D
1
K
1
,
K
2
,

,
K
m
D
n
D
1
D
1
D
2
D
1
K
1
,
K
2
,

,
K
m







Figure
10

-

An illustration of the transformation from discipline keywords to discipline feature vectors

A
fter t
he transformation, it is the vectors, instead of keywords
themselves, which

are used to
represent a discipline. Suppose that we have successfully represented two disciplines, say MIS and
Computer Science, using vectors of their corresponding keywords. Thei
r representative term vectors will
be:


= <w
11
, w
12
, … , w
1x
>


= < w
21
, w
22
, … , w
2x

>

Cosine Similarity

Following this, we can use cosine similarity, a

widely adopted

similarity assessing model in
vector space model,
to estimate the similarity between two
disciplines, i.e.,
vectors
. Take Figure 2 for
example.
T
he cosine similarity between
v

and
v1

is to measure the angle

1
. Similarly, the
cosine
similarity between
v

and
v
2

is to measure the angle

2
.

T
wo vectors are deemed more similar if they

Page
18

have a smaller angle in between. Given that

1

is smaller than


2
,

in Figure
11

v

and
v1

is more similar
than
v

and
v2
.


1

2
v
v
1
v
2

Figure
11

-

Illustration of cosine sim
ilarity

W
hile it is easy to discern an included angle in a two
dimensional

space
with naked eyes
, it
becomes difficult, if not impossible, to do so in a hyperspace with more than three
dimensions

which is
typical in vector spaces.
T
herefore we need a formula to calculate the included angle.
F
ollowing the
previous example of MIS and Computer Science vectors, the formula to calculate cosine similarity
between the two
disciplines

is as the following:

.

With this understanding, we can f
urther investigate the similarity between two disciplines by
different period of time. To do this, we can simply use the keywords appeared in the
particular

time
span to represent the
disciplines

(see Figure
12

for an example of five
-
year interval).


Page
19

D
1
D
1
MIS
@
1991
-
1995
K
1
K
x
K
2
w
11
w
12
w
1
x
w
21
w
22
w
2
x






MIS
D
n
D
1
D
1
D
2
D
1
K
1
,
K
2
,

,
K
m
D
2
D
1
K
1
,
K
2
,

,
K
m
MIS
@
1991
-
1995
D
1
D
1
D
2
D
1
K
1
,
K
2
,

,
K
m
MIS
@
1996
-
2000
D
1
D
1
D
2
D
1
K
1
,
K
2
,

,
K
m
MIS
@
2001
-
2005
D
1
D
1
D
2
D
1
K
1
,
K
2
,

,
K
m
MIS
@
2006
-
2010
MIS
@
1996
-
2000
MIS
@
2001
-
2005
w
31
w
32
w
3
x
w
41
w
42
w
4
x




MIS
@
2006
-
2010

Figure
12

-

five
-
year interval

The fo
llowing three diagrams (Figure
s

13
-
15
) shows the similarities of MIS with the other 11
disciplines with varied units of time spans (one year, two years, and five years).
T
he data points are
starting from 1990 because before then most of the papers did not contain keyword information.
Therefore, it is only possible to conduct keyword
-
based vector space model if keywords are available.


Page
20


Figure
13

-

The similarities of MIS with the other 11 disciplines using one
-
year time span


Figure
14

-

The similarities of MIS with the other 11 disciplines using two
-
year time span

0
0.1
0.2
0.3
0.4
0.5
1990
1991
1992
1993
1994
1995
1996
1997
1998
1999
2000
2001
2002
2003
2004
2005
2006
2007
2008
2009
2010
accounting
communication
computer science
economics
education
electrical engineering
medical informatics
management
marketing
psychology
sociology
0.0
0.1
0.2
0.3
0.4
0.5
1991-1992
1993-1994
1995-1996
1997-1998
1999-2000
2001-2002
2003-2004
2005-2006
2007-2008
2009-2010
accounting
communication
computer science
economics
education
electrical engineering
medical informatics
management
marketing
psychology
sociology

Page
21


Figure
15

-

The similarities
of MIS with the other 11 disciplines using five
-
year time span

Implications

It can be found
that

although similarities from one
-
year time span provide more data points,
they are rather unstable given that using keywords in a year to represent a
discipline

is rather risky.
I
t is
because these keywords in that year may not be representative enough. Therefore, it makes sense to
use the results with a slightly longer time interval.

Based on the experiments, the following implications are provides.

1.

From the di
agrams, it can be found that the most similar four disciplines of MIS are
Computer Science, Marketing, Management, and Medical Informatics.

2.

Although there are some
fluctuation
s, the order of similarities of different disciplines is
rather stable. That is,

Computer Science, Marketing, Management are at constant high
whereas Accounting, Psychology, and Education are always the least similar disciplines
to MIS.


0
0.1
0.2
0.3
0.4
0.5
1991-1995
1996-2000
2001-2005
2006-2010
accounting
communication
computer science
economics
education
electrical engineering
medical informatics
management
marketing
psychology
sociology

Page
22

3.

T
here is a growing trend among the similarities of MIS with management and marketing.
T
his implies

that in the past decades MIS has been gradually growing toward a business
discipline from its origin of operation research and computer science in the early years.

Limitations

This analysis is not without limitations.
T
he most significant limitation of t
his analysis is that
keywords are not always available.
F
or instance, most articles in the early years do not have keywords
(neither provided by the authors in the first place n
or being recorded in our source database
, i.e.,

ISI

Web of Science). In additio
n, articles in some
influential

publications, such as MIS Quarterly, provide no
keyword information to reader which makes certain bias in our keyword analysis.
N
evertheless, the bias
is expected to be
compensate
d by the abstract
-
based or citation
-
based ana
lyses in this study.

Discipline Correlation

by Citations

For our analysis we assumed that the top (A) journals of each discipline are the purest definition
of the discipline. Therefore we found articles that

listed

the top journals of each field.
For example, if
each discipline is a mountain in a range of mountains, the top of each mountain is the most distinct part
of the mountain. As
the mountain slopes down, it gets closer to the other mountains in the ran
ge. As the
top of the mountain is the furthest distance to all the other mountains, the top journals of a discipline
are going to be the furthest from the other disciplines.



Page
23

In order to find out which disciplines are most closely related to IS we used c
itation counts of
each top journal to every other top journal. The more cites between two journals, the closer the
mountains would be to each other. We found some pretty interesting things. The first thing we found is
that IS top journals cite Marketing an
d Management top journals much more than any other top
journals (see Figure
16
). It is interesting to see that CS and Accounting top journals are not highly cited
by IS.


Figure
16

-

Cites by IS

Seeing which
discipline is cited by IS is only half of the story. We also need to see which
disciplines cite IS (see
Figure
17
). This will show us which disciplines think of IS as
a reference discipline.
Disciplines that cite IS are going to be more closely related than disciplines that do not cite IS. Figure
17

shows a much different picture than Figure 1
6
. Figure
17

shows us that IS is most cited by Marketing,
Education, Library S
cience, and Healthcare.

-
200
400
600
800
1,000
1,200
1,400
1,600
80s
90s
00s
Discipline Cites by IS by Decade

Accounting
Communication
Computer Science
Economics
Education
Electrical Engineering
Healthcare
Library Science
Management
Marketing
Psychology
Sociology

Page
24


Figure
17

-

Cites to IS

Another thing to look at when determining how IS fits in with the other disciplines, is to see over
time how IS is being cited and how it is citing. Figure
18

shows a comparison to how much a discipline
cites other discipline to how much it gets cited by other disciplines. We can see that IS (pink) overtime
has less cites as a percentage to other disciplines (i.e. IS is citing itself more and more). We also see

that
of the cites IS receives, overtime IS is receiving a larger percentage of cites from other disciplines (i.e. as
a percentage it is being increasingly cited by other disciplines). This shows that IS is becoming a
reference discipline not only to itsel
f, but to other disciplines as well. This is in contrast to Electrical
Engineering (EE). EE is folding in on itself. We can see that EE is citing other disciplines less and less,
while being less and less cited by other disciplines. This shows that EE is
n
ot very correlated with a lot of
other disciplines.

-
100
200
300
400
500
600
80s
90s
00s
Discipline Cites to IS by Decade

Accounting
Communication
Computer Science
Economics
Education
Electrical Engineering
Healthcare
Library Science
Management
Marketing
Psychology
Sociology

Page
25


Figure
18

-

Cite Comparison

References

Analysis

As an additional step, year wise trends in citations received and references given were
developed using Google motion chart. Goog
le motion chart developed using Google visualization
API, is a dynamic chart to explore several indicators over time. The chart is rendered within the
browser using Flash.

The different charts developed are as elaborated below. The charts are visualized u
sing
Microsoft Word® chart tool in this document. The Google motion charts developed have been
attached as an html do
cument in the Appendix section. Note: Viewing the Google Motion chart
requires Internet connection.

0%
10%
20%
30%
40%
50%
60%
70%
80%
0%
10%
20%
30%
40%
50%
60%
% of non
-
discipline cites by discipline

% of cites by other disciplines

Accounting
Communication
Computer Science
Economics
Education
Electrical Engineering
Healthcare
IS
Library Science
Management
Marketing
Psychology
Sociology

Page
26

Number of citations received by a disci
pline

The line chart below shows the number of citations received by each discipline from 1970
till 2009. The
citations received

count does not include the self citations received by a discipline.


Figure
19
: Number of citations
received by discipline


Psychology and Economics have been the frontrunner disciplines is citations received since
1970. In 2009, Economics and Marketing discipline received the maximum number of citations,
followed by Marketing and Psychology.



Page
27


Figure
20
: Number of citation received by MIS

MIS discipline received its first citation in the year 1976, and has over the years slowly
developed to be a reference discipline. Referring to the citations count in 2009, MIS scored 456
citations from the 12 disciplines as shown in chart above.

In 2009, MIS received citations primarily from the following disciplines (excluding self
citations).

Table
1
-

Top Citers of MIS

Education

161

Marketing

56

Healthcare

92

Management

26

Library Science

59




Number of references given by a discipline

The second chart presents the number of references given by a discipline to other
disciplines from 1970 to 2009. This
references given

count does not include the self references
made by a discipline.


Page
28

Management clearly emerges as a discipline that includes maximum references to other
disciplines since 1970, although its count of references made has seen a sharp drop in the year
2009. Ac
counting and Marketing disciplines follow close second and third respectively.


Figure
21
: Number of references given by discipline

In 2009, MIS made 422 references to other disciplines in its top journal papers. The
references gi
ven

trend for MIS is very similar to the citations received trend, as shown by the chart
below.


Page
29


Figure
22
: Number of references given by MIS

In 2009, MIS made references primarily to the following disciplines (excluding self
cit
ations).

Table
2

-

Top Reference Disciplines of MIS

Marketing

165

Economics

25

Management

157

Psychology

22


It would be interesting to analyze the comparison between citations received and
references given for each discipline to get a clearer picture.

Number of Self citations made by each discipline

The third chart presents the self citations made by each disci
pline from 1970 to 2009. It is
interesting to observe much does a discipline cite itself in the top journal papers

The Economics discipline as evidenced from the first and second chart emerged as a
discipline that receives a lot of citations but does not r
efer other disciplines much. In the chart

Page
3
0

below, it is evident that Economics cites itself in majority of its papers (top journal). Other
frontrunners in self citations include Marketing and Management disciplines.


Figure
23
:
Number of self citation by discipline

Referring the chart below (Figure
24
), MIS is one of the disciplines that makes one of the
least number of self references/citations. But, interestingly this count is higher when compared to
the references made count.
For instance in 2009, MIS made 422 references and 514 self citations.


Page
31


Figure
24
: Self citations made by MIS

Number of citations received Vs number of references made

F
igure

25

exhibits the comparison analysis of the number of cit
ations received by each
discipline to number of references given by a discipline from 1970 to 2009.


Figure
25
: Number of citations received Vs number of references made


Page
32

When observing the MIS trend from 1970
-
2009, as depicted,
MIS is seen to have
emerged as a reference discipline off
-
late with higher number of citations received. In
comparison with other disciplines such as Economics which is a major reference discipline but
does not refer other disciplines much. Compare this wi
th Accounting which refers other
disciplines more, but does not receive many citations from other disciplines. Management and
Marketing on the other hand have maintained a fair balance in the number of citations received
and references made.

Market share o
f citations received by discipline

Figure 26

shows the market share of disciplines from 1970


2009. By market share, it
is implied that of the total citations count received by all disciplines for a given year, what was
the percentage share of each discip
line. This citations received count does not include the self
citations. This chart shows how the dynamics of the citations received by a discipline has
changed over the years. Psychology a
nd Economics dominate the share

of citations received;
this corrobo
rates the observations made in earlier charts.


Figure
26



Market Share of Citations Received by Discipline


Page
33

The market share of citations received for MIS discipline
is
indicated by the teal blue bars
(highlighted by black oval)
in Figure
27
. This shows that over the years MIS has increased
significance as a referenced discipline, becoming evident from 1988 onwards.
In 2009, it stands at
7.1%.


Figure
27

-

MIS highlighted

Market share of references given
by discipline

The last chart

(Figure 28)

in this series of Google motion chart shows the market share of
references given.


Page
34


Figure
28



Market Share of References Given by Discipline

By market share of references, it is implied t
hat of the total references made by all
disciplines for a given year, what was the percentage share of each discipline. This
references made

count does not include the self citations. This chart shows how the dynamics of the references given
by a disciplin
e has changed over the years.
Management
and
Marketing dominate the share

of
references made.


Page
35


Figure
29
: MIS highlighted

The market share of
references given

for MIS discipline
is
indicated by the teal blue bars
(highli
ghted by
black oval) in Figure
29

above
. This shows that over the years
, the market share of
MIS references made to other disciplines has increased significantly. In 2009 it stands at 6.6%.


Interaction Between MIS and Other Disciplines

In this section, we will
discuss the interaction represented as citation relationship between MIS
and other non MIS disciplines. In 1970, MIS research came out as an interdisciplinary discipline that
makes citations from many other fields of study. As it evolved, the importance of

MIS studies were
gradually recognized by researchers in other fields and inspired them in the research of their own fields.
To better understand this phenomenon, we will look into the data regarding citation statistics of how
many papers MIS has cited in
other disciplines, and how many MIS papers were cited by research in
other disciplines.


Page
36

The data we have is year
-
wise citation count of how many times papers belong to one discipline
cite papers in each of the discipline the discipline collection, from 197
0 to 2010. The discipline collection
includes: MIS, accounting, communication, computer science, economics, education, electrical
engineering, healthcare, library science, management, marketing, psychology and sociology. As is often
the case, each paper in

one discipline cites papers mostly from the same discipline, which usually exceed
60% of all citations of that paper. With huge amount of self
-
discipline citations, analysis of
interdisciplinary citation will be affected when we want to know the proporti
on of citation in each
disciplinary because inclusion of self
-
discipline citation leads to large denominator. Thus we will exclude
self
-
discipline citation and use only the count of interdisciplinary citations as denominator when
calculating some proportio
ns.

Here, we will introduce three indicators to measure the interaction

between MIS and other
disciplines: MIS market share (MISMS), contribution to MIS (C2MIS) and MIS consumption (MISC). They
are defined as follows:

D: The collection of all discussed
non
-
MIS disciplines. :={

accounting, communication, computer
science, economics, education, electrical engineering, healthcare, library science, management,
marketing, psychology and sociology}

MISMS

(d)
=











































, d

∈D



MISMS indicates among the all interdisciplinary citations in one field, what perce
ntage does MIS
takes. The higher MISMS, the more importance MIS holds and gives inspiration in this field. It can be

Page
37

analogous to the market share of MIS in another field, say
d
, if we see all disciplines except
d
as
competitors to ‘sell papers’ for d to
cite.

C2MIS

(d)
=















































, d

∈D



C2MIS indicates among the all interdisciplinary citations by MIS, what percentage does one field,
say d, takes. The higher C2MIS, the more d contributes MIS by providing some ideas, theory or
methodology.

MISC

(d)
=









































, d

∈D



MISC indicates among all cited MIS papers in other fields, what percentage does MIS citation i
n
one field, say d, takes. The higher MISC, the more MIS is related to d. It can be analogous to the regional
consumption of MIS, if we see all non MIS disciplines as ‘consumers’ of MIS papers. MISC differs from
MISMS in that it measures the importance of
one field to MIS, while MISMS measures the importance of
MIS in one field. MISC can be high in one field but with low MISM in that field, as we will see later that
such is the case in marketing.

Because we used citation statistics only in top journals in
all disciplines, the data is not sufficient
to cover all publications, causing fluctuation in data representation if year wise data is used. Such being
the case, analyses will be done using 5 years as unit of time to offset the fluctuation caused by data
s
hortage. See Table
s

3
,4,5

as the values of indicators from 1970 to 2010.


Page
38

Table
3

-

MISM statistics from 1970 to 2010

MIS Market
share

Accoun
ting

Communi
cation

Computer
Science

Econo
mics

Educati
on

Electrical
Engineering

Healthc
are

Library
Science

Manage
ment

Marketi
ng

Psychol
ogy

Sociolog
y

1971~1975

0

0

0

0

0

0

0

0

0

0

0

0

1976~1980

0

0

0

0

0

0

0

0.0493

0

0

0

0

1981~1985

0

0

0

0

0

0.0387

0

0.0251

0

0

0

0

1986~1990

0

0

0.1041

0

0.0343

0.1229

0.0263

0.1861

0.0023

0.0072

0

0

1991~1995

0

0.0117

0.273
6

0.0032

0.0222

0.1054

0

0.2007

0.0066

0.0165

0

0.0010

1996~2000

0.0061

0.0264

0.2200

0.0039

0.0113

0.1174

0.0341

0.2354

0.0139

0.0290

0.0050

0.0070

2001~2005

0.0025

0.0808

0.2488

0.0042

0.0679

0.125
0

0.073
7

0.1522

0.0161

0.0364

0.0079

0.0026

2006~2010

0.0150

0.0434

0.1446

0.0100

0.2021

0.138
0

0.1453

0.1760

0.0127

0.0752

0

0.0046


Table
4

-

CMIS statistics from 1970 to 2010

Contribution
to MIS

Accou
nting

Communi
cation

Computer
Science

Econo
mics

Educati
on

Electrical
Engineering

Health
care

Library
Science

Manage
ment

Market
ing

Psychol
ogy

Sociolog
y

1971~1975

0

0

0

0

0

0

0

0

0

0

0

0

1976~1980

0

0

0

0

0

0

0

0

0

0

0

0

1981~1985

0.0714

0

0.0595

0

0

0.1071

0

0

0.1309

0.5833

0.0238

0
.0238

1986~1990

0.0494

0.0035

0.0742

0

0.0035

0.0989

0

0.0070

0.2261

0.4840

0.0424

0.0106

1991~1995

0.0351

0.0039

0.0449

0.0273

0

0.0957

0.0019

0.0175

0.3027

0.375
0

0.0839

0.0117

1996~2000

0.0288

0.0196

0.0311

0.0461

0

0.0461

0

0.0103

0.3598

0.3713

0.0668

0.0196

2001~2005

0.0226

0.0062

0.0280

0.0631

0

0.0311

0.0046

0.0062

0.3616

0.3943

0.0779

0.0038

2006~2010

0.0123

0.0041

0.0242

0.0629

0.0030

0.0170

0.0098

0.0123

0.3588

0.4057

0.0722

0.0170






Page
39

Table
5

-

MISC statistics from 1970 to 2010

MIS
Consumpti
on

Accou
nting

Communi
cation

Computer
Science

Econo
mics

Educati
on

Electrical
Engineering

Health
care

Library
Science

Manage
ment

Market
ing

Psychol
ogy

Sociolog
y

1971~1975

0

0

0

0

0

0

0

0

0

0

0

0

1976~1980

0

0

0

0

0

0

0

1

0

0

0

0

1981~1985

0

0

0

0

0

0.4545

0

0.5454

0

0

0

0

1986~1990

0

0

0.0349

0

0.1118

0.1538

0.0069

0.5244

0.0629

0.1048

0

0

1991~1995

0

0.0173

0.1805

0.0034

0.0381

0.0868

0

0.3819

0.1319

0.1562

0

0.0034

1996~2000

0.0166

0.0360

0.1274

0.0055

0.0193

0.0858

0.0221

0.3462

0.1495

0.1634

0.0083

0.0193

2001~2005

0.0075

0.0567

0.0982

0.0037

0.0850

0.0812

0.1001

0.1096

0.2211

0.2230

0.0094

0.0037

2006~2010

0.0262

0.0215

0.0155

0.0047

0.2562

0.0465

0.1463

0.1469

0.0667

0.2656

0

0.0033



Page
40

Fig
ure
30

shows the MISMS trends for each discipline. As we can see, MIS has been important
for computer science, library science and electrical engineering, and contributed much to paper in these
three fields. But it was not dominantly important because the highe
st value did not exceed 0.3. We can
also see that the importance of MIS is emerging in the discipline of education, healthcare and marketing.
Especially in education, MISMS in this field has become the highest among all other fields.

Figure

31

shows the
C2MIS trends for each discipline. As we can see, marketing and
management literature have been and are still remaining to be the highest contributors to MIs. They
together acted as dominant interdisciplinary source of citation for MIS. To our surprise, the

emerging
research that use economics model in MIS is not reflected in our data. An explanation is that they are
still too few to be published in top MIS journals.

Fig
ure

32

shows the MISC trends for each discipline. As we can see, almost all the disciplin
es
decrease in MISC, indicating MIS is becoming less important in these fields. The four disciplines with
increasing MISC are library Science, education, healthcare and marketing. MIS is becoming more and
more important to them.









Page
41


Figure
30

-

MISMS trends


Figure
31

-

C2MIS trends

0
0.05
0.1
0.15
0.2
0.25
0.3
Accounting
Communication
Computer Science
Economics
Education
Electrical Engineering
Healthcare
Library Science
Management
Marketing
Psychology
Sociology
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
Accounting
Communication
Computer Science
Economics
Education
Electrical Engineering
Healthcare
Library Science
Management
Marketing
Psychology
Sociology

Page
42


Figure
32

-

MISC trends

What Makes a MIS Article a High
-
Quality Article?

Overview

In this section, we reviewed the ISI dataset for the
purpose of answering the question “What
makes an MIS article a ‘high quality’ article? To accomplish this, we reviewed a subset of the full ISI
dataset representing articles from eight MIS journals in order to identify the most significant factors
correla
ted to “high quality” articles. For the purpose of this study we define “high quality” to mean 100
or more citations.

For this study we filtered the total ISI dataset down to subset of eight significant journals having
a primary focus on the field of info
rmation systems. The journals we selected included
Information

&

Management
,
Decision Support Systems
,
Information Systems
,
MIS Quarterly
,
European Journal of
Information Systems
,
Information Systems Research
,
Journal of Management Information Systems
,
an
d
Journal of Strategic information Systems
.

We reviewed three previous studies involving MIS journals and
0
0.2
0.4
0.6
0.8
1
1.2
Accounting
Communication
Computer Science
Economics
Education
Electrical Engineering
Healthcare
Library Science
Management
Marketing
Psychology
Sociology

Page
43

drew from these to develop this list of eight journals (
Huang, 2005; Clark et al., 2009; Lin &Gregor,
2009
).

In our
MIS
-
only
dataset,

we identify 4868 articles from the eight
journals

published from 1978 to
2005.
We further filtered the data to
4745 articles labeled with “Article

,

Editorial Material

,

Proceedings Paper

, or
“Review

. Finally, we excluded
article
s

published
in the la
st five years in order
to avoid the results being biased by articles that were so new even high quality ones may have few
citations
.

Once we identified the dataset to be studied, we created six logistic regression models
to review
the data. The first of

these models is called the “standard” model and included the following factors:
years since publication, number of references, number of authors,

and

number

of pages. We called the
second one the “standard + name” model and added the “journal name” to th
e “standard” model. We
made sure

to isolate the journal name from the “standard” model because of the high likelihood that it
would dominate the model.

We then created four “conceptual phrase” models based on the text contained in the title,
author keywor
d and ISI keyword fields in the dataset. We used text mining to identify the most
frequently occurring terms and a
computational linguistic method

to group the terms into related
conceptual phrases. To create the remaining four models we added each of the

conceptual phrase lists
to the original “standard” model. See
Table
6

for a description of all six models.

Finally we ran all six models through a logistic regression to determine the significant factors
correlated to “high quality” articles. The remain
der of this portion of the paper will describe the
procedure and the results of these analyses in detail.



Page
44

Table
6

-

Description of Six Regression Models

Model Name

Description

Standard Model

Includes y
ears since publication
, n
umber of references
,
n
umber of authors
, n
umber of pages
, and t
ype of
document

Standard + Name

Standard model plus journal name

Standard + Title

Standard model plus conceptual phrases from article title

Standard + Author Keyword

Standard model plus
conceptual phrases from author
keyword

Standard + ISI Keyword

Standard model plus conceptual phrases from ISI keyword

Standard + Title + Author + ISI

Standard model plus conceptual phrases from article title,
author keyword and ISI keyword


Advanced st
atistical analysis of “high
-
quality” MIS articles

Since we have already defined what high
-
quality MIS articles are, in the next step, we will utilize
advanced statistical tools to discover the indicators of high
-
quality articles. This question can be
answered by conducting a regression analysis. Logistic

regression has the advantage in dealing with
binary variables and providing a method to show the importance of different variables. The dependent
variable in this task is the binary variable called “quality” which has two values 1 or 0. Several groups of
variables are put into several logit models, the pseudo R square of each model is calculated and the
coefficients of variables are used to explain the correlated factors to high
-
quality MIS articles.

The 6 Basic Variables


There are some explicit variable
s we can use to build the logit models.
Year
is a discrete
variable that shows the number of years passed since the article was first published.
Number of
references

is the discrete variable that shows the number of references cited in this article by the
authors.
Number of authors

is a discrete variable showing the number of authors in a certain article.
Document Type

is the categorical variable that shows whether the article is research article, editor
material, proceeding papers or review. These are basi
c variables we can use as independent variables in
our Logit model, but they cannot explain much of the variance.
Journal

Name

is a categorical variable

Page
45

that has eight values (names of the eight MIS journals), and it is an important variable that shows in
which jou
rnal the article was published.

Table
7

-

The 6 Basic variables Used in the Logit Model

Variable

T
ype

Distribution or Values

Number years since
article published

Numeric

Min=5, Max=32, Mean=23.22, StD=
6.6426

Number of
re
ferences

Numeric

Min=0, Max=411, Mean=33.252,
StD=27.630

Number of pages

Numeric

Min=1, Max=77, Mean=14.400,
StD=7.795

Number of authors

Numeric

Min=1, Max=17, Mean=2.166, StD=1.058

Document type

Categorical

{Article, Editorial Material, Proceeding
Paper, Review}

Journalname

Categorical

{
Information & Management, Decision
Support Systems, Information Systems,
MIS Quarterly, European Journal of
Information Systems, Information
Systems Research, Journal of
Management Information Systems
,

Journal of Strategic information Systems
}

The Generation of Textual Variables

Since we have all data from the title, author keyword, ISI keyword fields for each article, we can
make good use of them by converting the unstructured textual data into stru
ctural variable
s. If we need
to convert string “title” into structural
variable
s, several steps are needed here.

In the first step
, we use
the Statistical Natural
Language Process method called
term extraction

to
extract frequently used terms

from
the
titl
e

field
.

We extracted more than 1000 terms from the titles. This list is too long to be useful,
so we grouped terms that have close meanings together in a higher
-
level semantic unit we called a
conceptual phrase
. The computational linguistic method used here is called
lexical series algorithm
,
which helps generate higher
-
level semantic units by grouping terms with similar meanings together.

Table

8

shows that “innovation adoption” and other terms such as “adopti
on of technology” are
in the same group called “adoption,” meaning that they are all about “adoption.” With this technique

Page
46

we generated 100 conceptual phrases from the titles of all the 4745 MIS articles, and 50 from author
keywords and 50 from ISI keyword
s.


Table
8

-

Terms that are within the Conceptual Phrase “Adoption”

Terms

Number of Words

Adoption

1

Innovation adoption

2

Electronic marketplace adoption

3

Electronic billing adoption

3

EIS adoption

2

EDI adoption

2

E
-
commerce Adoption

2

Adoption of technology

3

Adoption of online

3

Adoption of inter
-
organizational

3

Adoption of client
-
server

2

Adoption of determinants

3

The results of Logistic Regression Analyses

In order to show the importance of different variables toward high
-
quality MIS articles, we
conduct 6 logistic regression analyses. As showed in
Table
9
, we illustrate the six logit models we used in
which different combinations of the variables are put in
to the model as the independent variables.

Table
9

-

The Variables used in Logit Models

Model

-
2 Log/df

Cox & Snell

Nagelkerke

Standard Model

176.731/7

.037

.135

Standard + Name

407.946/14

.082

.304

Standard + Title

405.301/107

.082

.302

Standard + Author Keyword

307.701/57

.063

.232

Standard + ISI Keyword

328.428/57

.067

.247

Standard + Title + Author + ISI

659.500/207

.130

.479


By comparing the Nagelkerke

R square of all the logistic models, we can clearly know the
performance of different models. The “standard” model has an R square of 0.135, which is the lowest.
The “standard + name”
model
has an R square of 0.304, much higher than that in the “standard m
odel”.
This means that “name of the journal” can explain a lot about why the article received many citations

Page
47

and was selected as a high
-
quality article. It is common sense that high quality journals only accept high
-
quality articles. In the MIS field, the
articles published in
MIS Quarterly

and
Information Systems
Research

receives many more citations compared with those published in other journals.

The next step is to put the
journal
name
into the model, and we labeled with “standard + name”
model. The resu
lt shows that the six independent variables in “standard + name” help reduce the
-
2 log
likelihood statistic at 407.946 (degree of freedom =14, sig. = 0.0). As to the R squares, the result shows
that the Cox and Snell R Square is 0.082 and Nagelkerke R Squ
are is 0.304, significantly higher than those
in “standard” model. The estimations of coefficients and the p
-
value of all the variables are

showed in
the following table.

Table
10

-

The Logistic Regression “standard + name” model (C
ox & Snell =.082; Nagelkerke = .304)


Variable


Coefficient

Std. Error

Wald

Sig.

Intercept

-
3.12

.631

24.322

.000

Number of References

.018

.003

27.822

.000

Number of Pages

.002

.012

.029

.866

Number of Authors

.079

.084

.879

.348

Years since
published

.002

.017

.009

.923

Article Document Type

.458

.365

1.573

.210

Editorial Document Type

-
1.676

1.111

2.273

.132

Proceedings Document Type

.268

.527

.258

.612

Review Document Type

0 (benchmark)




Decision Support Systems

-
3.269

.439

55.545

.000

European Journal of Information Systems

-
4.078

1.022

15.913

.000

Information & Management

-
2.827

.334

71.766

.000

Information Systems

-
2.886

.403

51.335

.000

Information SystemsResearch

-
.506

.233

4.696

.030

Journal of Management Information
Systems

-
1.719

.355

23.511

.000

Journal of Strategic Information Systems

-
3.687

1.014

13.227

.000

MIS
Quarterly

0 (benchmark)





The result of logistic regression for the “standard + name”
model

objectively generates an
answer to the question “What is the best journal in the area of MIS.” In the model category “MIS
Quarterly” is selected as the benchmark category (with coefficient of 0). We can find that all the other

Page
48

categories have significant

(p
-
value<0.05) negative coefficients (ISR has p
-
value of 0.30, since it is a
relatively new journal). If the coefficient of the journal has a very large absolute value, it means that it is
far away from MIS quarterly as the best journal in the area of MIS
. According to the analysis, the ranks
of MIS journals could be: 1. MISQ, 2. ISR, 3. JMIS, 4. I&M, 5. IS, 6.DSS, 7. JSIS, 8. EJIS. We can compare
the ranks of MIS journals with those in other studies.

Table
11

-

The statistics of th
e number of citations of each journal

Journal

Mean

Median

Max

SDev

# of
articles

MIS QUARTERLY

60.31

31

2298

123.42
5

626

INFORMATION SYSTEMS RESEARCH

49.10

30

760

70.375

265

JOURNAL OF MANAGEMENT INFORMATION
SYSTEMS

24.48

13

463

42.278

255

INFORMATION
& MANAGEMENT

14.31

8

290

21.891

1216

JOURNAL OF STRATEGIC INFORMATION
SYSTEMS

12.86

7

112

16.584

203

EUROPEAN JOURNAL OF INFORMATION
SYSTEMS

12.55

8

285

21.242

304

DECISION SUPPORT SYSTEMS

12.17

7

174

17.088

989

INFORMATION SYSTEMS

8.68

3

197

16.885

887


Additionally, as you can see from the
T
able 11
, an article published in a high
-
quality MIS journal
such as MISQ and ISR will have higher possibility to get more citations and a greater chance to become
high
-
quality article. For instance,
626 papers
p
ublished in
MIS Quarterly

have an
average
and median
number of citations of
60.31

and 3
1

respectively
.

From this we can clearly see how the variable “
Name
of journal
” can dominate the other variables in our “standard + name” model.

According to the result
of the “standard model”, except
Number of Author

and
Document Type
,
the other three basic variables have the significant influences on the quality of an article.
Number of
references

and
number of pages
,
and

year since published
are positively related to the quality of the
articles. This follows our logical reasoning that when the article is a high quality paper, it probably has
solid support from the well
-
established theories and research models. This requires deeper literature

Page
49

r
eview into the previous studies, making the article and reference list longer. As for the year, it is
common sense that as an article ages, it will have a greater opportunity to receive additional citations
over time.

Also, neither the
number of authors,
nor any of the categories of
Document Type

havea
significant relationship wi
th the quality of the article.

Table
12

-

The Logistic Regression for “standard model” (Cox & Snell =.037; Nagelkerke = .135)

Variable

Coefficient

Std.
Error

Wald

Sig.

Intercept

-
6.505

.586

123.405

.000

Number of References

.023

.003

50.235

.000

Number of Authors

.074

.078

.889

.346

Year since published

.052

.015

12.540

.000

Number of Pages

.053

.011

24.406

.000

Article Document Type

.512

.355

2.075

.150

Editorial Document Type

-
.498

1.095

.207

.649

Proceedings Document Type

-
.041

.513

.006

.936

Review Document Type

0 (benchmark)





As to the “standard + title”, “standard + author” keyword, “standard + ISI” keyword and
“standard + title + author + ISI” models, we conduct very similar logistic regression analyses, and get the
coefficients of all the variables.
Table
13

contains a list
of the conceptual phrases that appear to have a
positive impact on the quality of an article.

Table
13

-

Conceptual Phrases That Have Positive Impact on Article Quality

Title

Weight

Author Keyword

Weight

ISI Keyword

Weight

computing

2.144

Cost

1.778

economics

2.102

building

1.742

O
nline

1.597

research

1.543

influence

1.539

computing

1.456

users

1.030

success

1.419

Q
uality

1.127

computer

0.973

commerce

1.342

U
ser

0.770

theory

0.946

research

1.075

research

0.747

strategy

0.723

theory

1.050

technology

0.730

information

0.619

technology

0.947



media

0.1551

investigation

0.933





perspective

0.744





study

0.559






Page
50


Similarly,
Table
14

below contains a shorter list of the conceptual phrases that appear to have a
negative impact on the quality of an article.

Table
14

-

Conceptual Phrases That Have Negative Impact on Article Quality

Title

Weight

Author Keyword

We
ight

ISI Keyword

Weight

support

0.939

decision

2.841

design

1.372

information

1.755





systems

1.860





impact

2.025






In both tables above, the conceptual phrases are ranked in terms of coefficients weights and
thus are sorted in order of
greatest impact. To explain this in detail, if the title of a MIS article contains
conceptual phrases called computing, no matter the actual term is “cloud computing” or “computing
algorithm,” this article will probably be a high
-
quality article. In contra
st, if the author keyword of a MIS
article contains “decision,” no matter whether the actual term is “decision making” or “decision
support,” this article will probably not be a high
-
quality article.

To analyze it more deeply, we can see that “study” “inve
stigation” and “research” are the
conceptual units that emphasize the research contributions of the article if they appear in the title, so
this might reflect the quality of the article to some extent. Similarly, “theory” “perspective” and
“building” are c
onceptual phrases that can indicate the theoretical background of the study, while
“computing” and “commerce” are talking about the research topics or domains. If we can find evidence