Download MIS and Methodology and more Study Guides, Projects, Research Management Information Systems in PDF only on Docsity!
Communications of the Association for Information Systems
Volume 11 Article 16
March 2003
Management Information Systems Research:
What's There in a Methodology?
Prashant Palvia The University of North Carolina at Greensboro , [email protected]
En Mao University of Wisconsin-Milwaukee , [email protected]
A.F. Salam The University of North Carolina at Greensboro , [email protected]
Khalid S. Soliman Hofstra University , [email protected]
Follow this and additional works at: https://aisel.aisnet.org/cais
This material is brought to you by the AIS Journals at AIS Electronic Library (AISeL). It has been accepted for inclusion in Communications of the Association for Information Systems by an authorized administrator of AIS Electronic Library (AISeL). For more information, please contact [email protected].
Recommended Citation
Palvia, Prashant; Mao, En; Salam, A.F.; and Soliman, Khalid S. (2003) "Management Information Systems Research: What's There in a Methodology?," Communications of the Association for Information Systems : Vol. 11 , Article 16. DOI: 10.17705/1CAIS. Available at: https://aisel.aisnet.org/cais/vol11/iss1/
Management Information Systems Research: What’s There in a Methodology? by P. Palvia, E. Mao,
MANAGEMENT INFORMATION SYSTEMS RESEARCH: WHAT’S THERE IN A METHODOLOGY?
PRASHANT P ALVIA
Information Systems and Operations Management Department The University of North Carolina at Greensboro [email protected]
EN M AO School of Business Administration University of Wisconsin – Milwaukee
A. F. S ALAM Information Systems and Operations Management Department The University of North Carolina at Greensboro
K HALID S. S OLIMAN Business Computer Information Systems and Quantitative Methods Department Hofstra University
ABSTRACT
Management information systems (MIS) is both a young and unique field , constantly experiencing rapid change and turmoil. Consequently, MIS research faces dual changes of rigor and relevance. Many research methodologies exist that provide various combinations of rigor and relevance. The MIS researcher selects a methodology based on several factors including rigor, relevance, subject area, and personal preferences. In this article, we examine thirteen different methodologies as used by seven leading MIS journals during a recent five-year period. The results of this extensive analysis yielded some interesting results. Survey methodology consistently ranks at the top; while frameworks and conceptual models, laboratory experiments, and case studies also found significant use among the MIS community. Several trends were observed, one of them being a greater use of the case study method and other qualitative technologies over the years. Clear patterns also emerged based on the journal itself and the subject areas. At a macro level, this information should help authors in the choice of appropriate methodologies to use in specific subject areas and in targeting manuscripts to appropriate journals. It would also be helpful to journal editors in assessing the type of research and methods being used across journals and subjects, and whether they need to make any changes in the emphases of their own journals.
Keywords: Information systems research, research methodologies, MIS journals, meta analysis
I. INTRODUCTION
Compared to many disciplines in business and social sciences, management information systems (MIS) research is relatively young, having been in existence for only about 35 years. At the same
Management Information Systems Research: What’s There in a Methodology? by P. Palvia, E. Mao,
Table 1: Methodologies in MIS research
Methodology Definition
Speculation/commentary Research that derives from thinly supported arguments or opinions with little or no empirical evidence.
Frameworks and Conceptual Models Research that intends to develop a framework or a conceptual model.
Library Research Research that is based mainly on the review of existing literature.
Literature Analysis Research that critiques, analyzes, and extends existing literature and attempts to build new groundwork, e.g., it includes meta analysis.
Case Study Study of a single phenomenon (e.g., an application, a technology, a decision) in an organization over a logical time frame.
Survey Research that uses predefined and structured questionnaires to capture data from individuals. Normally, the questionnaires are mailed (now, fax and electronic means are also used).
Field Study Study of single or multiple and related processes/ phenomena in single or multiple organizations.
Field Experiment Research in organizational setting that manipulates and controls the various experimental variables and subjects.
Laboratory Experiment Research in a simulated laboratory environment that manipulates and controls the various experimental variables and subjects.
Interview Research in which information is obtained by asking respondents questions directly. The questions may be loosely defined, and the responses may be open-ended.
Secondary Data A study that utilizes existing organizational and business data, e.g., financial and accounting reports, archival data, published statistics, etc.
Qualitative Research Qualitative research methods are designed to help understand people and the social and cultural contexts within which they live. These methods include ethnography, action research, case research, interpretive studies, and examination of documents and texts.
While not exhaustive or necessarily identical to other lists, Table 1 represents the essence of most methodologies for MIS research. Most of the definitions in Table 1 are self-explanatory; we provide a brief discussion below. Benbasat [1984] provides a comparative discussion and analysis of many of the methodologies that we consider in this article.
The first category labeled as “speculation/commentary” refers to articles/research that are not really based on any hard evidence. They largely reflect the knowledge and experience of the authors. By definition, they tend to be somewhat visionary in nature. Typically, they signal the arrival of new trends and directions in the technology, its management or application. Examples are editorials pieces published in many journals [e.g., in MIS Quarterly- Issues and Opinions section, and Information Systems Research]. On some occasions, opinion pieces reflect debates that emerge on specific topics [e.g., the rigor vs. relevance debate in MIS Quarterly and the Communications of AIS ].
Management Information Systems Research: What’s There in a Methodology? by P. Palvia, E. Mao,
“Frameworks and conceptual models” are especially useful for a discipline that generally lacks and defies attempts to develop theory. In lieu of theory, frameworks helped guide the work of many MIS researchers over the years. Some early noteworthy frameworks for the MIS discipline are by Gorry and Scott-Morton [1970], Mason and Mitroff [1973], and Ives, Hamilton and Davis (1980). Frameworks emerged even in sub-domains of MIS. For example, Sprague [1980] proposed a framework for Decision Support Systems, and Ein-Dor, Segev, and Orgad [1992] presented a work in the emergent area of global information systems.
The difference between library research and literature analysis needs some clarification. Library research (which is also part of most of the other methodologies) summarizes and synthesizes past research, and highlights some of the important conclusions. Some journals [e.g., ACM Computing Surveys , Communications of the ACM ] would publish such work to provide a good synopsis of a certain area. However, literature analysis as we define it, examines many (perhaps all) past studies in a particular area and conducts a scientific meta analysis of the cumulative knowledge, in effect treating each study as one data point. An example of such research is by Alavi and Joachimsthaler [1992] where they conducted meta analysis of DSS implementation research.
Over the years, the “survey” method was extensively used in MIS research and is still in predominant use. It appeared to be suitable to descriptive studies characteristic of the 1970s and 1980s. While the method can attain high levels of external validity, it is known to suffer from lack of control and internal validity. It has come under attack and researchers were urged to use alternate methods. Pinsonneault and Kraemer [1993] conducted an assessment of survey-based studies based on 144 articles and identified five major weaknesses: single-method designs where multiple methods were needed, unsystematic and inadequate sampling procedures, low response rates, weak linkages between units of analysis and respondents, and over-reliance on cross- sectional surveys where longitudinal surveys were really needed. Kraemer and Dutton [1991] investigated three charges against survey research, i.e.,
- it is unable to yield cumulative knowledge,
- it is atheoretical, and
- it is ill-suited to address the subtleties of IT in complex settings.
Lyytinen [1999] contrasts the views of North American IS scholars with those of European scholars on such empirical research. Lyytinen [1999, pp. 26] states that “Too often North American IS researchers want to work with specific research solutions that are looking for problems… Few engage in systematic attempts to solicit problems…” Thus research based on survey methodology should be evaluated in light of the identified weaknesses and furthermore, researchers should take proper measures to ensure that survey methodology is not being used under inappropriate contexts.
“Case study” is a methodology that seems to be getting wider acceptance over the last decade. This inference was evidenced by the appointment of Allen Lee, a strong proponent of the case method, as the Editor-in-Chief of MIS Quarterly. Many of the articles published in the “Application” section of MIS Quarterly are in fact cases. Information & Management regularly publishes case studies. Many of the other leading journals also publish case studies. There is now even a new journal on case-based research: Journal of Information Technology Cases & Applications. Case studies allow the opportunity to study a single phenomenon in much depth, typically in an organizational setting. Generally, it is credited with much internal validity. Lee [1989] provides a succinct description of this “scientific” methodology and argues that case study research can have as much rigor as quantitative research. The difference between a case study and a field study is less clear, and some may use the terms interchangeably. For our purposes, a case study generally refers to the in depth study of a single phenomenon (e.g., one application, one technology) over time in a single organization. On the other hand, a field study can be broader, i.e., it may study several related phenomena or processes in multiple organizations and it may be cross-sectional or longitudinal.
Management Information Systems Research: What’s There in a Methodology? by P. Palvia, E. Mao,
Table 2. Selected MIS Journals Used in the Study
- Communications of the ACM (CACM)
- Decision Sciences (DS)
- Information & Management (I&M)
- Information Systems Research (ISR)
- Journal of Management Information Systems (JMIS)
- Management Science (MS)
- MIS Quarterly (MISQ)
All articles, published between 1993 and 1997 in these journals, were screened. Following the procedure outlined by Grover, Lee and Durand [1993], MIS and related articles were selected by examining the title for information systems keywords. A total of 843 articles were selected, reviewed and coded using content analysis [Weber, 1990].
Six MIS doctoral students under the supervision of an MIS faculty member were involved in the data collection. To divide work and ensure uniform evaluation, each reviewer (coder) was assigned an equal number of articles to evaluate and was asked to code articles on a special coding sheet. This approach amounts to structured content analysis as described by Smith et al. [1991]. The enormity of the task was such that it took more than a year to code all of the 843 articles in the study.
Articles were coded according to the Barki, Rivard, and Talbot [1988] classification scheme. The classification scheme presents the most comprehensive classification of MIS topics and was used in previous studies [e.g., Alavi and Carlson, 1992]. The classification list contains seven levels. The first level represents the broadest topic classification while each lower level incrementally refines the topic. The three top levels of the scheme were selected as the base for the subject classification in this study. Continual developments in information technology broadened the scope of MIS to include subjects that were not listed in the Barki, Rivard, and Talbot classification. Accordingly, several topics were added to come up with the final subject classification list for this study (Table 3).
Each article may deal with multiple subjects and may employ multiple methodologies. Therefore, the coding sheet that each reviewer used allowed for up to three different subjects per article and up to two different research methodologies. The methodology classification shown earlier in Table 1 was adopted and used to code the methodologies for the articles. Because of possible multiple subjects and multiple methodologies per article, the total subject count was 1579 and the total methodology count was 1031.
To ensure uniformity of coding and to reduce coding ambiguity, the coders were trained in the coding method. In these training sessions coders met and discussed the subject areas and research methodologies to be used in the coding process. Each coder was then required to code independently the same set of articles. A discussion was held based on individual coding outcomes and a consensus was reached regarding the final coding scheme. The coders coded a second set of articles and the results were identical in each instance, achieving very high inter- rater reliability. This method ensured that the coders were properly trained in the coding method
Management Information Systems Research: What’s There in a Methodology? by P. Palvia, E. Mao,
Table 3: Subjects or Topics Classification
- Theory of MIS
- Artificial Intelligence (AI) /Expert System (ES) / Neural Networks (NN)/ Knowledge management (KM)
- Global Information Technology (GIT)
- Hardware
- Software / Programming Languages
- Networks / Telecommunications
- Internet
- Electronic Data Interchange (EDI)
- Electronic Commerce (EC)
- Multimedia
- Databases / DBMS
- Internal / External Environment
- Organizational Design / Business Process Reengineering (BPR)
- Innovation
- Resource Management / IS Management issues
- IS Planning
- IS staffing
- IS Evaluation / Control
- Security
- IS Development / Methods and Tools
- IS Implementation
- IS Usage
- End User Computing (EUC)
- Executive Information Systems (EIS)
- Decision Support Systems (DSS)
- Group Decision Support Systems (GDSS)
- IS Function Applications
- IS Education
- IS Research
prior to embarking on coding the entire range of articles this study. The method ensured that they all had a common thread of understanding of the terms (i.e., the subject areas and the research methodologies) in the coding scheme, thereby significantly eliminating ambiguity from the coding process. Such high agreement between the coders was achieved because they were all advanced doctoral students in MIS and understood very well the content and the methodologies covered in the articles. Given this level of agreement and to contain the duration of the project, each article was then coded by only one coder. Still, it took more than a year to code all of the 843 articles.
Management Information Systems Research: What’s There in a Methodology? by P. Palvia, E. Mao,
qualitative research. Moreover, qualitative research was more frequently used as a secondary methodology than the primary methodology.
METHODOLOGY USAGE TRENDS
Analyzing data year-by-year during 1993-97, we found some interesting trends. Overall, results show that survey, frameworks/conceptual models, and laboratory experiments are the most frequently used research methodologies. However, through the years, some methodologies become more frequently used while others fall in usage (Figures 1 and 2).
0
10
20
30
40
50
60
70
93 94 95 96 97 Year
Count
Speculation/comme ntary Frameworks and Conceptual Models Library Research
Literature Analysis
Case Study
Survey
Field Study
Figure 1. Methodology Usage Trends
0
5
10
15
20
25
30
35
40
93 94 95 96 97 Year
Count
Field Experiment
Laboratory Experiment Mathematical Model
Qualitative Research Interview
Secondary data
Figure 2. Methodology Usage Trends
Management Information Systems Research: What’s There in a Methodology? by P. Palvia, E. Mao,
Survey methodology holds its own and remained the strongest research methodology throughout. It is ranked 2 nd^ in 1993 and then first from 1994 through 1997. Although it was not the top one in 1993, it had a high percentage of use nevertheless (19.0%). “Framework/conceptual model” based research declined continually. While it was the highest used in 1993, it dropped to second place in 1994, fourth in 1995, third in 1996, and finally sixth place in 1997. Apparently, journals wanted to publish actual research more than frameworks that guide research.
On the contrary, the case study method became popular over the years. Starting with sixth and fifth rankings in 1993 and 1994, it moved to second place in 1996 and third place in
- Another gain is in speculation/commentary, which came out strong, ranking second in
- These trends are perhaps a consequence of the constant inflow of emerging information technologies and a lack of theoretical groundwork to guide research in these new areas.
Laboratory experiments remained in the top three ranks except in 1996, when it dropped just slightly below to fourth place. When manipulation of the independent variables is desired, IS researchers favor the laboratory experiment methodology because of the control if affords. Library research, which is based primarily on literature review alone, became less predominant over the years. This trend shows that MIS research is moving towards maturity and that more sophisticated methodologies are being used increasingly.
Speculation and case study are being utilized more often in the later years of our study period. Once again, these trends are an indication of the newer technologies that spring up and the lack of attendant theories to study them. In general, library research, literature analysis, field experiments, laboratory experiments, and mathematical model are less frequently used. The trends of the high use of the survey method and lesser use of the field study method are generally stable over the years. Qualitative research was rarely used throughout these years.
MIS SUBJECT TRENDS
The most-written about MIS topic is Resource Management/IS Management issues, followed by Theory of MIS and IS Evaluation/Control in third and fourth places (Table 5). This finding is consistent with the views of many in the academic field that equate MIS to “Management of Information Systems” and focus primarily on management and control issues of Information Systems. In second place came IS development and in fifth place came networks/telecommunications, both of which have been (and continue to be) important issues for quite some time.
Subjects in the middle of the list in Figure 5 appear to be receiving at least a moderate amount of attention. What is interesting to note are the subjects near the bottom of the list. It seems that some important issues current then were not very well represented. For example, EIS, global IT, electronic commerce, and Internet were listed among the last. Of these, interest in EIS (Executive Information Systems) peaked and underlying EIS topics are being addressed by other areas, such as data warehousing, data mining, and Online Analytic Processing (OLAP). The subject of Global IT really became prevalent in the 1990s and there now seems to be a steady stream of research activity in it, especially with two journals devoted specifically to global IT topics.
Management Information Systems Research: What’s There in a Methodology? by P. Palvia, E. Mao,
Table 6: Subject Trends between 1993-
Subject 1993 1994 1995 1996 1997 Total Resource Management/ IS Management Issues 19 29 23 43 38 152 IS Development /Methods and Tools 35 33 16 22 37 143 Theory of MIS 14 46 38 15 19 132 IS Evaluation/ Control 19 12 13 27 32 103 Networks/ Telecommunications 7 30 9 12 30 88 EUC 20 16 25 14 12 87 IS Functional Applications 18 24 7 9 29 87 Internal/ External Environment 11 19 7 17 32 86 IS Research 11 23 15 11 20 80 AI/ES/ ANN/ Knowledge Management 20 14 17 11 6 68 DSS 24 16 10 8 9 67 IS Planning 11 21 4 12 13 61 Software/Programming Language 7 15 11 9 15 57 GDSS/ GSS/ Collaborative Systems 8 12 11 11 5 47 Organizational design/ BPR 6 13 1 11 13 44 IS Implementation 10 15 2 8 6 41 IS Usage 11 7 3 9 7 37 DBMS/databases 6 11 7 5 5 34 Innovation 2 8 1 6 13 30 IS Staffing 4 4 9 4 5 26 Security 5 3 3 3 5 19 IS Education 0 2 8 5 4 19 Global IT 3 5 1 1 4 14 EIS 2 0 6 2 4 14 Electronic Commerce 1 6 4 1 1 13 Hardware 3 3 1 3 2 12 Multimedia 0 3 3 1 5 11 Internet 0 0 0 4 3 7
SUBJECT BY METHODOLOGY
Insights can be obtained by examining the use of methodologies by subject areas. It was evident that some methodologies are used more heavily than others. Moreover, the chi-square (χ^2 ) test of independence between two factors was used to assess whether the differences in methodology usage across subjects are statistically significant. The χ^2 statistic (χ^2 = 826.12, p = .0000) was significant and therefore indicates that methodology use across subjects is not homogeneous. The differences are not random and are attributable to definite usage preferences.
To show the usage patterns clearly and prominently, we decided to select the top fifteen subject areas of research in IS based on the frequency ranking in r Table 5 and the subject areas that are comparatively more recent and are in early stages of development. The rationale was that it will allow us to examine research methodologies that are used in more mature subjects as well as in up- and-coming areas of interest. Table 7 shows the frequency of each methodology appearance by subject area. For easier interpretation, we also provide the top three methodologies for each subject area, listed as Roman numeral I, II, and III.
Some comments are in order. The following areas used survey research methodology the most: Resources management/IS management issues, IS functional applications, IS planning, IS usage, IS staffing, IS implementation, End-user computing, IS (meta) research, Internal/external
Communications of the Association for Information Systems (Volume 11, 2003) 289-
301
Management Information Systems Research: What’s There in a Methodology? by P. Palvia, E. Mao,A.F. Salam, and K.S. Soliman
Table 7: Subject by Methodology
Spec./Comm.
MathModel
Qual.Res.
Interview
Sec.data
Frame/conceptModel
LibraryRes.
Lit. Analysis
CaseStudy
Survey
FieldStudy
FieldExp.
Labora-tory Expt.
Resource Management/ IS
Management Issues
10
9
13
5
(II) 29
6
10
(III) 23
(I) 53
15
3
12
IS Development /Methods and
Tools
9
9
6
8
(I) 39
6
7
22
(II) 37
8
6
(III) 25
Theory of MIS
6
17
6
2
(II) 25
5
18
10
(I) 39
8
6
(III) 23
IS Evaluation/Control
5
11
2
6
5
(III) 13
1
6
(II) 16
(I) 34
12
1
9
Networks/ Telecommunications
9
5
3
2
(I) 24
2
4
(III) 13
(II) 23
7
1
12
EUC
7
7
3
(II) 16
2
6
5
(I) 43
5
4
(III ) 14
IS Functional Applications
(III) 15
7
1
4
4
13
1
3
(II ) 20
(I) 24
6
1
7
Internal/ External Environment
10
4
1
5
4
(II) 16
4
7
(III) 11
(I) 28
5
5
IS Research
(III) 17
2
1
5
3
10
5
(II) 18
1
(I) 25
3
1
7
AI/ES (under types of sys)/
ANN/ Knowledge Mgt.
5
(I) 20
3
1
(III ) 11
3
3
(IV) 6
(II) 14
5
5
(II) 14
DSS
0
10
3
2
(II) 16
5
8
(III) 11
3
(I) 28
GDSS/GSS/Collaborative
Systems
0
3
2
3
(II) 10
3
1
2
3
(III) 5
(I) 29
IS Planning
3
(III) 10
5
5
8
6
(II) 12
(I) 25
6
1
Software/Programming
Language
(III) 9
(III) 9
2
(II) 12
1
1
6
6
7
2
(I) 15
Organizational Design/BPR
2
2
3
6
2
(III) 7
(I) 19
(II) 9
4
IS Implementation
1
1
2
2
(III) 5
3
4
(II) 7
(I) 15
(III) 5
(II) 7
4
Global IT
0
1
(II) 3
1
(III) 2
(III) 2
(I) 7
Electronic Commerce
0
(III) 1
(III) 1
(III) 1
(III) 1
(I) 6
(II) 4
(III) 1
Internet
(III) 1
(III) 1
(I) 3
(II) 2
-^
Spec/Comm: Speculation/ commentary, Math Model: Mathematical Model, QualRes: Qualitative Research, Sec. Data: Secondary Data, Frame /concept Model:Framework/Conceptual Model, Library Res.: Library Research, Lit. Analysis: Literature Analysis, Field Exp.: Field Experiment.
-^
Note: 1, 2, 3… represent the count of methodology per subject; I, II, and III represent the rank of the methodology for the subject
Communications of the Association for Information Systems (Volume 11, 2003) 289-
303
Management Information Systems Research: What’s There in a Methodology? by P. Palvia, E. Mao,A.F. Salam, and K.S. Soliman
Table 8: Rank of Methodologies Used in Journals (1993 – 1997)
MISQ
Information &Management
JMIS
DecisionSciences
ISR
CACM
Management
Science
Methodology
Rank
%
Rank
%
Rank
%
Rank
%
Rank
%
Rank
%
Rank
%
Survey
1
1
1
2
3
3
2
Frameworks/ConceptualModels
4
2
2
4
4
2
1
Laboratory Experiment
6
5
4
1
1
5
3
Case Study
2
3
3
9
5
4
8
Mathematical Model
12
10
5
3
2
7
4
Speculation/Commentary
10
8
11
8
10
1
11
Literature Analysis
5
4
6
5
6
9
7
Field Study
3
6
7
7
9
8
6
Interview
9
7
8
11
7
10
9
Library Research
7
9
10
12
12
6
10
Secondary Data
13
12
9
6
8
11
5
Field Experiment
8
11
12
10
11
12
12
Qualitative Research
11
13
13
13
13
13
13
Management Information Systems Research: What’s There in a Methodology? by P. Palvia, E. Mao,
mathematical models at second or third rank. It is noteworthy that the survey method still comes in at a respectable second or third rank in both journals.
The third category termed “conceptual” includes Communications of the ACM and Management Science. CACM published predominantly research utilizing speculation/commentary methodology as well as frameworks and survey research. In a similar vein, Management Science publishes research that put more focus on developing frameworks and conceptual models, however it also publishes more quantitative research than CACM.
VI. DISCUSSION
LIMITATIONS
We first state some limitations of the study before discussing the results. The primary limitation of the study stems from the period (1993-1997) that was selected for the study. Whether articles published in other periods exhibit similar trends cannot be stated conclusively. However, this time-period is recent and the usage patterns of methodologies over this period should at least have a reasonable degree of correlation with the present. Especially, the time trends should tell us about the direction we are moving.
Another limitation is that we targeted only seven journals in the field. Although highly acclaimed by IS professionals and some of the best research being published in these journals, there are other specialty niche journals such as Decision Support Systems , Journal of Global Information Technology Management, and the various IEEE Transactions. These journals have their own preferences in topics and methodologies.. Thus our remarks should be generalized to the entire MIS research with some caution.
Finally, a methodological concern of this study is that each article was coded by only a single coder. But, given the caution, care, and formal procedures used by the coders, this limitation does not diminish the findings or the value of the study significantly.
THE RESULTS
Despite the limitations, one heartening conclusion that can be drawn is the significant maturity of the MIS field. Over the years, researchers in MIS used a variety of research tools to investigate the diverse phenomena that arose because off the complex interplay between the rapid developments in IT and the changing business environment. In this study alone, we were able to investigate the use of thirteen different methodologies for 29 different IS subject areas.
Among the thirteen different methodologies, survey research was found to be the most-widely used. This outcome is not totally surprising; survey methodology was probably employed to maximize the generalization of results at the cost of lower realism of context and precision of measurement [Scandura and Williams, 2000]. As in a new field, there may be a tendency to find external conclusion validity of a particular theory rather than the refinement of that theory in greater depth or in a particular context. Consequently, studies based on surveys should be evaluated based on the appropriateness of the context as well as how effectively the researchers addressed the weaknesses that accompany this type of research. Frameworks and conceptual models took second place. The newness of the field and lack of formal theory most likely motivated researchers to develop frameworks/conceptual models to anchor the basic concepts and guide further research. In addition, the second position of framework and conceptual models (in lieu of theories) also bodes well in comparison to more mature and well-established sister disciplines such as management where formal theory also takes second place [Scandura and Williams, 2000]. Laboratory experiments and case study took 3 rd^ and 4 th^ positions respectively. It will be interesting to see how case study and other qualitative methodologies will fare in the long run given the current broadening interest in this methodology.
Management Information Systems Research: What’s There in a Methodology? by P. Palvia, E. Mao,
In spite of the different levels of diversity in journals, it appears that most journal editors maintained a good balance among the top methodologies (i.e., survey, frameworks and conceptual models, case study and laboratory experiment) between 1993 and 1997. For example, MISQ showed a healthy balance between survey (22.1%), case study (15.7%), field study (10.7%) and frameworks and conceptual models (8.6%) among the articles published between 1993 and 1997. Similar trends can be seen in the case of JMIS. In addition, journals such as ISR maintained a sound equilibrium among such methodologies as laboratory experiment (21.1%), mathematical model (21.1%), survey (15.5%), and frameworks and conceptual models (12.8%). Thus while journals seem to exhibit methodological preferences, they are not exclusive to any single methodology. God research can be published anywhere irrespective of the methodology employed. This is again a sign of a maturing field in that it can appreciate and tolerate diverse research ‘lenses’ for investigating important and interesting problems.
VII. CONCLUSION
The continual self-introspection by any field is useful for it to mature and thrive. This is particularly true in MIS due to the youngness of the field and the explosive growth in the technology itself. Our analysis of publication patterns and trends in leading MIS journals in the years 1993 to 1997 provide but one snapshot of the state of MIS research. It is evident that similar efforts need to be undertaken in MIS not only to establish and verify research trends but also to verify the cumulative findings and their relevance to the practice of MIS.
Besides knowing the current state of research methodologies in use, there are some immediate implications of this study for both authors and journal editors. Authors are made aware of what methodologies and what subject domains are in wide use and the preferences of journals for the two. At least at a macro level, this information should help authors in the choice of appropriate methodologies to use in specific subject areas. It should also help them in targeting existing or future manuscripts to appropriate journals. Hopefully, the editors are aware of the journals’ methodological and subject preferences; our study should give them corroborating evidence. It is expected that these preferences are by explicit editorial design and not by accident. In any case, if the evidence does not match the design expectations, the editors should be able to take corrective action.
Editor’s Note: This article was fully peer reviewed. It was received on April 26, 2002 and was with the author 3 months for two revisions. The article was published on February 27, 2003
REFERENCES
Alavi, M., and E. Joachimsthaler (1992) "Revisiting DSS Implementation Research: A Meta- Analysis of the Literature and Suggestions for Researchers," MIS Quarterly (16)1, pp. 95 –116.
Alavi, M., and P. Carlson (1992) "A review of MIS research and disciplinary development," Journal of Management Information Systems, (8)4, pp. 45.
Barki, H., S. Rivard, and J. Talbot (1993) "A Keyword Classification Scheme for IS Research Literature: An Update," MIS Quarterly , June, 209-
Benbasat, I. (1984) "An Analysis of Research Methodologies," In McFarland F. W. (ed.) (1984) The Information Systems Research Challenges , Boston, MA: Harvard Business School Press, pp. 47-88.
Culnan, M. and E. Swanson (1986) "Research in Management information Systems, 1980-1984: Points of work and reference," MIS Quarterly (7)3, pp. 156-172.
Davenport, T. and M. Markus (1999) "Rigor vs. relevance revisited: Response to Benbasat and Zmud," MIS Quarterly (23)1, pp. 19-23.
Management Information Systems Research: What’s There in a Methodology? by P. Palvia, E. Mao,
Ein-Dor, P., E. Segev, and M. Orgad (1993) “The Effect of National Culture on IS: Implications for International Information Systems,” Journal of Global Information Management , (1)1, pp. 33-44.
Fjermestad, J., and S. Hiltz (1999) "An assessment of group support systems experiment research: Methodology and results," Journal of Management Information Systems (15)3, pp. 7-
Gillenson, M. and D. Stutz (1991) "Academic issues in MIS: Journals and books," MIS Quarterly (15)4, pp. 447-452.
Gorry, G. and M. Scott-Morton (1970) “A Framework for Management Information Systems,” Sloan Management Review , (13)1, pp. 55-70.
Grover, V.; C. Lee, and D. Durand (1993) "Analyzing methodological rigor of MIS survey research from 1980-1989," Information & Management , (24)6, pp. 305-317.
Ives, B., S. Hamilton, and G. Davis (1980) “A Framework for Research in Computer-Based Management Information Systems,” Management Science , (26)9, pp. 910-934.
Jarvenpaa, S., G. Dickson, and G. DeSanctis (1985) “Methodological Issues in Experimental IS Research: Experiences and Recommendations,” MIS Quarterly , pp. 141-156.
Kaplan, B., and D. Duchon (1988) "Combining Qualitative And Quantitative Methods In Information Systems Research: A case study," MIS Quarterly (12)4, pp. 571-586.
Klein, H. and M. Myers (1999) "A set of principles for conducting and evaluating interpretive field studies in information systems," MIS Quarterly (27)1, pp. 67-88.
Kraemer, K. and W. Dutton (1991) "Survey Research in the Study of Management Information Systems," In Kraemer K. (ed.) (1991) The Information Systems Research Challenge: Survey Research Methods , Boston, MA: Harvard Business School Press, pp. 3-57.
Lee, A. (1989) "A Scientific Methodology For MIS Case Studies," MIS Quarterly (13)1, pp. 33-50.
Lee, A. (1999) "Rigor and relevance in MIS research: Beyond the approach of positivism alone," MIS Quarterly (23)1, pp. 29-33.
Lyytinen, K. (1999) "Empirical research in information systems: On the relevance of practice in thinking of IS research," MIS Quarterly (23)1, pp. 25-27.
Markus, M. and A. Lee (1999) "Special issue on intensive research in information systems: Using qualitative, interpretive, and case methods to study information technology--foreword," MIS Quarterly (23)1, pp. 35-38.
Mason, R. (1989) "MIS Experiments: A Pragmatic Perspective," In Benbasat, I. (ed.) (1989) Information Systems Research Challenge: Experimental Research Methods , Boston, MA: Harvard Business School Press, pp. 3-20.
Mason, R., J. McKenney, and D. Copeland (1997) "An historical method for MIS research: Steps and assumptions," MIS Quarterly (21)3, pp. 307-320.
Mason, R. and I. Mitroff. (1973) “ A program for research on management information systems,” Management Science , 19(5), pp. 475-487.
Pinsonneault, A. K. and Kraemer (1993) "Survey research methodology in management information systems: An assessment," Journal of Management Information Systems (10)2, pp.
Sakaguchi, T., P. Palvia, and B. Janz (2001) “Business Practices on the World Wide Web: A Comparison of Japanese and U.S. Web Sites,” Journal of Global Information Technology Management, (4)2, pp. 5-21.
Scandura, T., and E. Williams (2000) “Research methodology in management: Current practices, trends, and implications for future research,” Academy of Management Journal , (43)6, pp. 1248-
