Study with the several resources on Docsity
Earn points by helping other students or get them with a premium plan
Prepare for your exams
Study with the several resources on Docsity
Earn points to download
Earn points by helping other students or get them with a premium plan
An in-depth exploration of case studies in empirical software engineering. It covers the definition of case studies, their types (causal and exploratory), study propositions, unit of analysis, and the process of generalizing findings from a case study to theory. The document also discusses analytic and statistical generalization, data analysis strategies, and the characteristics and benefits of case study analysis.
Typology: Assignments
1 / 55
This page cannot be seen from the preview
Don't miss anything!
382C Empirical Studies in Software Engineering
Lecture 6
© 2000-present, Dewayne E Perry © 2000-present, Dewayne E Perry
Adapted from Perry, Sim & Easterbrook,Case Studies for Software Engineering, ICSE 2004 Tutorial
382C Empirical Studies in Software Engineering
Lecture 6
It is not a subset or variant of other methods, such asexperiments, surveys or historical study.
Phenomena under study cannot be separated from context.Effects can be wide-ranging. How and why questions
Days, weeks, months, or years rather than minutes or hours.
382C Empirical Studies in Software Engineering
Lecture 6
Example: To understand the capability of a new tool Example: To identify factors affecting communication in codeinspections Example: To characterize the process of coming up to speedon a project
Exploration- To find what’s out there Characterization- To more fully describe Validation- To find out whether a theory/hypothesis is true
An intervention, e.g. tool, technique, method, approach todesign, implementation, or organizational structure An existing thing or process, e.g. a team, releases, defects
382C Empirical Studies in Software Engineering
Lecture 6
Strategy
Form of Research
Question
Requires Controlof Behavioral
Events?
Focuses oncontemporaryevents?
Experiment
How, why?
Yes^
Yes
Survey
Who, what, where,how many, how much?
No^
Yes
ArchivalAnalysis
Who, what where,how many, how much?
No^
Yes/No
History
How, why?
No^
No
Case Study
How, why?
No^
Yes
382C Empirical Studies in Software Engineering
Lecture 6
Investigates a contemporary phenomenon within its real-lifecontext, especially when The boundaries between phenomenon and context are notclearly evident.
Copes with the technically distinctive situation in which therewill be many more variables of interest that data points, andas one result Relies on multiple sources of evidence, with data needing toconverge in a triangulating fashion, and as another result Benefits from the prior development of theoreticalpropositions to guide data collection and analysis.
382C Empirical Studies in Software Engineering
Lecture 6
Parts of a Case Study Research Design
Deals more with the logic of the study than the logistics Plan for moving from questions to answers Ensures that the data is collected and analyzed to producean answer to the initial research question Strong similarities between a research design and a systemdesign
Research questions 2. Propositions (if any) 3. Unit(s) of analysis 4. Logic linking the data to the propositions 5. Criteria for interpreting the findings
382C Empirical Studies in Software Engineering
Lecture 6
^ Explanatory^
Adjudicates between competing explanations Example: How important is implementation bias in requirementsengineering?^ ¾^ Rival theories: existing architectures are useful for anchoring, vs.existing architectures are over-constraining during RE ^ Descriptive^
Describes sequence of events and underlying mechanisms Example: How does pair programming actually work? Example: How do software immigrants naturalize? ^ Causal^
Looks for causal relationship between concepts Example: Requirements errors are more likely to cause safety-related defects than programming errors are^ ¾^ See study by Robyn Lutz on the Voyager and Galileo spacecraft ^ Exploratory^
Criteria or parameters instead of purpose Example: Christopher Columbus’ voyage to the new world Example: What do CMM level 3 organizations have in common?
382C Empirical Studies in Software Engineering
Lecture 6
Example: “Organizations collaborate because they derivemutual benefits”
Example: Define and ascertain the specific benefits to eachorganization
Note: Even exploratory studies should have both clearly-stated purposes and clearly-stated criteria for success
382C Empirical Studies in Software Engineering
Lecture 6
Individuals Development team Organization
Programming episode Pairs of programmers Development team Organization
Modification report File System Release Stable release
382C Empirical Studies in Software Engineering
Lecture 6
Describe several potential patterns, then compare the casestudy data to the patterns and see which one is closer
382C Empirical Studies in Software Engineering
Lecture 6
Generalizing from Case Study to Theory
382C Empirical Studies in Software Engineering
Lecture 6
Analytical and Statistical Generalization
Figure 2.2 Making Inferences: Two Levels
382C Empirical Studies in Software Engineering
Lecture 6
Avoid thinking in terms of samples when doing case studies
382C Empirical Studies in Software Engineering
Lecture 6
Concepts being studied are operationalized and measuredcorrectly
Establish a causal relationship and distinguish spuriousrelationships
Establish the domain to which a study’s findings can begeneralized
Demonstrate that the study can be repeated with the sameresults
