


Study with the several resources on Docsity
Earn points by helping other students or get them with a premium plan
Prepare for your exams
Study with the several resources on Docsity
Earn points to download
Earn points by helping other students or get them with a premium plan
An assignment for a graduate-level computer science course, where students are required to re-read three research papers, identify gqm goals, list models and metrics, and write new abstracts. They must also evaluate the studies based on context guidelines and write critiques. Preliminary guidelines for empirical research in software engineering are provided.
Typology: Assignments
1 / 4
This page cannot be seen from the preview
Don't miss anything!



The goal of this assignment is to make you a more informed/critical reviewer of papers in the literature, and to give you some experience applying GQM.
This is a follow-up to the annotated bibliography project. Each of you will re-read your three papers from assignment 1 and write a set of GQM goals that represent the basic study of the work reported in the paper, list the set of models used with definition of terms, and give the metrics or data collected and analyzed. You will state as many context variables as you can.
Then you should evaluate the study based upon its consistency in making clear its goals, providing sufficient information to do the analysis, and interpreting the results that they provided in the paper. If the paper is an empirical study of some kind, you might consider some of the criteria on context guidelines from Kitchenham, et.al., “Preliminary Guidelines for Empirical Research in Software Engineering, IEEE TSE August 2002 (guidelines attached)
Based upon this analysis, you should generate a new abstract.
For each abstract you write you should fill in a set of goal templates (Analyze O for the purpose of X with respect to M from the point of view of P in the context of C), if possible, and assure the conforms to the requirement, i.e., the abstract should make clear the entity being studied, (i.e., the process, product, model, metric,...), the attributes of the entities that are of interest, the purpose of the study, (i.e., whether the study is aimed at characterizing, understanding, evaluating, predicting, or improving), and for who the study should be of value, (i.e., a researcher, project manager, corporation,...). The context should also be clear, i.e., a company, the relevant characteristics of the company, if it is a class project the assumed background of the participants.
Note whatever information is important to understand the model or metric, e.g., certain definitions, environmental characteristics, information about process conformance, underlying models, etc., should be in the abstract. Each abstract should be between 250 and 300 words.
As a separate activity, write a critique of the paper from the point of view of its success in expressing sufficient information for you to understand the results in context and/or replicate the experiment, selecting several items from the attached guidelines.
Send your paper to Cathy Sinex ([email protected]).
Preliminary Guidelines for Empirical Research in Software Engineering Kitchenham, et.al., IEEE TSE August 2002
Context Guidelines:
C1: Be sure to specify as much industrial context as possible. In particular, clearly define the entities, attributers, and measures that are capu5ring the contextual information.
C2: If a specific hypothesis is being tested, state clearly prior to the study and discuss the theory from which it is derived, so that its implications are apparent.
C3: If the research is exploratory, state clearly and, prior to data analysis, what questions the investigation is intended to address and how it will address them.
C4: Describe research that is similar to, or has a bearing on, the current research and how current work relates to it.
Experimental Design:
D1: Identify the population from which the subjects and objects are drawn.
D2: Define the process by which the subjects and objects were selected.
D3: Define the process by which the subjects and objects are assigned to treatments.
D4: Restrict yourself to simple study designs or, at least, to designs that are fully analyzed in the statistical literature. If you are not suing a well-documented design and analysis mthod, you should consult a statistician to see whether yours is the most effective design for what you want to accomplish.
D5: Define the experimental unit.
D6: For formal experiments, perform a pre-experiment or recalculation to identify or estimate the minimum required sample size.
D7: Use appropriate levels of blinding.
D8: If you cannot avoid evaluating your own work, then make explicit any vested interests (including your sources of support) and report what you have done to minimize bias.
D9: Avoid the use of controls unless you are sure the control situation can be unambiguously defined.
D10: Fully define all treatments (interventions).
P4: Present the raw data whenever possible. Otherwise, confirm that they are available for confidential review by the reviewer independent auditors.
P5: Provide appropriate descriptive statistics.
P6: Make appropriate use of graphics.
Interpretation of Results:
I1: Define the population to which inferential statistics and predictive models apply.
I2: Differentiate between statistical significance and practical importance.
I3: Define the type of study.
I4: Specify any limitation to the study.