Download Research and Program Evaluation Ultimate Exam and more Exams Technology in PDF only on Docsity!
Evaluation Ultimate Exam
Question 1. Which philosophical stance asserts that reality exists independently of human perception and can be measured objectively? A) Constructivism B) Pragmatism C) Positivism D) Critical Theory Answer: C Explanation: Positivism holds that an objective reality exists and can be studied through empirical observation and measurement. Question 2. In the context of research, the primary distinction between basic and applied research is: A) The use of qualitative versus quantitative methods B) The focus on theory development versus solving practical problems C) The length of the study duration D) The requirement for Institutional Review Board (IRB) approval Answer: B Explanation: Basic research aims to generate new knowledge and theory, whereas applied research seeks to address specific, practical issues. Question 3. Which evaluation theory emphasizes the reduction of programs to measurable components for assessment? A) Social Justice model
Evaluation Ultimate Exam
B) Systemic model C) Reductive model D) Utilization-Focused Evaluation Answer: C Explanation: The Reductive model breaks programs into discrete parts to evaluate each component separately. Question 4. Utilization-Focused Evaluation (UFE) primarily concentrates on: A) Theoretical rigor of evaluation designs B) Ensuring findings are used by intended stakeholders C) Maximizing statistical power in quantitative studies D) Developing complex logic models Answer: B Explanation: UFE is designed to produce evaluations that are directly useful to decision-makers and program stakeholders. Question 5. A logic model is most useful for: A) Testing causal hypotheses in experimental designs B) Visualizing the relationship between resources, activities, and outcomes C) Conducting meta-analyses of published studies D) Calculating effect sizes in regression analysis Answer: B
Evaluation Ultimate Exam
B) A written agreement signed by the participant C) Permission obtained from individuals lacking full decision-making capacity, such as children D) Optional when the study is low risk Answer: C Explanation: Assent is the affirmative agreement of individuals (often minors) who cannot legally give full consent. Question 9. According to the AEA Guiding Principles, an evaluator should: A) Prioritize the sponsor’s interests above all else B) Remain independent and avoid conflicts of interest C] Avoid communicating findings to stakeholders D) Use only quantitative methods to ensure objectivity Answer: B Explanation: The American Evaluation Association emphasizes independence, integrity, and avoidance of conflicts of interest. Question 10. The Data Universal Numbering System (DUNS) is primarily used to: A) Assign unique identifiers to research participants B) Track funding sources for program evaluations C) Provide a unique identifier for organizations in data reporting D) Encode qualitative interview transcripts Answer: C
Evaluation Ultimate Exam
Explanation: DUNS numbers uniquely identify business entities, facilitating data linkage and reporting. Question 11. Cultural competence in evaluation practice includes: A) Using only English-language instruments B) Ignoring cultural differences to maintain objectivity C) Adapting methods to respect participants’ cultural contexts D) Relying solely on Western theoretical frameworks Answer: C Explanation: Cultural competence requires tailoring research designs and instruments to be respectful and appropriate for diverse cultures. Question 12. In a Randomized Controlled Trial (RCT), the primary purpose of random assignment is to: A) Increase external validity B) Ensure equal sample sizes across groups C) Control for confounding variables by equalizing groups at baseline D) Simplify data analysis procedures Answer: C Explanation: Randomization distributes known and unknown confounders evenly, enhancing internal validity.
Evaluation Ultimate Exam
D) The statistical power is low Answer: A Explanation: Non-random sampling limits the ability to generalize findings to a broader population. Question 16. Phenomenological research is best suited for investigating: A) Causal relationships between variables B) The lived experiences and meanings of participants C) Large-scale population trends D) The effectiveness of a new policy Answer: B Explanation: Phenomenology focuses on describing how individuals experience a phenomenon. Question 17. In grounded theory, theoretical saturation is achieved when: A) All participants have been interviewed B) No new concepts emerge from additional data collection C) The researcher reaches a predetermined sample size D) The statistical model reaches a high R-square value Answer: B Explanation: Saturation occurs when further data no longer generate novel categories or insights.
Evaluation Ultimate Exam
Question 18. Which trustworthiness criterion corresponds most closely with the quantitative concept of reliability? A) Credibility B) Transferability C) Dependability D) Confirmability Answer: C Explanation: Dependability reflects the stability of data over time and conditions, analogous to reliability. Question 19. An explanatory sequential mixed-methods design begins with: A) Qualitative data collection followed by quantitative analysis B) Simultaneous collection of both data types C) Quantitative data collection, then qualitative follow-up to explain findings D) Only qualitative data, with quantitative validation later Answer: C Explanation: Explanatory sequential design uses quantitative results first, then qualitative work to clarify them. Question 20. Triangulation in mixed-methods research primarily serves to: A) Increase sample size B) Reduce the cost of data collection
Evaluation Ultimate Exam
Question 23. Content validity is best evaluated by: A) Comparing test scores to an external criterion B) Examining the theoretical relevance of items to the construct C) Having subject-matter experts review items for representativeness D) Conducting factor analysis Answer: C Explanation: Experts assess whether items adequately cover the domain of interest, establishing content validity. Question 24. A Likert-scale item that asks respondents to indicate agreement on a five-point scale is an example of: A) Nominal measurement B) Ordinal measurement C) Interval measurement D) Ratio measurement Answer: B Explanation: Likert responses are ordered categories but the intervals between points are not guaranteed to be equal. Question 25. Which sampling technique is most appropriate when the researcher wants to ensure representation of specific sub-groups (e.g., age, gender) within the sample? A) Simple random sampling B) Convenience sampling
Evaluation Ultimate Exam
C) Stratified random sampling D) Snowball sampling Answer: C Explanation: Stratified sampling divides the population into homogenous strata and samples proportionally from each. Question 26. Snowball sampling is particularly useful for: A) Large, randomly selected populations B) Hard-to-reach or hidden populations C) Ensuring equal representation across demographics D) Reducing sampling error to zero Answer: B Explanation: Participants recruit others, facilitating access to populations that are otherwise difficult to locate. Question 27. Power analysis primarily helps researchers to determine: A) The most appropriate statistical test B) The required sample size to detect a specified effect size with a given alpha level C) The reliability of their measurement instrument D) The level of internal validity of their design Answer: B Explanation: Power analysis calculates the sample size needed to achieve a desired probability of detecting an effect.
Evaluation Ultimate Exam
Answer: B Explanation: Probing prompts participants to provide richer, more detailed information. Question 31. An observational checklist that records the frequency of a specific behavior is an example of: A) Structured observation B) Unstructured observation C) Participant observation D) Ethnographic field notes Answer: A Explanation: Structured observation uses predefined categories and counts occurrences of target behaviors. Question 32. Which descriptive statistic is most appropriate for summarizing a skewed distribution of income? A) Mean B) Median C) Mode D) Standard deviation Answer: B Explanation: The median is less affected by extreme values and better represents central tendency in skewed data. Question 33. The chi-square test is used to examine relationships between:
Evaluation Ultimate Exam
A) Two continuous variables B) One continuous and one categorical variable C) Two categorical variables D) Paired sample means Answer: C Explanation: Chi-square assesses independence or association between categorical variables. Question 34. In a multiple regression model, the coefficient for an independent variable represents: A) The probability that the variable is significant B) The change in the dependent variable for a one-unit change in the predictor, holding other variables constant C) The overall fit of the model D) The correlation between the predictor and the outcome Answer: B Explanation: Regression coefficients quantify the unique contribution of each predictor. Question 35. Cohen’s d is a measure of: A) Statistical power B) Effect size for differences between two means C) Correlation strength D) Model fit in logistic regression Answer: B
Evaluation Ultimate Exam
D) Ethnography Answer: C Explanation: Grounded theory uses constant comparison to refine categories and develop theory. Question 39. CAQDAS software (e.g., NVivo, ATLAS.ti) primarily assists researchers with: A) Performing advanced statistical regression B) Managing and analyzing large bodies of qualitative data C) Conducting randomized controlled trials D) Designing sampling frames Answer: B Explanation: Computer-Assisted Qualitative Data Analysis Software helps organize, code, and retrieve textual or multimedia data. Question 40. Which of the following best describes a formative evaluation? A) Assessment of final outcomes after program completion B) Ongoing assessment of implementation processes to improve the program while it is running C) Economic analysis of program cost-effectiveness D) Evaluation conducted by external auditors only Answer: B Explanation: Formative (process) evaluation provides feedback for program improvement during implementation.
Evaluation Ultimate Exam
Question 41. A needs assessment is typically conducted at which stage of the evaluation lifecycle? A) Prior to program design (pre-implementation) B) After program completion (post-evaluation) C) During the impact evaluation phase D) Only when the program is being discontinued Answer: A Explanation: Needs assessments identify gaps and justify the development of a new program. Question 42. Cost-benefit analysis differs from cost-effectiveness analysis in that: A) Cost-benefit converts outcomes into monetary terms; cost-effectiveness compares costs to a non-monetary outcome metric B) Cost-effectiveness always yields a higher return on investment C) Cost-benefit analysis does not consider program outcomes D) Cost-effectiveness analysis is only used for health interventions Answer: A Explanation: Cost-benefit assigns monetary values to outcomes; cost-effectiveness evaluates cost per unit of effect (e.g., cost per life saved). Question 43. When engaging stakeholders, the primary purpose of a “buy-in” meeting is to:
Evaluation Ultimate Exam
Answer: B Explanation: Gantt charts visually map project schedules, showing start/end dates and overlapping activities. Question 46. The Scope of Work (SOW) in an evaluation contract should explicitly define: A) The evaluator’s personal opinions about the program B) The detailed tasks, deliverables, timelines, and responsibilities of all parties C) The budget for unrelated organizational activities D) The political agenda of the sponsoring agency Answer: B Explanation: A clear SOW outlines what will be done, when, and by whom, preventing misunderstandings. Question 47. Reflective practice for evaluators most directly contributes to: A) Increasing sample size B) Enhancing methodological rigor and professional growth through self-assessment C) Reducing the cost of data collection D) Ensuring statistical significance of results Answer: B Explanation: Reflective practice involves continuous learning and improvement of one’s evaluation skills.
Evaluation Ultimate Exam
Question 48. Which of the following is an example of a non-experimental, descriptive design? A) Randomized Controlled Trial B) Pre-test/Post-test with control group C) Cross-sectional survey describing prevalence of a health condition D) Time-series analysis of an intervention effect Answer: C Explanation: Descriptive cross-sectional surveys collect data at one point to describe characteristics without manipulating variables. Question 49. In a non-equivalent control group design, the main limitation is: A) Inability to measure any outcomes B) Lack of random assignment leading to potential selection bias C) Excessive cost compared to RCTs D) Over-reliance on qualitative data Answer: B Explanation: Without randomization, groups may differ systematically, threatening internal validity. Question 50. The Hawthorne effect refers to: A) Changes in participants’ behavior due to awareness of being observed B) The influence of historical events on study outcomes
